IA on the job: does success with assistance demonstrate competence?
In SHR — Swiss Human Resources’ “IA on the job” programme, the distinction between an achieved result and an acquired competence deserves particular attention. The management question is specific: what assessment should underpin competence validation when an assistant contributes to success?
What the literature establishes
Nicholas C. Soderstrom and Robert A. Bjork (2015, Perspectives on Psychological Science) distinguish observable performance during learning from the lasting changes that constitute learning itself. Their review shows that conditions supporting immediate success do not necessarily support retention or transfer. Applied to AI, this distinction cautions against treating a successful AI-assisted deliverable as sufficient evidence of lasting competence.
What a recall test reveals
Henry L. Roediger III and Jeffrey D. Karpicke (2006, Psychological Science) compared repeated study of texts with conditions involving recall tests. In their experiments, the advantage depends on the assessment delay: repeated study can support immediate performance, whereas recall tests support delayed retention. Ease experienced during an exercise and capability available later are therefore not interchangeable indicators.
The premature conclusion
One might conclude that every validation assessment must exclude AI. That would confuse two different objectives: performing a task with permitted resources and personally drawing on the knowledge needed to check the result. Management should specify what it is validating, rather than make an unaided assessment the universal measure of competence. (our executive and employee training programmes)
What these studies do not establish
These publications do not test current generative assistants in Fribourg businesses. A text-recall task reproduces neither the handling of a production deviation nor the coordination of a bilingual team. The studies justify distinguishing assessment methods, but establish neither how much work should be done without AI nor a universal success threshold.
A practical check in Fribourg
In Fribourg, SHR could propose a two-stage assessment to a bilingual food-processing or industrial company, potentially with methodological support from the university sector, using fictional production deviations: an AI-assisted solution followed, one week later, by a comparable case requiring participants to identify without AI the elements that call for escalation. Before the exercise, subject-matter leads would define the expected elements and have the French and German versions checked for equivalence, without grading language quality. The measure would be the proportion of participants who passed the assisted exercise and subsequently identified all predefined critical elements, with the corresponding participant counts retained. This result would not measure competence in its entirety, but would make the gap between assisted success and personally available essential knowledge verifiable. To go further: explore the AI on the Job training in Fribourg in the canton of Fribourg, or browse our executive and employee training programmes in Switzerland.
In pictures: AI on the Job in Fribourg
- ai on the job
- Fribourg
- research
- training fribourg