AI adoption: what still needs checking?
In a team adopting AI, asking people to “always check” sounds prudent, but leaves the work required for that verification undefined. The managerial question is more precise: which outputs should be checked, by what procedure, and how can we tell whether those checks remain effective?
What the literature establishes
John D. Lee and Katrina A. See (2004, Human Factors) review research on trust in automated systems. Their analysis distinguishes trust from the actual decision to rely on a system and emphasises the importance of calibrating trust to its real capabilities. For team managers, the objective is therefore not to maximise trust, but to make explicit the conditions under which a suggestion can be accepted.
The mechanism: delegation can also weaken monitoring
Raja Parasuraman and Victor Riley (1997, Human Factors) examine appropriate and inappropriate uses of automation, including overreliance and underuse. Their review describes how excessive trust can lead to insufficient monitoring of automated outputs. Applied cautiously to AI, this mechanism suggests distinguishing acceptance of a suggestion from verification of the evidence supporting it.
The premature conclusion
One might conclude that mandatory human approval is enough to make use safe. Yet a signature demonstrates neither that a check took place nor that the person had the information needed to detect an error. The task is to define what must be compared with an independent source, rather than adding formal approval to every output. (our executive and employee training programmes)
The limits of the evidence and of general instructions
Both articles review automation research predating today’s generative AI tools: they do not directly measure these tools’ reliability in precision industries. They therefore establish neither a universal checking rate nor an expected productivity gain. The instruction to “check the answers” is itself incomplete unless it specifies criteria, accessible sources and the time devoted to checking.
A practical check in Neuchâtel
As part of the programme “Conduire l’adoption de l’IA dans son équipe”, an exercise in Neuchâtel could use non-conformity summaries drawn from closed case files authorised for this purpose, in microtechnology, watchmaking or precision microelectronics, without affecting production. Using the same set of suggested outputs prepared for the exercise, containing errors known only to the evaluator, compare a general proofreading instruction with a protocol specifying the references to check and the authoritative documents. For each approach, measure the proportion of inserted errors actually detected and the checking time per case. This local test would not certify the tool, but would establish whether the protocol improves detection and at what cost in staff time. To go further: explore the Leading AI Adoption in Your Team training in Neuchâtel, or browse our executive and employee training programmes in Switzerland.
In pictures: Leading AI Adoption in Your Team in Neuchâtel
- leading ai adoption in your team
- Neuchâtel
- research
- training neuchâtel