AI in quality control: what do we know about unflagged cases?
When AI helps select parts for inspection, the manager is not merely deciding where to focus effort: that decision also shapes the evidence available for subsequent learning. The precise question is this: how can the system be evaluated if unflagged parts are rarely re-examined?
What the literature establishes
Jerker Denrell (2003, Organization Science) shows through theoretical analysis how the underrepresentation of failures in observable experience can distort learning and the evaluation of practices. A practice may appear beneficial because the situations in which it fails are less visible. This work does not address industrial AI, but establishes a problem relevant to its evaluation: observed cases do not necessarily represent all cases processed.
Learning also requires understanding how cases become visible
James March, Lee Sproull and Michal Tamuz (1991, Organization Science) examine opportunities for learning when available experience is sparse and emphasise the role of interpretation. Accumulating events is therefore not enough to determine what an organisation can learn from them. Applied to AI-directed inspection, this reasoning suggests recording why a part was examined, not just the inspection outcome.
The premature conclusion
One might conclude that every part must be inspected in the same way, or that targeted inspection should be abandoned. These studies justify neither conclusion: concentrating resources may remain appropriate, subject to applicable quality requirements. The distinction is between operational targeting of inspections and the sampling needed to evaluate that targeting. (our executive and employee training programmes)
What routine results cannot establish
The defect rate among flagged parts tells us about that selection, not about defects outside it. If only those parts inform corrections to the system, the organisation risks perpetuating selective observation without measuring its consequences. However, neither cited article establishes the magnitude of this risk for a particular AI system or an appropriate sample size for a workshop in Neuchâtel.
A practical check in Neuchâtel
In a microtechnology, watchmaking or precision microelectronics workshop in Neuchâtel, retain mandatory inspections and add a random inspection of parts not flagged by AI over a predefined period, following a protocol approved by the quality function. Record the relevant population, the sampling rule, the number of parts examined and the non-conformities confirmed against the usual criteria. Track the proportion of non-conformities in this sample, together with its uncertainty: observing no defects does not mean zero risk. Within SHR’s « Manager à l'ère de l'intelligence artificielle » programme, this protocol provides a concrete management exercise: organise observation before drawing conclusions about performance. To go further: explore the Managing in the Age of Artificial Intelligence training in Neuchâtel in the canton of Neuchâtel, or browse our executive and employee training programmes in Switzerland.
In pictures: Managing in the Age of Artificial Intelligence in Neuchâtel
- managing in the age of artificial intelligence
- Neuchâtel
- research
- training neuchâtel