AI agent hallucinations: designing for verification
Is an AI agent that errs rarely but confidently more dangerous than one that errs often? Research on trust in automation sheds light.
A recognised phenomenon
Ziwei Ji and co-authors (2023, ACM Computing Surveys) review work on 'hallucinations' in text generation: content unsupported by sources or factually wrong. No method fully eliminates them.
Calibrated trust
John Lee and Katrina See (2004, Human Factors) propose appropriate trust: it should match the system's real capabilities. Their framework is conceptual, grounded in prior human-factors studies.
What this implies
A convincing agent hides its errors better than a clumsy one. Fluency is not an indicator of reliability. (our executive and employee training programmes)
The limit of common practice
No-code assistants often display answers without sources, leaving users no quick way to check.
A practical check in Lugano
Require the agent to cite the internal document behind each factual claim, then audit twenty random answers. The share of correctly sourced claims is a more useful reliability measure than satisfaction. To go further: explore the Building AI Assistants and Agents Without Coding training in Lugano, or browse our executive and employee training programmes in Switzerland.
In pictures: Building AI Assistants and Agents Without Coding in Lugano
- building ai assistants and agents without coding
- Lugano
- research
- training lugano