AI on the job: is a precise forecast a reliable forecast?
In Basel's pharmaceutical, life sciences and Rhine logistics activities, forecasts help coordinate interdependent commitments. The management question is specific: how can we check whether a probability produced with AI deserves to guide an operational decision?
What the literature establishes
Don A. Moore and Paul J. Healy (2008, Psychological Review) distinguish three forms of overconfidence: overestimating one's performance, placing oneself too favourably relative to others, and assigning excessive precision to one's beliefs. These phenomena are distinct and do not necessarily move together. For an AI-assisted forecast, this distinction encourages scrutiny of the stated uncertainty rather than merely whether the outcome seems plausible.
The format of uncertainty matters too
Gerd Gigerenzer and Ulrich Hoffrage (1995, Psychological Review) show that Bayesian reasoning problems are solved more successfully when information is presented as natural frequencies rather than probabilities under the conditions studied. The format helps people understand relationships between cases, without itself providing new data. For management, this suggests asking which observed events support a probability, rather than simply requesting a more accessible reformulation.
The premature conclusion
One might conclude that it is enough to ask the assistant for its confidence level and translate it into a frequency. These studies do not show that a model's stated confidence corresponds to its actual success rate. A frequency expressed without a reference population or observations remains an assertion, not a validation. (our executive and employee training programmes)
What the evidence cannot guarantee
These studies concern human judgement, not the calibration of generative assistants in today's industrial processes. Applying them to AI is therefore a working hypothesis to be tested against observable outcomes. In international R&D, rare events, long time horizons and protocol changes can undermine comparisons, even when forecasts are systematically recorded.
A practical check in Bâle
Within SHR — Swiss Human Resources' “IA on the job” programme, a Basel team could select a recurring, non-critical event: missing an agreed sample receipt date for an international life sciences project at the interface between pharmaceuticals and Rhine logistics. Before each deadline, it would record the probability emerging from the AI-assisted process and actually adopted by the team, the deadline and a stable definition of delay, without changing operational controls. At the end of a cycle defined in advance, it would compare the average forecast probability with the observed proportion of delays within groups of similar probabilities, reporting each group's size and including every case whose deadline had passed. The gap is verifiable; if cases are too few or insufficiently comparable, the result remains exploratory and does not justify automating the decision. To go further: explore the AI on the Job training in Basel, or browse our executive and employee training programmes in Switzerland.
In pictures: AI on the Job in Basel
- ai on the job
- Basel
- research
- training basel