IA on the job: should a prompt library keep only successes?
An organisation deploying AI may be tempted to build a prompt library from its teams’ best results. The management question is precise: on what basis can a prompt be declared reusable if only its successful uses are retained?
What the literature establishes
Jerker Denrell (2003, Organization Science) analyses how the underrepresentation of failures in observed experience can distort learning from others. His theoretical work shows that a practice can appear associated with success because the situations in which it fails remain outside the field of observation. For a prompt library, this calls for a distinction between a successful example and a method whose conditions for success are understood.
The mechanism that reinforces selection
James G. March (1991, Organization Science) models the tension between exploiting available knowledge and exploring new possibilities. The more immediate and predictable benefits of exploitation can lead an organisation to favour established practices at the expense of exploration. Applied to AI by analogy, this mechanism suggests that a highly visible prompt in a library could be reused more often without that frequency demonstrating its superiority.
The premature conclusion
It would be excessive to conclude that organisations should abandon prompt libraries or retain every attempt indiscriminately. A selected example can help people get started and make a practice easier to share. The important distinction is to present it as a resource that works under certain conditions, rather than a recipe validated merely by its inclusion in the catalogue. (our executive and employee training programmes)
What these studies do not establish
These articles address organisational learning, not evaluations of prompt libraries for today’s generative models. They illuminate a selection risk without measuring its magnitude in this context. In practice, a prompt stored without its input data, tool version and acceptance criteria offers little basis for determining whether a difference in results stems from the prompt or its environment.
A practical check in Fribourg
As part of SHR’s “IA on the job” programme, an exercise in Fribourg could bring together teams from the agri-food sector, bilingual industry and the university community to work on fictional cases or cases authorised for sharing. For each prompt proposed for the library, they would define acceptance criteria in advance, then record every trial across distinct cases, including rejected outputs; intended uses in French and German would be examined separately. The measure would be the proportion of accepted outputs among all recorded trials, by task family and language, with access to the cases needed to verify the calculation. This record would not establish general reliability, but it would reveal what a collection of successes alone leaves out. To go further: explore the AI on the Job training in Fribourg, or browse our executive and employee training programmes in Switzerland.
In pictures: AI on the Job in Fribourg
- ai on the job
- Fribourg
- research
- training fribourg