Custom AI assistants: what studies really measure
Can an AI assistant built by a non-developer really save time? Available studies suggest so — if we look at which tasks they measure.
An often-cited result
Sida Peng, Eirini Kalliamvakou, Peter Cihon and Mert Demirer (2023, arXiv preprint) find in a controlled experiment that assisted developers complete a defined programming task faster. It is a preprint, on one bounded task.
Structural limits of models
Emily Bender, Timnit Gebru and co-authors (2021, Proceedings of FAccT) recall that language models produce plausible text without guaranteed understanding. The paper is argumentative and urges caution about reliability.
What this means for an assistant
Measured gains concern tasks with clear scope and verifiable output. A do-everything assistant falls outside that frame. (our executive and employee training programmes)
The limit of common practice
Building an assistant 'for the whole department' scatters effort and makes evaluation impossible.
A practical check in Basel
Pick one recurring task, time ten occurrences without and ten with the assistant, including verification time. The net balance, not the impression, decides what comes next. To go further: explore the Building AI Assistants and Agents Without Coding training in Basel, or browse our executive and employee training programmes in Switzerland.
In pictures: Building AI Assistants and Agents Without Coding in Basel
- building ai assistants and agents without coding
- Basel
- research
- training basel