Comparing options: what separate evaluation can conceal
A committee may review supplier proposals individually, then bring them together in a comparison table and find that its ranking changes. The management question is specific: should options be evaluated separately or jointly to give relevant criteria their appropriate weight?
What the literature establishes
Christopher K. Hsee (1996, Organizational Behavior and Human Decision Processes) demonstrated experimentally that preferences can reverse between separate and joint evaluations of the same options. His evaluability hypothesis explains this phenomenon through the difficulty of assessing certain attributes without a point of comparison. A criterion may therefore receive little weight not because it matters little, but because its value is difficult to interpret in isolation.
The mechanism: comparison makes a criterion interpretable
Christopher K. Hsee, George F. Loewenstein, Sally Blount and Max H. Bazerman (1999, Psychological Bulletin) provide a review and theoretical analysis of these preference reversals. Joint evaluation makes certain differences easier to interpret when they convey little useful information in separate evaluation. In technical procurement, for example, a performance measure that is hard to assess may gain weight once a competitor provides a reference point; this is a management illustration, not a sector-specific finding from these studies.
The premature conclusion
One might conclude that options should always be compared side by side. These studies do not establish that this method guarantees better decisions: making a difference more visible does not prove that it matters for actual use. The comparison table must therefore remain subordinate to needs and acceptance thresholds defined before reviewing the proposals. (our executive and employee training programmes)
What the evidence does not settle
Reversals observed in experimental tasks do not directly measure the success of a purchase, partnership or organisational investment. In practice, expertise, regulatory criteria and previous reference points may already make an attribute interpretable without immediate comparison. A change in ranking is therefore a signal to investigate, not proof that either the first or the second judgement is wrong.
A practical check in Lausanne
Within SHR’s decision-making programme, an exercise applicable in Lausanne involves examining an actual supplier or partner selection, whether in a medtech company, a higher education institution, an international sports organisation or a Lake Geneva region scale-up. Before the group discussion, each evaluator scores the same proposals separately against criteria set in advance, then reassesses them side by side without additional information. The measure is the proportion of evaluators whose preferred option changes, accompanied by the criterion they identify as decisive at each stage. This check makes sensitivity to the evaluation mode observable, without establishing on its own which ranking is better or isolating a possible order effect. To go further: explore the Decision-making training in Lausanne, or browse our executive and employee training programmes in Switzerland.
In pictures: Decision-making in Lausanne
- decision-making
- Lausanne
- research
- training lausanne