Performance appraisal: what a rating actually measures
Companies reform their appraisal systems regularly, rarely on the basis of what psychometrics has established. The prior question is simple: when a manager assigns a rating, how much of that rating depends on the person being rated?
The founding result
Steven Scullen, Michael Mount and Maynard Goff (2000, Journal of Applied Psychology) decomposed the variance of multi-source ratings: a substantial share reflects rater idiosyncrasy — the rater's own way of scoring — rather than observed performance. The finding does not disqualify appraisal; it sets its precision.
What the recent debate clarified
Seymour Adler and colleagues (2016, Industrial and Organizational Psychology) set out the arguments for and against abolishing ratings. Neither side has robust evidence that abandonment improves performance; what the controversy establishes is that the rating is only one element of a system that includes goals, feedback and administrative decisions.
The nuance that changes the decision
Kevin Murphy (2020, Journal of Organizational Behavior) argues that performance evaluation will not disappear, because organisations must settle pay and promotion decisions. The choice is therefore not between rating and not rating, but between explicit evaluation and implicit evaluation — less traceable and no less subjective. (our executive and employee training programmes)
What follows for a leadership team
If part of the rating belongs to the rater, the priority is not the scale but calibration: confronting managers' judgements on real cases. This reading remains conditional — the data come from varied contexts, and transposing them to a given organisation calls for caution.
A practical implication, verifiable in-house
A check requiring no instrumentation: for a single job level, compare the distribution of ratings given by each manager. A systematic gap between raters signals a calibration issue, not a performance one. This is the analysis we run with Basel leadership teams before changing any appraisal system. To go further: explore the Executive Leadership training in Basel, or browse our executive and employee training programmes in Switzerland.
In pictures: Executive Leadership in Basel
- executive leadership
- Basel
- research
- training basel