The appointment process for judges is somewhat opaque. We may know how many judges there are in Canada, but why some lawyers are chosen over other lawyers remains a bit of a mystery.
Can statistics help us identify who would make a good judge? Can it help us answer a very difficult question?
Over the years, it has been observed that humans have a tendency to substitute a hard question with an easy question. For instance, “the question we face is whether this candidate can succeed. The question we seem to answer is whether she interviews well.” (Thinking Fast and Slow, Daniel Kahneman)
Applying statistics may be of help. In the Undoing Project, Michael Lewis discusses using statistics to identify talent in the context of basketball. He points to Daryl More, the general manager of the Houston Rockets basketball team. Daryl uses a statistically based hiring process over instinct. “Job interviews were magic shows.” Which meant that hopeful recruits charmed interviewers, leading to some poor hires. While people that didn’t look the part, but had the skills, were sometimes overlooked.
To combat this, Daryl More built a statistical model. This model helped predict who would be future stars, and helped avoid people from selecting recruits based on confirmation bias. Or on selecting people based on substituting a hard question with an easier question.
Building this model required measuring many factors, not just how many points someone scored. But also, the score of the game when someone was on the bench. The points and rebounds per minute rather than points by game. The number of genuine opportunities for rebounds someone had over how many he snagged. And so on.
However, there was a limit to the objective data. You still needed human judgement in the hiring process. “Humans sometimes had access to information that the model did not.” For example, models are bad at knowing that someone performed poorly one year because they hated their coach and stopped trying.
Although choosing judges is different than choosing basketball players, statistics may be a helpful tool in predicting which lawyers compared to other lawyers will make great judges.
(Views are my own and do not reflect the views of any organization.)