Appointing Judges: Can Statistics Help?
The appointment process for judges is somewhat opaque. We may know how many judges there are in Canada, but why some lawyers are chosen over other lawyers remains a bit of a mystery.
Can statistics help us identify who would make a good judge? Can it help us answer a very difficult question?
Over the years, it has been observed that humans have a tendency to substitute a hard question with an easy question. For instance, “the question we face is whether this candidate can succeed. The question we seem to answer is whether she interviews well.” (Thinking Fast and Slow, Daniel Kahneman)
Applying statistics may be of help. In the Undoing Project, Michael Lewis discusses using statistics to identify talent in the context of basketball. He points to Daryl More, the general manager of the Houston Rockets basketball team. Daryl uses a statistically based hiring process over instinct. “Job interviews were magic shows.” Which meant that hopeful recruits charmed interviewers, leading to some poor hires. While people that didn’t look the part, but had the skills, were sometimes overlooked.
To combat this, Daryl More built a statistical model. This model helped predict who would be future stars, and helped avoid people from selecting recruits based on confirmation bias. Or on selecting people based on substituting a hard question with an easier question.
Building this model required measuring many factors, not just how many points someone scored. But also, the score of the game when someone was on the bench. The points and rebounds per minute rather than points by game. The number of genuine opportunities for rebounds someone had over how many he snagged. And so on.
However, there was a limit to the objective data. You still needed human judgement in the hiring process. “Humans sometimes had access to information that the model did not.” For example, models are bad at knowing that someone performed poorly one year because they hated their coach and stopped trying.
Although choosing judges is different than choosing basketball players, statistics may be a helpful tool in predicting which lawyers compared to other lawyers will make great judges.
(Views are my own and do not reflect the views of any organization.)
Agreed! More reliance on stats & objective criteria could produce a more meritorious, and also more diverse bench.
The tricky bit is: which objective criteria should be used? Unlike sports, the practice of law does not produce many measurable and comparable outcomes that would predict merit as a judge. So here’s one idea:
#1. Appoint a large number of part-time judges. Recruit some of them actively, to catch people who could be fabulous judges but are not actively applying and self-promoting for the role.
#2. Track their performance as judges using objective criteria: % of decisions overturned, aggregate satisfaction rates of litigants, timeliness and thoroughness of reasons, number of times observed napping .
#3. Promote the best to full-time permanent positions.
Interesting idea. Raises a question about the effectiveness of requiring application as the only means of recruitment in the first place. I know of some who might make great judges, but for a few good reasons, would never actively seek such a role in this way. The formal application process has seemingly obvious transparancy benefits, but it might have some downsides too, which could also be subject to better empirical analysis.
Very Good Noel Semple
I believe we need to start a permanent record for every Lawyer In Canada . That way all Canadians can see who & why they were pick . Besides all the education they have, plus boards they sit on in their record ,You need to add all complaints made against a Lawyer & why the complaint was made even if the complaint is dismissed . These stats could show/ prove future problems they will have with SRLs & perhaps clients ?