Probability, Nike Trainers, and Murder: English Court Case Bars Bayes and Raises Ruckus

Last year at about this time the Court of Appeal for England and Wales overturned the conviction of one T (R. v. T. [2010] EWCA Crim 2439) on the charge of murder. It was an identification issue and a critical element in linking the defendant to the crime had to do with a shoeprint left at the scene. The judgment is heavily redacted, reminiscent of some grudging government compliance with a FOI order; nevertheless, it’s clear that shoes found at the defendant’s house had soles that matched the patterns found at the scene of the crime. The important questions so far as evidence was concerned had to do with how common the particular shoe was and, thus, the likelihood that the mark was made by the defendant and not by another person.

To put it shortly, the expert presenting this evidence did it badly, and the data he used to form his estimate of probability were less than precise. No commentator has suggested that the Court of Appeal’s conclusion in the instant case was wrong. However, the Court went further and anathematized the use of Bayes’ Theorem, the mathematical tool commonly used by scientists to calculate “likelihood ratios”. Paragraph 90 contains this fatal phrase:

It is quite clear therefore that outside the field of DNA (and possibly other areas where there is a firm statistical base), this court has made it clear that Bayes theorem and likelihood ratios should not be used.

As the recent story in The Guardian relates, barring Bayes has led to a strong reaction from the scientific and mathematics communities and the formation of a group of more than 60 members aimed at explicating and defending the use of probabilistic reasoning in the criminal courts. As they say on their main page:

Proper use of probabilistic reasoning has the potential to improve the efficiency, transparency and fairness of criminal trials by enabling the relevance of evidence – especially forensic evidence – to be meaningfully evaluated and communicated. If more widely and effectively used, it could lead to fewer cases being revisited by the Court of Appeal.

The problem is that probabilistic reasoning isn’t easy and isn’t always (or even usually) intuitive. It is, however, at the core of all trials, given that certainty based on evidence is rarely possible; the law must content itself with probabilities. Bayes’ Theorem allows us to give probability some greater accuracy in certain cases, cases where information can be translated into numerical data. Particularly, it allows us to combine probabilities in a rational way: this has a certain likelihood; that has a certain likelihood; how likely is a combination of this and that. Or, to put it better perhaps, what is the degree of confidence that something is true?

There’s a formula, of course:

But this is unlikely to help most lawyers who may be less than entirely comfortable with mathematics. Yet, anyone who deals in proof, and particularly proof in criminal cases, should become adept at using Bayes. In which case, I recommend Eliezer S. Yudkowsky’s “An Intuitive Explanation of Bayes’ Theorem — Bayes’ Theorem for the curious and bewildered; an excruciatingly gentle introduction.”

He uses as an example the sort of statistics we all confront in the daily papers, data concerning the incidence of cancer in certain groups of people, leading us to combining probabilities in the correct way with understanding.