If the Mouse Roared, Then the Court Whimpered

Mark Twain wrote in Mark Twain, “Chapters from My Autobiography”, 598 North American Review (Sept. 7, 1906):

I wrote the rest of “The Innocents Abroad” in sixty days, and I could have added a fortnight’s labor with the pen and gotten along without the letters altogether. I was very young in those days, exceedingly young, marvellously young, younger than I am now, younger than I shall ever be again, by hundreds of years. I worked every night from eleven or twelve until broad day in the morning, and as I did two hundred thousand words in the sixty days, the average was more than three thousand words a day–nothing for Sir Walter Scott, nothing for Louis Stevenson, nothing for plenty of other people, but quite handsome for me. In 1897, when we were living in Tedworth Square, London, and I was writing the book called “Following the Equator” my average was eighteen hundred words a day; here in Florence (1904), my average seems to be fourteen hundred words per sitting of four or five hours.

I was deducing from the above that I have been slowing down steadily in these thirty-six years, but I perceive that my statistics have a defect: three thousand words in the spring of 1868 when I was working seven or eight or nine hours at a sitting has little or no advantage over the sitting of to-day, covering half the time and producing half the output. Figures often beguile me, particularly when I have the arranging of them myself; in which case the remark attributed to Disraeli would often apply with justice and force: “There are three kinds of lies: lies, damned lies, and statistics.”

[Project Gutenberg. Retrieved 2012-12-23.]

Ian Binnie wrote, extra-judicially, while still a judge of the Supreme Court of Canada, in The Honourable Mr. Justice Ian Binnie, “Science in the Courtroom: The Mouse that Roared” (2007) 56 U.N.B.L.J. 307 at 326-27:

Finally, the parties to these complex cases have a legitimate expectation that the reasons for judgment (where the case is tried without a jury) will make it clear the judge has heard and understood the technical aspects of the case. While in Canada, jurors cannot be asked about their deliberations (therefore no “wah wah wah”), a case heard by a judge alone should result in a decision that traces the judge’s pathway through the scientific evidence to the result. In this way, the trial judge establishes a better record for review by the appeal court, and provides assurance to the parties as well as the broader public that the issues have been properly addressed. Appeal courts should be prepared to consider rejection of boilerplate reasons from a trial judge that fail to come to grips with the scientific debate under-lying the issues being litigated.

In Goodman v. Viljoen2012 ONCA 896 aff’g 2011 ONSC 821 the Ontario Court of Appeal affirmed, by a 2-1 majority, a trial judgment in which the trial judge’s finding of factual causation on the balance of probability was explicitly based on her acceptance of the plaintiffs’ experts’ theory of factual causation. That theory was based on a method of analyzing data known as the Bayes Theorem, which is used to produce conditional statistical probabilities. The majority of the Court of Appeal affirmed the decision on the basis that there was nothing about the trial decision that allowed the court to reverse or vary the trial decision.

[142] The standard of review in this case is to defer to the evidentiary findings of the trial judge unless her findings have no basis in the evidence. That is not the case here. In my view, the appellants are asking this court to re-examine all the evidence and to come to different conclusions than those reached by the trial judge – i.e., to retry the case. That is not the role of an appellate court. See Stein v. Kathy K (The), [1976] 2 S.C.R. 802; Housen v. Nikolaisen, 2002 SCC 33, [2002] 2 S.C.R. 235, at paras. 15-18; and H.L. v. Canada (Attorney General), 2005 SCC 25, [2005] 1 S.C.R. 401.

[143] This was a complex medical malpractice case. The causation issue was particularly difficult because, as is not uncommon, there were no specific scientific studies on which the experts or the parties could rely for definitive answers to the specific issues before the court. The administration of steroids prenatally in the case of premature births has had such a significant positive effect, including on the incidence of CP from certain causes, that the accepted standard of care is to administer the steroids in all such cases. However, that does not mean that the failure to do so in any particular case caused the CP in that case.

[144] The trial judge well understood both the deficiencies in the available experimental evidence and the analysis she had to conduct to be satisfied of causation to the required standard.

[145] In my view, the trial judge made no error of law or palpable and overriding error in the apprehension of the evidence, including the expert opinion evidence, or in her findings of fact based on the robust and pragmatic approach.

The trial judge provided some explanation of why she accepted that analysis, summarized in this paragraph:

[203] Dr. Barrett testified to, and I accept his explanation, as to why the Bayesian analysis is appropriate in these circumstances.

The majority must have been satisfied that the judge’s analysis was both consistent with the evidence and legally valid. However, the majority did not discuss the meaning of Bayesian probability analysis theory. You’ll find explanations here, here, and for the mathematically able and not-statistically challenged, here.

The Stanford Encyclopedia of Philosophy has an explanation which does not require a mathematical background.

Bayes’ Theorem is a simple mathematical formula used for calculating conditional probabilities. It figures prominently in subjectivist or Bayesian approaches to epistemology, statistics, and inductive logic. Subjectivists, who maintain that rational belief is governed by the laws of probability, lean heavily on conditional probabilities in their theories of evidence and their models of empirical learning. Bayes’ Theorem is central to these enterprises both because it simplifies the calculation of conditional probabilities and because it clarifies significant features of subjectivist position. Indeed, the Theorem’s central insight — that a hypothesis is confirmed by any body of data that its truth renders probable — is the cornerstone of all subjectivist methodology.


The probability of a hypothesis H conditional on a given body of data E is the ratio of the unconditional probability of the conjunction of the hypothesis with the data to the unconditional probability of the data alone.

In Goodman, the “hypothesis H” of the plaintiffs’ experts was that the defendant’s negligence was a necessary part of the cause of the infants’ cerebral palsy.

There is a bit of an O. Henry twist to the trial judge’s acceptance of the plaintiffs’ theory. The Bayesian analysis did not produce a probability greater than 50% that the negligence was a necessary part of the cause. The probability was only 40%. However, the plaintiff’s experts (Drs. Barrett and Pearlman) used that figure, in addition to their clinical knowledge, as the basis for their conclusion that the defendant’s conduct (which was held to be negligent) was a necessary part of the cause. The trial judge wrote:

[193] Having accepted that the Cochrane data establishes a 40% reduction in CP, the court must address the defendant’s submission that this reduction does not meet the necessary legal burden of proof of more likely than not. Instead, this 40% magnitude of reduction results only in a loss of chance which is not compensable in medical malpractice cases (See: Cottrelle et al v. Gerrard (2003), O.J. No. 4194, (C.A.).

[194] Perhaps if the court were to look at this figure in isolation, this argument may have some merit. However, to do so would be to hold the plaintiffs to a standard of scientific certainty which is not required by law and to ignore the relevant context.

[195] The statistical information is but one piece of the puzzle …

[196] Since the figure of 40% represents an all or nothing proposal, it does not accurately reflect the total impact of ACS on CP. Based on this evidence, it is a logical conclusion that if one were able to measure the total effect of ACS on CP, the statistical measure of that effect would be inflated beyond 40%.

[197] Unfortunately, this common sense conclusion has never and can never be tested by science. As Dr. Perlman testified, such a study would be impossible to conduct because it would require pre-identification of those persons who go on to develop CP. Furthermore, because the short term benefits of ACS are now widely accepted, it would be unethical to withhold steroids to conduct further studies on long term outcomes.

[198] I agree with the plaintiffs’ submission that they cannot be denied recovery because science has not advanced far enough to answer the question to a degree of scientific precision, particularly in the face of a very persuasive trend. …

[199] Although science can never prove this logical inference, it does find support when the Cochrane data is viewed using a Bayesian approach to statistical analysis. The court heard from two experts qualified as bio-statisticians as to the appropriate statistical approach to this data.

[200] Both Dr. Willan and Dr. Platt presented their evidence in a fair and balanced manner. Again, the court is faced with two equally competent experts disagreeing. The real issue of contention between the two gentlemen is the statistical approach used by each. The actual mathematical calculations by either were not at issue. For the reasons that follow, in the end, I prefer the evidence of Dr. Willan.

[202] The Bayesian approach used by Dr. Willan was heartily endorsed by Dr. Barrett. For reasons already mentioned, I found Dr. Barrett to be exceedingly knowledgeable and particularly well placed to give an opinion on the medical issues relevant to this trial and I preferred his evidence to that of the other medical experts. His endorsement of Dr. Willan’s analysis therefore carries some weight.

[203] Dr. Barrett testified to, and I accept his explanation, as to why the Bayesian analysis is appropriate in these circumstances.

[206] This statistical information is of course just one piece of evidence the court must consider in determining the issue of causation. Statistics represent generalizations and not the particular circumstances of a given case. At best, they are numbers which are dependent on the underlying data and the limits of scientific knowledge. Context, human experience and interpretation are required if statistics are to have any real meaning.

[207] Both Drs. Barrett and Perlman, exercising their clinical judgment and expertise, considering the statistical evidence, answered unequivocally that it is more likely than not that had the twins received a full course of steroids, they would not have suffered the injuries they did, or the magnitude of those injuries would have been reduced. I agree with this conclusion. It is supported by the statistical and medical evidence and the common sense inferences that flow therefrom. I therefore find that the plaintiffs have established on a balance of probabilities that the defendant’s negligence caused the CP from which they now suffer.

I have provided a link to the ONCA decision in Cottrelle. Tea leaf readers might want to look at who two of the three members of the ONCA panel were and consider who they are now, if pondering the likelihood that leave to appeal will be granted by the SCC (assuming leave is sought).

The majority reasons contain only one explicit reference to the trial judge’s acceptance of a Bayesian analysis of data.

[130] In my view, the trial judge made no error by accepting Dr. Perlman’s evidence in her consideration of this issue. Dr. Perlman gave his opinion in this case based not only on the Cochrane analysis of 48 cases, but also on his knowledge of the earlier 2004 study and analysis, his knowledge of animal studies showing a relationship between ACS and CP, his expertise and experience, as well as the biological plausibility approach and the Bayesian probability analysis by Dr. Willan. The biological plausibility approach in particular addresses the likelihood that steroids have a maturational effect on all membranes including the brain membranes whose disruption leads to PVL.

There are more references to the Bayes Theory in the dissenting reasons; however, the dissent does not discuss the acceptability, in law, of Bayesian analyses. It may well be that the dissenting judge was prepared to accept it could be valid for the purposes of the dissent. The substance of the dissent was that the plaintiff’s evidence did not connect increased risk to probable causation. The facts of the case are outlined in the dissent. The majority accepted the accuracy of that outline: see para. 110.

[13] The twins had developed CP by the time they were 18 months old. CP is a non-progressive nerve disorder and describes a cluster of conditions that includes motor difficulties, such as spasticity of the limbs, and sometimes includes cognitive problems. Daniel, the first born, is physically disabled but does very well in school. Calvin has more significant medical problems than Daniel. He also has some behavioural problems.

[14] CP develops in the months or even years following birth. CP is sometimes associated with medical problems that occurred during or shortly after birth. For example, some CP is associated with respiratory distress syndrome (“RDS”) experienced by some premature babies during the birth process. RDS can have a direct connection to the subsequent development of CP, or it can lead to a variety of other medical problems that in turn are associated with the later development of CP. For example, some kinds of intraventricular hemorrhaging (“IVH”) are associated with RDS and the later development of CP in premature babies.

[15] In many cases, especially in full-term pregnancies, CP cannot be attributed to any specific event or medical cause. The risk of CP is known to be much higher among premature infants than full-term babies. Premature twins are more susceptible to CP than are single premature babies.

[16] The experts all agreed that the twins’ CP was caused by a condition known as diffuse periventricular leukomalacia (“PVL”). PVL is among the many problems that premature babies face in making the adjustment to life outside of the womb.

[17] PVL involves the inadequate blood supply to an area of the premature baby’s brain referred to as the watershed zone. Arterial blood supplies to the brain meet in the watershed zone. In premature infants, the arterial membranes may not develop fully. The brain cells affected by PVL are unstable and vulnerable. The expert evidence suggested that the damage to the affected areas of the brain caused by PVL occurs during delivery or in the first few days following birth.

[18] Babies born before 34 weeks gestation are more prone to PVL than other babies. PVL is the most common cause of brain injuries suffered by premature babies.

[19] PVL is associated with the later onset of CP. The descending nerve tracts to the legs and arms pass through the area of the brain adversely affected by PVL.

[20] PVL may be caused by hypoxia, that is reduced oxygen in the blood flow circulating to the affected area of the brain, or ischemia, that is a deficiency in the blood supply to the affected area of the brain due to reduced blood pressure. Either hypoxia or ischemia results in the death of brain cells in the affected area of the brain. The twins’ PVL was likely caused by ischemia.

[98] In my view, the absence of any evidence capable of quantifying the reduced risk of PVL flowing from a course of ACS precluded a finding that the failure to administer ACS caused the twins’ PVL which in turn caused their CP. That was the primary basis on which the trial judge found causation.

[99] The trial judge did address the quantification of the reduction of the risk of developing CP in the context of her alternative finding of causation based on her determination that the data in the Cochrane analysis established that the risk of CP would be reduced regardless of the specific medical cause of the CP. That analysis cannot remedy the trial judge’s failure to address the quantification of the risk in the context of her finding that the failure to administer a course of ACS caused the twins’ PVL. Nor can it salvage her finding that the CP was caused by the failure to administer ACS regardless of its immediate medical cause. I have already explained, at paras. 63-68, why in my view that finding is unsupported by the evidence. For the sake of completeness, however, I will address this aspect of her reasons.

[100] The trial judge accepted that the Cochrane analysis data established that a course of ACS would reduce the risk of CP in premature babies by 40 per cent. The trial judge also recognized that a 40 per cent reduction in the risk would not establish legal causation. She then went on to hold that the Cochrane analysis data did not identify those cases in which babies who had developed CP, even though their mothers had received a course of ACS, had developed significantly less severe disabilities than those babies who had developed CP and whose mothers had not received a course of ACS. In the trial judge’s assessment, this category of babies who were less seriously handicapped was sufficiently large to inflate the effect of the impact of ACS on CP beyond the 40 per cent reduction in CP. According to the trial judge, many babies who developed CP would be less seriously handicapped because of the administration of ACS to the mother prenatally. The cumulative effect of the two categories, that is babies who did not suffer CP and babies whose disabilities were significantly lessened, was sufficiently large to reach the required legal threshold of a balance of probabilities.

[101] The Cochrane analysis in fact provided no evidence that a course of ACS had any effect on the severity of the symptoms of those babies who developed CP. The Cochrane analysis did not describe the level of disability of any of the 48 babies referred to in the studies it considered. Clearly, the Cochrane analysis made no attempt to compare the level of disability among the 28 babies whose mothers had not received ACS with the level of disabilities among the 20 babies who had developed CP despite the fact that their mothers had received a course of ACS.

[102] Although the Cochrane analysis data itself could offer no support for the assertion that a course of ACS would reduce the severity of the disabilities suffered by babies who developed CP, the plaintiffs’ experts, applying I think the “biological plausibility” explanation, had testified that it only made common sense that if a course of ACS eliminated CP in 40 per cent of the cases, it would reduce the severity of the disabilities flowing from CP in some unspecified number of other cases. The plaintiffs’ experts acknowledged that the reduction in the severity of symptoms associated with CP in babies who had received a course of ACS had never been studied much less reported in the scientific literature.

[103] Although the evidence to support the reduced severity contention is far from strong, I think it was open to the trial judge to accept that evidence. In my view, however, it did not get the plaintiffs over their causation difficulties.

[104] The trial judge began her causation analysis by accepting that the Cochrane analysis data showed a 40 per cent reduction in the occurrence of CP when the mothers had received a course of ACS. She then placed on top of that 40 per cent an additional unquantified number of cases in which the severity of the disabilities suffered by the babies had been materially reduced. The two combined met the balance of probabilities standard.

[105] For reasons set out above, the Cochrane analysis data cannot reasonably support a finding that a course of ACS would reduce by 40 per cent the risk that a premature baby would develop CP regardless of the underlying medical cause of the CP. The data goes no further than to suggest that a course of ACS may reduce the risk of CP in cases other than those in which a connection has been shown between the underlying medical cause of CP and a course of ACS. The quantification of that reduction in premature babies at 40 per cent, the trial judge’s starting point in her causation calculus, has no basis in the evidence.

[106] Putting the causation case at its best for the plaintiffs, there was evidence suggesting that a course of ACS would reduce the risk that premature babies would develop CP in an unspecified number of cases. There was also evidence that in an unspecified number of cases, a course of ACS may have a positive effect on the seriousness of the disabilities suffered by babies who did develop CP. Placing these two unquantifiable amounts together could not demonstrate the requisite causal connection on the balance of probabilities. Pushing the evidence to the edge of its reasonableness limits does not provide a basis upon which the trial judge could reasonably find that it was more likely than not that the twins would not have suffered CP or that their disabilities associated with CP would have been significantly reduced had their mother received a full course of ACS prenatally.

One would never know it from either level of decision in Goodman, or indeed from the content of any other reported Canadian decision, but most courts in the U.K, Australia, and the U.S. have held that Bayes Theorem – regardless of its validity in the world of science – is not appropriate in the court room. The issue usually arises (when it does) in criminal law cases. (Simon Fodden has reminded me that he mentioned one of the U.K. cases, recently, here, and wrote a piece on Bayes Theorem even earlier. He provided the link.)

The issue is not whether Bayes Theorem is junk science. It is not. The trial judge briefly dealt with this issue, without referring to case law.

[125] Dr. Willan described different statistical approaches and in particular, the frequentist or classical approach and the Bayesian approach which differ in their respective definitions of probability. Simply, the classical approach allows you to test the hypothesis that there is no difference between the treatment and a placebo. Assuming that there is no difference, [that] allows one to make statements about the probability that the results are not due to chance alone. To reach statistical significance, a standard of 95% is required. A new treatment will not be adopted into practice unless there is less than a 5% chance that the results are due to chance alone (rather than due to true treatment effect).

[126] Dr. Willan explained the results of the Cochrane meta-analysis in frequentist terms using the CP analysis as an example (Exhibit 10, page 341). The risk ratio (RR) is calculated by taking the probability of CP in the treatment group divided by the probability of CP in the placebo group. A risk ratio that is equal to one means that there is no treatment effect at all – in other words, the placebo and the treatment have the same effect.

[127] In the case of CP, the RR is 0.60 or a 40% risk reduction in CP where there has been administration of antenatal corticosteroids. The upper limit of the confidence level just crosses one at 1.03, which means that the analysis just fails to reach the level of statistical significance in order to justify a treatment policy change. …

[128] [Dr Willan] testified that it is now impossible to undertake further clinical trials because ethically, such studies would never be approved as a standard of care is already in practice for the use of steroids. Because no further trials are possible, there is strong evidence that the use of steroids is beneficial on a long-term basis. The short-term data already confirms this. In order to determine the probability that the risk of CP is reduced, one must use the Bayesian method which uses a different definition of probability. It is an expression of the degree of belief about the unknown.

[129] Dr. Willan acknowledged that in clinical trials the standard is to use the classical or frequentist approach. However, the Bayesian approach is a highly acceptable, reliable, statistical method. His work using the Bayesian approach has been peer reviewed. In his opinion, the Bayesian approach is appropriate in the circumstances of this case, to make a probability statement about whether the risk of developing CP is reduced with the use of steroids.

[135] In cross-examination, Dr. Willan acknowledged that the classical approach is the standard used for the purpose of treatment. He further acknowledged that there have been no studies involving the impact of steroids using the Bayesian approach.

[138] Dr. Willan agreed that the best estimate of the magnitude of risk reduction is 40% in the case of CP using the frequentist approach. The doctor noted that it would be very rare in clinical practice and research to find a new treatment that would ever half the risk of an outcome. It has only happened to him once in his lengthy career.

[139] Dr. Willan did not calculate the magnitude of the reduction in risk using the Bayesian method. However, he agreed that the 70% increase in risk represents the same 40% risk reduction.

[140] Dr. Willan further acknowledged that he did not review the studies underlying the Cochrane analysis and could offer no assistance on the quality of these papers. He did not apply his statistical analysis to the clinical factors, but agreed that such an exercise should be undertaken by the court to consider a particular case.

[146] When questioned about Dr. Willan’s Bayesian analysis and his theory that ACS probably reduces the risk of CP, Professor Platt opined that the quality of the underlying data does not support the plaintiffs’ conclusions. However, Dr. Platt did acknowledge that the Bayesian approach is a well-known and perfectly valid statistical method.

[147] Dr. Platt also rejected Dr. Barrett’s assertion that an increase in sample size would achieve statistical significance

A search for any reported mention of the Bayes Theorem in Canadian jurisprudence (on CanLII and Westlaw Canada’s Canadian court decision databases) produces only one hit: Goodman. Extending the search to Westlaw Canada’s articles database produces 18 more hits, most of which are articles which discuss, in various ways, the asserted conflict (or at least tension) between the assumptions that underlie the Bayesian analysis and those which underlie judicial fact finding, whether by judge or by jury.

Professor Russell Brown, of the University of Alberta Law School, has succinctly explained why in one of the articles available through the Westlaw Canada search: Brown, “The Possibility of ‘Inference Causation’: Inferring Cause-In-Fact And The Nature Of Legal Fact-Finding” (2010) 55 McGill L. J. 1 at 27ff [Brown, “Inference Causation”]. Brown provides, at 27-28, a useful general explanation of how the Bayesian analysis is performed.

In essence, Bayes’s Theory furnishes a mechanism for incrementally revising probability estimates in light of new information, thereby allowing a fact-finder to update continually an opinion about the relative likelihood of a fact. This requires two pieces of data, the first being an a priori estimate of the probability that a fact is proven or not. Bayesians posit that a fact-finder begins with an original estimate of the likelihood of a fact, such as a causal link. That prior probability is then continually revised to reflect the statistical impact of new, relevant information as it is received and incorporated into the probabilistic calculation of the likelihood of the fact. In Bayesian terms, the statistical impact of new information constitutes the second piece of data, being a likelihood ratio, which is multiplied by the prior probability to create the posterior possibility. The process is repeated until finally, the fact-finder supposedly arrives at the final modification of the probability of the causal linkage.

Brown then explains at 28-30 why the premises of the Bayesian analysis are “incompatible with legal fact-finding”:

Bayesian methodology suffers from several defects, however, making it incompatible with legal fact-finding. The principal objection challenges prior probability: it is in essence a reference class, since it forms the a priori basis from which the probability of a proposition such as a causal link is made. Everything that follows —the continual updates of the prior estimate in light of discovered bits of evidence relevant to cause-in-fact, and the final modification of the assessment of the proposition’s likelihood — depends upon that starting point. Unless, however, that preexisting probability is truly a priori — that is, an analytic proposition that derives from logic and reason, as opposed to a synthetic proposition based on experience and observations — there is no reason to accept that pre-existing probability’s status within Bayesian methodology is inherently reliable. … Given the scientific fact-finder’s claim to formal objectivity, the Bayesian process is ironically totally dependent upon the intuition that probabilists eschew as fuzzy and unreliable, and which statistical evidence ostensibly allows fact-finders to avoid.

Another objection to statistical evidence and to its processing in litigation in a Bayesian manner derives from the empirical limitations of human computational capacity. The notion of a fact-finder who continually and mathematically refines accumulating data is simply implausible. The nature of evidence itself, moreover, defies easy Bayesian statistical reduction. Take again the problem of soft variables in evidence. Bayesian analysis cannot account for the significance to be ascribed to the meaning of a witness’s smile in describing a possible causal sequence because both the factual question of whether it was more a smirk than a smile, and the contrasting hypotheses (derision or humour) pose difficulties and perhaps impossibilities for statistical reduction. …

[At 28-29, internal footnotes omitted.]

Brown’s article provides references to the literature on the issue of the compatibility or incompatibility of Bayesian analysis and evidence theory. There are proponents of compatibility. The literature outside of Canada, particularly in the U.S., is extensive. There’s useful Australian literature too that’s easily found by a search on AUSTLII, even a very recent piece arguing that Bayesian analyses are consistent with the premises of legal fact-finding: James Franklin, “The Objective Bayersian Conceptualisation of Proof and Reference Class Problems” (2011) 33 Sydney Law Review 545. (The link will get you to the site to download the paper.) This is the abstract.

The objective Bayesian view of proof (or logical probability, or evidential support) is explained and defended: that the relation of evidence to hypothesis (in legal trials, science etc) is a strictly logical one, comparable to deductive logic. This view is distinguished from the thesis, which had some popularity in law in the 1980s, that legal evidence ought to be evaluated using numerical probabilities and formulas. While numbers are not always useful, a central role is played in uncertain reasoning by the ‘proportional syllogism’, or argument from frequencies, such as ‘nearly all aeroplane flights arrive safely, so my flight is very likely to arrive safely’. Such arguments raise the ‘problem of the reference class’, arising from the fact that an individual case may be a member of many different classes in which frequencies differ. For example, if 15 per cent of swans are black and 60 per cent of fauna in the zoo is black, what should I think about the likelihood of a swan in the zoo being black? The nature of the problem is explained, and legal cases where it arises are given. It is explained how recent work in data mining on the relevance of features for prediction provides a solution to the reference class problem.

As mentioned, in Goodman, neither the trial judge, nor the majority, nor the dissent, discussed whether the Bayesian methodology was legally valid, regardless of its statistical validity. The majority, though, has to have accepted that the Bayesian analysis was legally valid as well as scientifically valid in order to validly affirm the trial verdict. I leave it to others to decide whether Goodman deals, adequately, with the former issue, particularly if we take the comments of then Justice Binnie as expressing what the views of the Supreme Court of Canada would be if asked to address the question.

Retweet information »

Comments

  1. I should clarify a point that I (perhaps) didn’t make clear enough in the posting. I attempted to do so in the last paragraph.

    I am not dealing with the issue of whether the trier of fact (judge or jury) should employ some form of Bayesian analysis in deciding whether it is probable or not that some thing occurred in the past. That is not what happened in Goodman. The trial judge did not purport to apply, herself, some form of Bayesian analysis of the data.

    Rather, she accepted an expert opinion on the probability of some fact where the opinion was based on a Bayesian analysis of the data. The issue I have raised – which is why I led with the quotations from Twain and Binnie – is whether the trial reasons, and appellate reasons, adequately explain why an opinion based on a Bayesian analysis of the data was admissible at all. The majority reasons stand or fall with the trial reasons, because the majority didn’t provide an independent explanation.

    I’m going to repeat a portion of the passage I quoted from ex-SCC Justice Binnie’s article:

    … a case heard by a judge alone should result in a decision that traces the judge’s pathway through the scientific evidence to the result. … Appeal courts should be prepared to consider rejection of boilerplate reasons from a trial judge that fail to come to grips with the scientific debate under-lying the issues being litigated.

    It is true that the trial judge explained, in some detail, what it was the plaintiffs’ experts said the Bayesian analysis of the data (the statistics) meant and how the result compared to traditional methods of statistical analysis. However, I saw nothing in the trial reasons when I first read them, and still see nothing – for whatever that is worth in the scheme of things – that convinces me that the trial judge understood the principles of Bayesian methodology enough to know why there is tension, if not in conflict, between that methodology and the methodology of fact finding in law.

    Bayes Theory is not “junk science”. The trial judge adverted to that issue when she mentioned that the work of the expert who provided the Bayesian analysis of the data had been been peer reviewed. But the fact that it is not junk science – the fact that it is an accepted methodology in science – does not, of itself, mean that there is no conflict between Bayesian methodology and the principles that law uses in determining what is admissible evidence and the weight to be given that evidence. Professor Brown highlighted that problem in the excerpt that I quoted. I’ll repeat the key passage.

    Everything that follows —the continual updates of the prior estimate in light of discovered bits of evidence relevant to cause-in-fact, and the final modification of the assessment of the proposition’s likelihood — depends upon that starting point. Unless, however, that preexisting probability is truly a priori — that is, an analytic proposition that derives from logic and reason, as opposed to a synthetic proposition based on experience and observations — there is no reason to accept that pre-existing probability’s status within Bayesian methodology is inherently reliable.

    I don’t see anything in the trial judge’s reasons that satisfies me that the “prior probability” – the pre-existing probability – upon which the plaintiff’s theory of “biological plausibility” relied was inherently reliable. That is because there is no discussion of that issue, at all, in the reasons. (Perhaps it is hinted at when the judge explained why additional testing was no longer permitted, on ethical grounds.) Perhaps that issue was canvassed in the evidence. But we don’t know that. The majority appellate reasons don’t indicate whether it was.

    The trial judge may accurately set out what it was the experts said, but that does not mean the trial judge understood that evidence. At the end of the day, then, without something in the trial reasons that shows that the trial judge understood the principles upon which Bayesian methodology is based, the trial reasons are inadequate. If the trial reasons are, then so are the majority reasons in the ONCA. We are not dealing with a principle of law which allows us to apply the presumption of regularity to the trial or majority reasons.

    It is open to a trial judge to reject or accept admissible expert evidence on valid grounds. But what we don’t see, I suggest, in the Goodman trial reasons is why the Bayesian analysis was legally valid in the circumstances [just] because the trial judge accepted that it was medically valid, preferring the evidence of the plaintiffs’ experts over that of the defendant’s experts. Medical validity – here, statistical validity – doesn’t entail legal validity. What we see, I think, is the judge presuming the Bayesian analysis was legally valid as a matter of evidence theory. Perhaps that is because of the way the case was presented. I don’t know. Make what you will of this paragraph in the trial reasons:

    [200] Both Dr. Willan and Dr. Platt presented their evidence in a fair and balanced manner. Again, the court is faced with two equally competent experts disagreeing. The real issue of contention between the two gentlemen is the statistical approach used by each. The actual mathematical calculations by either were not at issue. For the reasons that follow, in the end, I prefer the evidence of Dr. Willan.

    I have added the emphasis. That paragraph could be read to mean that the defence did not challenge the basic admissibility of the opinion – did not raise the issue of whether a Bayesian analyisis is compatible with law’s principles of evidence – but only its weight.

    I’m going to quote, again, from the trial reasons where, I think, we see where the judge made the jump from the argument about medical validity to the assumption of legal validity. That will put para. 200 in context.

    [77] Dr. Barrett, a maternal fetal medicine specialist, was of the opinion that the twins’ outcome would have been substantially better if they had had the benefit of a full course of corticosteroids.

    [78] Dr. Barrett testified that it is widely accepted and supported by the scientific literature, that corticosteroids will decrease the severity of any RDS, chronic lung disease and the possibility of brain hemorrhage. He went on to testify that it was logical that if the short-term outcomes are much improved with steroid use that one would expect the long-term outcomes such as CP to be improved.

    [79] Dr. Barrett testified that the Cochrane meta-analysis is regarded as the best source of information and the best evidence that is available with respect to this issue. When he initially prepared his report in 2004, the Cochrane meta-analysis was not available and his opinion took into account the findings from the Crawley meta-analysis.

    [80] As a clinician, he looks at all available data to see how close it is to reaching statistical significance. The Crawley data reached about 87% versus the required 95%. A purist might disregard this information, but Dr. Barrett did not feel that approach was appropriate.

    [81] The Cochrane meta-analysis determined that the data on the use of steroids and CP almost reached statistical significance, which he felt was remarkable considering the still rather small sample size. In his opinion, one would need a large sample size of 3,000 to reach statistical significance.

    [82] Using the Cochrane data, Dr. Barrett’s opinion was that one must then consider the issue of biological plausibility. In other words, does the data make sense? He testified that we know ACS have multiple effects on short-term outcomes. We know corticosteroids stabilize all membranes and it is those membranes in the brain which break and can cause CP. He concluded that on the basis of this probability, the outcome of these babies would have been significantly better had they had the steroids.

    [83] Dr. Barrett had the benefit of reading the reports of Dr. Willan and hearing him testify. He was very excited about Dr. Willan’s Bayesian analysis because this approach allowed him to statistically prove what he says intrinsically makes sense.

    [84] In his view, the null hypothesis is not appropriate with steroids. The null hypothesis is used when one is trying to determine if a treatment makes a difference or not. With steroids, we already know they make a difference, so to use the frequentist statistical analysis does not make sense. He opined that you have to use something else which takes into account what you already know and that is the Bayesian approach.

    [85] Dr. Barrett’s opinion was not undermined during his lengthy cross-examination. He acknowledged that the scientific standard demands 95% certainty. He is, however, prepared to accept a lower degree of certainty. To him, 85 % is not merely a chance outcome.

    [86] The doctor admitted that the Bayesian analysis used by Professor Willan is not found anywhere else in the literature. He also acknowledged that there is good reason why science requires a 95% probability. He went on to explain that because of the short term effects of steroids, no one will withhold their use and, therefore, we are unable to have any new studies to look at the long term effects of no steroid use.

    [87] He acknowledged that scientific evidence in support of the use of corticosteroids has never shown statistical significance with respect to CP. However, he explained it is very close at 93.5%. He cautioned that if you use a black and white outlook and ignore the obvious trends, you will falsely come to the conclusion that there is no effect.

    [88] The doctor would not accept that there is no basis to conclude that there is a causal connection between steroid use and CP. Instead, he said it is so close to statistical significance that he would use it anyway. He is convinced that there is a difference and it will not change his management.

    [128] The professor testified that it is now impossible to undertake further clinical trials because ethically, such studies would never be approved as a standard of care is already in practice for the use of steroids. Because no further trials are possible, there is strong evidence that the use of steroids is beneficial on a long-term basis. The short-term data already confirms this. In order to determine the probability that the risk of CP is reduced, one must use the Bayesian method which uses a different definition of probability. It is an expression of the degree of belief about the unknown.

    [129] Dr. Willan acknowledged that in clinical trials the standard is to use the classical or frequentist approach. However, the Bayesian approach is a highly acceptable, reliable, statistical method. His work using the Bayesian approach has been peer reviewed. In his opinion, the Bayesian approach is appropriate in the circumstances of this case, to make a probability statement about whether the risk of developing CP is reduced with the use of steroids.

    [146] When questioned about Dr. Willan’s Bayesian anaylis and his theory that ACS probably reduces the risk of CP, Professor Platt opined that the quality of the underlying data does not support the plaintiffs’ conclusions. However, Dr. Platt did acknowledge that the Bayesian approach is a well-known and perfectly valid statistical method.

    [176] Obviously, the decision of the court on this issue is heavily dependent on the expert evidence and opinions expressed during the course of the trial. As is not unusual in medical malpractice cases, the court is faced with eminently qualified doctors who disagree as to the cause of the plaintiffs’ injuries. Here, Drs. Perlman and Barrett were adamant that the failure of Mrs. Goodman to receive ACS directly affected the health of the twins. Drs. Hellmann and Farine were just as insistent that no such connection can be made.

    [177] On the totality of the evidence before the court, I prefer the opinions of Drs. Perlman and Barrett to that of Drs. Farine and Hellmann. I come to this conclusion for several reasons.

    [187] The defendant is correct in stating that none of the literature filed specifically found a connection between the use of ACS and PVL. However, using the robust and pragmatic approach to fact finding, logical inferences may be drawn from the many recognized affects of ACS and their effect on other organs and systems in humans and animals. This, in combination with the clinical judgment and experiences of Drs. Perlman and Barrett, leads me to conclude that the plaintiffs have made the case for causation.

    [188] On the totality of the evidence before me, it is reasonable to infer that since PVL results from the immaturity of the pre-term infant’s brain and vascular system, and ACS have a maturational effect beyond lung function to mature these systems, that it is more likely than not that the administration of ACS would reduce the risk of PVL. Thus, but for the defendant’s negligence, the twins would not have suffered from PVL, and consequently would not have suffered from CP.

    [189] Even if I am incorrect in my conclusion that the PVL (and therefore the CP) would not have occurred but for the failure to receive a full course of ACS, this is not fatal to the plaintiffs’ claim. I am satisfied that the evidence before me establishes on a balance of probabilities that ACS reduce the risk and severity of CP in general, that is, no matter what the cause of the CP. In other words, I am satisfied that the twins would not have developed CP (or the CP would not have been as severe) but for the failure to receive a full course of ACS brought about by the defendant’s negligence.

    [190] The scientific basis for this finding comes in large part from the Cochrane analysis. …

    [193] Having accepted that the Cochrane data establishes a 40% reduction in CP, the court must address the defendant’s submission that this reduction does not meet the necessary legal burden of proof of more likely than not. Instead, this 40% magnitude of reduction results only in a loss of chance which is not compensable in medical malpractice cases

    [196] Since the figure of 40% represents an all or nothing proposal, it does not accurately reflect the total impact of ACS on CP. Based on this evidence, it is a logical conclusion that if one were able to measure the total effect of ACS on CP, the statistical measure of that effect would be inflated beyond 40%.

    [197] Unfortunately, this common sense conclusion has never and can never be tested by science. As Dr. Perlman testified, such a study would be impossible to conduct because it would require pre-identification of those persons who go on to develop CP. Furthermore, because the short term benefits of ACS are now widely accepted, it would be unethical to withhold steroids to conduct further studies on long term outcomes.

    [198] I agree with the plaintiffs’ submission that they cannot be denied recovery because science has not advanced far enough to answer the question to a degree of scientific precision, particularly in the face of a very persuasive trend. The fact that science will never be able to prove this relationship, in my view, makes this precisely the type of situation, the Supreme Court of Canada was referring to in Laferriere v. Lawson

    [199] Although science can never prove this logical inference, it does find support when the Cochrane data is viewed using a Bayesian approach to statistical analysis. The court heard from two experts qualified as bio-statisticians as to the appropriate statistical approach to this data.

    [200] Both Dr. Willan and Dr. Platt presented their evidence in a fair and balanced manner. Again, the court is faced with two equally competent experts disagreeing. The real issue of contention between the two gentlemen is the statistical approach used by each. The actual mathematical calculations by either were not at issue. For the reasons that follow, in the end, I prefer the evidence of Dr. Willan.

    [201] It was Dr. Willan’s evidence that an analysis of the Cochrane data using the Bayesian method, demonstrates a 91 to 97% probability that the risk of CP is reduced with the administration of ACS. Dr. Willan also considered the increase in risk of CP resulting from a failure to give steroids. Using a random effects model and based on the randomized clinical trials data only, Dr. Willan testified that the increase in risk of CP where ACS are not given is in the order of 70%.

    [202] The Bayesian approach used by Dr. Willan was heartily endorsed by Dr. Barrett. For reasons already mentioned, I found Dr. Barrett to be exceedingly knowledgeable and particularly well placed to give an opinion on the medical issues relevant to this trial and I preferred his evidence to that of the other medical experts. His endorsement of Dr. Willan’s analysis therefore carries some weight.

    [203] Dr. Barrett testified to, and I accept his explanation, as to why the Bayesian analysis is appropriate in these circumstances.

    [204] As well, other than by Dr. Farine, the Cochrane collaboration meta-analysis was described as the best available evidence with respect to this issue, yet, Dr. Platt suggested that the underlying data of the Cochrane analysis is poor and unreliable. It is of note that this criticism was testified to for the first time at trial.

    [205] It was very telling that some of the criticisms Dr. Platt levied at Dr. Willan were evident in his own research publications without any words of caution.

    [206] This statistical information is of course just one piece of evidence the court must consider in determining the issue of causation. Statistics represent generalizations and not the particular circumstances of a given case. At best, they are numbers which are dependent on the underlying data and the limits of scientific knowledge. Context, human experience and interpretation are required if statistics are to have any real meaning.

    [207] Both Drs. Barrett and Perlman, exercising their clinical judgment and expertise, considering the statistical evidence, answered unequivocally that it is more likely than not that had the twins received a full course of steroids, they would not have suffered the injuries they did, or the magnitude of those injuries would have been reduced. I agree with this conclusion. It is supported by the statistical and medical evidence and the common sense inferences that flow therefrom. I therefore find that the plaintiffs have established on a balance of probabilities that the defendant’s negligence caused the CP from which they now suffer.

    I have added the emphasis

    DC

  2. I will finish by adding my standard complaint about judicial claims that “common sense” somehow provides proof and justification for the claim that the evidence shows a valid in law causal connection between wrong and injury. (This apart from the fact that the judge used the phrase because the Supreme Court has said that common sense is the underlying principle by which the but-for test is to be applied, hence the judge needs to at least mention that mantra.)

    Let’s look at how the trial judge referred to “common sense” other than when quoting or paraphrasing case law. That occurs in para. 12.

    Bear in mind that what we’re considering is the significance of evidence whose relevance has been explained to the fact-finder (here a judge alone) by an expert, who was permitted to testify and whose opinion is relevant only because the significance of that evidence is not something a judge (or jury) is assumed to understand without that opinion. So, when a judge asserts that common sense (whatever the judge means by common sense – the term is not explained: nor was it in Goodman) supports the expert’s opinion of a causal connection, aren’t we left to wonder how the judge would know that? If, however, the expert says it’s common sense, then the expert means common sense based on the expert’s knowledge, which has to be the proverbial horse of a different colour. (But not a zebra.)

    [74] Dr. Perlman’s opinion was not shaken in his lengthy cross-examination. Dr. Perlman was firm in his opinion that common sense and clinical experience tell him that CP is not a condition that is either present or absent, but rather is on a continuum. Common sense leads to the conclusion that for those cases where CP is not prevented in its entirety, it is likely that it is less severe for those who do go on to develop it.

    [196] Since the figure of 40% represents an all or nothing proposal, it does not accurately reflect the total impact of ACS on CP. Based on this evidence, it is a logical conclusion that if one were able to measure the total effect of ACS on CP, the statistical measure of that effect would be inflated beyond 40%.

    [197] Unfortunately, this common sense conclusion has never and can never be tested by science. As Dr. Perlman testified, such a study would be impossible to conduct because it would require pre-identification of those persons who go on to develop CP. Furthermore, because the short term benefits of ACS are now widely accepted, it would be unethical to withhold steroids to conduct further studies on long term outcomes.

    I’ve added the emphasis. Remember the point that Professor Brown made about the validity of the Bayesian method depending on the assumption of the validity of the prior probability. What does the sentence in bold tell us?

    [207] Both Drs. Barrett and Perlman, exercising their clinical judgment and expertise, considering the statistical evidence, answered unequivocally that it is more likely than not that had the twins received a full course of steroids, they would not have suffered the injuries they did, or the magnitude of those injuries would have been reduced. I agree with this conclusion. It is supported by the statistical and medical evidence and the common sense inferences that flow therefrom. I therefore find that the plaintiffs have established on a balance of probabilities that the defendant’s negligence caused the CP from which they now suffer.

    [210] This is once again an argument that is persuasive on its face, but loses its appeal when viewed in context. It was the uncontested testimony of Dr. Barrett that 80% of CP in all gestations have no associated infection, RDS or IVH (these being conditions upon which ACS have a recognized effect). This being the case, it defies common sense that the 40% reduction of CP in its entirety (shown in the Cochrane analysis) would be attributable solely to the aforementioned causes, which account for only a small percentage of all CP cases. As such, and notwithstanding my conclusion that the PVL would not have occurred but for the defendant’s negligence, I have little difficulty finding that proof of that conclusion is not required for the plaintiffs to succeed.

    All emphasis added by me.

    What does the trial judge’s use of “common sense” add in any of these paragraphs to the question of whether the evidence established a causal connection, on the balance of probability, between the negligence and the injury?. If its only the judge’s adjective describing her view of the validity of the expert’s opinion, it adds nothing. If it is something more, what is that something more? Bear in mind that, when the trial judge quoted case law, one of the passages she quoted (at para. 12) was from an Ontario Court of Appeal decision – Fisher v. Victoria Hospital 2008 ONCA 759 at para. 59 where the ONCA reiterated that “common sense cannot become a substitute for resort to the evidence. … The mere application of “common sense” cannot conjure up a proper basis for inferring that an injury must have been caused in one way rather than another.” [Internal quotation marks omitted.]

    So, let’s go back to the lines in paras. 207 and 210 that seem to summarize the trial judge’s use of “common sense”.

    [207] … It is supported by the statistical and medical evidence and the common sense inferences that flow therefrom. …

    We are entitled to ask, about para. 207, what “common sense inferences” if these are anything other than what the expert(s) said. If they are only common sense conclusions by the experts then “common sense” adds nothing, not the least because these conclusions aren’t common sense.

    [210] … it defies common sense …

    This seems to be the judge’s common sense. If it is, then is it not using “common sense” as a substitute for evidence? Is it not using “common sense” to “conjure up” a basis for finding a causal connection?

    After all, the plaintiffs had to resort to the non-traditional Bayesian analysis precisely because the traditional methods of statistical analysis did not produce the answer that the plaintiffs needed to succeed if the but-for test was applied.

    Why did the plaintiffs have to use the but-for test? Because the trial judge ruled that the effect of Resurfice Corp v. Hanke 2007 SCC 7 was that the Athey material contribution test (whatever it now meant as a result of Resurfice: my interpolation – DC) was not applicable. Presciently, the trial judge, after quoting the usual paragraphs from Athey, wrote, anticipating the Clements two-tortfeasor requirement.

    [174] In my view, the case at bar does not meet the requirements in Hanke to allow resort to the material contribution test. The present scenario involves the tortious act of one individual, Dr. Viljoen. While there may be other non tortious “causes” of the injuries to the twins, there is no proverbial second shooter, nor is there an intervening third party. Likewise, although there is arguably a lack of scientific evidence as to the cause of PVL, this lack of scientific evidence does not produce the situation of “circular causation” with regard to the tortious causes that would be required to open the door to the material contribution test.

    [175] Accordingly, in order to succeed, the plaintiffs must prove, on a balance of probabilities, that but for Dr. Viljoen’s negligence, the Goodman twins would not have suffered the injuries they did or the magnitude of those injuries would have been reduced.

    As I have written, elsewhere, it defies reality – some might say common sense – to suggest that the trial judge would NOT have applied Athey material contribution to injury as the test before Resurfice.

    I’ll end with another oddity in the trial reasons. Recall that the posting which starts this thread had a quotation from the discussion in Prof. Russell Brown’s “Inference Causation” of the incompatibility between Bayesian methodology and legal fact finding. The trial judge in Goodman cited Professor Brown’s article – but only on the issue of the scope of the material contribution test after Resurfice.

    [173] Since the release of Hanke in 2007, the courts have taken a very restrictive approach to the availability of the material contribution test (See: Russell Brown, “The Possibility of ‘Inference Causation’: Inferring Cause-in-Fact and the Nature of Legal Fact-Finding” 2010 McGill L.J. 1-45). Frankel J.A. set out that, “It is only in exceptional circumstances that resort to the material-contribution test will, as a matter of policy, be appropriate” (Clements, supra at para. 48).

    The reference to Clements is to the British Columbia Court of Appeal decision.

    For those who care, what I expect will be my last – for a very long time – significant (well … I think it is significant: opinions may vary – I thought “Snark” was significant, too) contribution to the literature on the meaning of factual causation in negligence, in Canadian law, is now available on SSRN. Its partial title is “Black Holes, Aether, and Negligence in the Air”. I uploaded a revised version on December 25, 2012. Wait until that appears. The “Aether” is not a misspelling, a typo, or a conceit.

    As some of you know, the Supreme Court of Canada has yet another case with causation issues under reserve: Ediger v. Johnston, docket no. 34408, appeal argued Dec 4, 2012. There is only one tortfeasor in Ediger so I don’t expect the Court to revisit material contribution principle. While circumstances might change, what the Court might do with (or to) but-for principle I plan to leave to others.

  3. I still wonder why this was not argued as a Snell case- does the impossibility of proving causation from either side take it out of that paradigm?

  4. Brian:

    I assume what you mean is “why wasn’t the case argued under Snell without reference to the Bayes Theorem?” since the trial judge applied the robust & pragmatic approach – see paras. 70-77, 121-122 of the ONCA reasons.

    Reading between the lines of the trial reasons, I suspect the Goodmans’ lawyer made that argument too – that Snell supported the inference even without the use of a Bayesian analysis of the statistics, but the trial judge didn’t mention it. if you look at the trial reasons, you’ll see both Drs. Barrett and Perlman testified for the plaintiffs on causation. The trial judge doesn’t mention Bayes in setting out Perlman’s opinion, summarized in paras. 68-69, 76

    [68] Dr. Perlman advised that the conclusion of the study was that developmental delay was reduced by slightly more than half. With respect to CP, the study showed a “trend” that CP was reduced by 40%. The term “trend” indicates that the finding did not reach statistical significance (95%), although it came extremely close (93.5%). The study did not deal with the extent of the injury and Dr. Perlman testified that one would assume that even if CP was not avoided in its entirety, it would be less severe.

    [69] Dr. Perlman opined that if Mrs. Goodman had received an earlier administration of corticosteroids, it would be far more likely than not that the disabilities the twins suffer would have been prevented or less severe.

    [76] Taking all factors into account and weighing all the information and evidence available to him, Dr. Perlman reiterated his opinion that it is more likely than not that the twins would have seen a material advantage had they received the optimal course of steroids.

    Bayes first appears in the context of Dr. Barrett’s evidence.

    [83] Dr. Barrett had the benefit of reading the reports of Dr. Willan and hearing him testify. He was very excited about Dr. Willan’s Bayesian analysis because this approach allowed him to statistically prove what he says intrinsically makes sense.

    What seems clear enough from the trial reasons, as written, is that the trial judge wasn’t prepared to hold that the evidence was enough, in total, even applying the robust & pragmatic approach, without the interpretation of the statistical evidence provided by the Bayesian analysis.

    My reading of the ONCA reasons is that the Goodmans’ case was argued in the alternative there, too. The ONCA majority reasons, however, make it seem that the Bayes analysis was also part of the basis of Dr. Perlman’s evidence (see paras. 130-131.)

    David

  5. Brian:

    Just in case I misread your comment:

    does the impossibility of proving causation from either side take it out of that [Snell] paradigm

    If it was impossible to prove or disprove factual causation on the balance of probability, using the but-for test – since after Clements that seemingly all we have – then the action should have been dismissed. There’d have been a gap in the evidence from possibility to probability but no way of validly leaping it, and no alternative method of satisfiying the causation requirement since Clements says explicity that the material contribution test requires two tortfeasors pointing their fingers – you choose the digit- at each other.

    (Actually, if you choose the right digit they’d be giving the finger to the plaintiff, but that’s a different image.)

    David

  6. Bernard Robertson

    There are a number of points that could be made in response to this comment. I will restrict myself to the main points. There is substantial literature on this issue and it is disappointing that the author chooses to quote only one misconceived article.

    1. The author refers inconsistently to Bayes Theorem and Bayes Theory. It is a theorem, it is logically true.
    2. It is unclear how legal and scientific decision making can differ. The question is what inferences can be drawn from evidence in conditions of uncertainty. Whatever method is adopted must comply with the requirements of logic. These do not vary according to context.
    3. No one piece of evidence can prove a proposition, it can only make the proposition more or less probable than it would be without the evidence. The measure of how much it does so is determined by the probability that the evidence would be found if the proposition is true divided by the probability that the evidence would be found if some other proposition is true. This is called the Likelihood Ratio and is one component of Bayes Theorem.
    4. I cannot find a statutory definition of “relevance” in Canada but in R. v. Cloutier 1979 CanLII 25 (SCC), (1979), 48 C.C.C. (2d) 1 (S.C.C.) at 27 it is said that relevance “requires a determination whether, as a matter of human experience and logic, the existence of a particular fact, directly or indirectly, makes the existence or non-existence of a material fact more probable than it would be otherwise”. This corresponds with point 3 above.
    5. The point of the Bayesian approach to evidence is precisely that the expert does NOT assume a prior probability or a posterior probability, those are matters for the tribunal of fact. The witness should state only the Likelihood Ratio, the amount by which the evidence should logically alter the assessment of probability of the proposition.
    6. One example of past bad practice was the “probability of paternity”. Witnesses used in many jurisdictions, and still do in some, to give this probability. This involved multiplying the “paternity index” which is a Likelihood Ratio, by a prior probability arbitrarily chosen by the witnesses of 50:50. This form of evidence involved assuming a prior probability and assuming an alternative proposition. The latter should depend on the arguments in the case and in particular on the defence theory. Thus the pre-Bayesian methods of evidence were logically invalid and usurped the role of the tribunal of fact, but courts accepted them and lawyers and judges express great nostalgia for them.
    7. Pre-Bayesian methods also led to strange judgments in which judges said things such as that legal and mathematical probability were different, as they could not work out how to combine the evidence with the other evidence in the case. There is no logical way of doing so.
    8. The judgments in the UK and elsewhere which have criticised Bayesian reasoning have been subjected to vigorous criticism in the literature. They are riddled with basic errors and misunderstandings. This is not surprising as they consist of judges issuing their more or less untutored views. So far as I know, there has never been an appellate case in which counsel gave prepared argument on the subject.
    9. If Bayesian reasoning is to be rejected, it is incumbent on the critic to suggest some other coherent system which presents logical criteria for analysing decisions. The author does not attempt to do so.

  7. Mr. Robertson,

    1. You are right. The use of Bayes “theory” was sloppiness on my part.

    2. With respect, if you are who your name and comment suggests you are, you know exactly how legal and scientific decision making do differ, particularly if you were referring to the truth of the decision as opposed to the process of decision-making. And, there is a difference in the latter, too. Ultimately, since in law “ought” gets to trump “is”, there may always be a difference both definition of “truth” and the definition of validity.

    3. Law gets to define the minimum requirements for legal truth. In that sense, law is like religion, not science. See 2.

    4. There is none. Relevance means whatever the particular legal system defines it to mean. See 2.

    5. It was not my intent to argue the merits of the Bayes Theorem. There isn’t enough in the reasons (at least for me) to form an opinion as to whether the Bayesian analysis was done validly – in the procedural sense- by the expert who testified. My premise in my posting was that it was but that the defence position was that the data the expert relied on was insufficient for the valid application of Bayes Theorem, even if the calculations were performed properly on the data that existed. In addition, my point was that one cannot tell from the reasons (or at least I can’t) what the evidence was upon which the trial judge decided what the prior probability was (beyond accepting whatever was in the report that the particular expert delivered.) Again, the evidence might have been adduced at trial. But, it wasn’t in the reasons.

    Also, I point out, I attempted to distinguish between the trier of fact”s use of Bayesian analysis to decide questions of probability – the point you address (about which there is extensive literature as you mention) – and the trier of fact’s acceptance of the probability of the occurrence (or non-occurrence) of some fact, which the trier then uses in a so-called “common sense” conclusion as to the truth of some proposition.

    6. I don’t dispute your point that what passes for logical analysis in law – whether statistical or otherwise- often isn’t at all. I’d quote the concluding (about law and logic) from Lord Halsbury’s Quinn v. Leathem aphorism; however, you likely know it as well as I do and I’d be misusing it.

    7. True. But a valid Bayesian analysis presumes the decision-maker has taken into account all of the evidence that could affect the prior probability. It’s not open to the decision maker to reject the evidence because allowing it to be considered would bring the field of science to which the data relates into disrepute.

    8. Bear in mind in Canada we still use the jury system for both criminal and civil trials so one would never know how, in cases tried with a jury, whether the jury understood or not even if there is a finding consistent with a Bayesian analsyis of the data; nor, for that matter, why the jury chose to accept or not accept any bit of evidence. We do not use the U.S. system of long detailed questions to the jury which might provide a hint.

    9. If, by the author, you mean me, you are right that I did not attempt to do so since that wasn’t the point about which I was writing; nor would I have attempt what you propose on this site even were I inclined to such an endeavour.

    I’ll make 2 points in conclusion. You wrote, in your first paragraph, “There is substantial literature on this issue and it is disappointing that the author chooses to quote only one misconceived article”.

    1. Indeed there is, but “this issue” is the appropriateness of judicial trier of fact use of Bayesian analyse to make the probability decision. I mentioned there was extensive literature. I didn’t see any need to quote it given the purpose of my postings (which I attempted to clarify) in my additions to the initial posting.

    2. I quoted from two articles.

    Kind regards,

    David Cheifetz

  8. Serendipitously (since I was writing about legal decision-making in Canadian common law jurisdictions), a recent Alberta appellate decision emphasizes the point that legal truths are what the law holds that truth to be, not what the truth might be in reality. It does not matter that the court was interpreting the meaning of a causation in workers’ compensation legislation. The court accepted this explanation of the meaning of the but-for test in Canadian common law.

    The “but for” test is not about discovering what factor really caused the accident, in the real world. Nor is it about what really happened factually to bring about the turn of events which resulted in injury. The only purpose the test serves is to determine the link between the at-fault conduct of the defendant and the plaintiff’s alleged harm caused by that at-fault defendant.

    The court said elsewhere, too, the correct answers to the causation question also depend on the policy of the the legislation.

    DC

  9. A recent, neutral, wry, overview piece discussing Bayesian analysis and its use in law is here: http://ssrn.com/abstract=2020052

  10. At any moment in the life of a dispute which is before the court, it does not matter what commentators think the correct answer ought to be. What matters is what the current law is. Mr. Robertson’s second point is:

    2. It is unclear how legal and scientific decision making can differ. The question is what inferences can be drawn from evidence in conditions of uncertainty. Whatever method is adopted must comply with the requirements of logic. These do not vary according to context.

    [My emphasis.]

    Now consider this passage from David Goldstein, “How Science Works” in Federal Judicial Center, Reference Manual on Scientific Evidence, 3d ed. (Washington D.C.: The National Academies Press, 2011) at 53.

    Beyond the meanings of certain key words, science and the law differ fundamentally in their objectives. The objective of the law is justice; that of science is truth. These are among the highest goals to which humans can aspire, but they are not the same thing. Justice, of course, also seeks truth, but it requires that clear decisions be made in a reasonable and limited period of time. In the scientific search for truth there are no time limits and no point at which a final decision must be made.

    If Mr. Goldstein is correct, then there is no requirement that legal and scientific decision making be the same.

    Russell Brown (the author of the article deprecated by Mr. Robertson) has written, elsewhere:

    These distinct legal and scientific fact-finding thresholds are not as significant, however, as are the distinct methodologies that underlie them. In a situation where we know what has happened but do not know why it has happened, science responds by repeating experiments until the event can be replicated. The emphasis here, then, is upon continual refinement of what we know by eliminating anomalies. Until we reach scientific fact-finding’s confidence interval, causal determinations must always remain contingent or provisional. Legal fact-finders, however, do not replicate events in this way. For one thing, as a practical matter law needs finality, not an ongoing process of ongoing refinement of knowledge. Indeed, such further investigation may be futile. The very notion that evidence might be ‘complete’ is only a ‘scientific ideal’. Because factual uncertainty derives from the unavailable of evidence which by necessity we do not know, nobody can predict what would be proven by the unavailable evidence, were it available. There are, furthermore, simply too many variables, many of them subjective, for which legal fact-finders would have to account.

    see Russell Brown, “Inferring Cause-in-Fact and the Search for Legal ‘Truth’”, in Richard Goldberg, ed., Perspectives on Causation (Oxford: Hart Publishing, 2011) 93 at 97. [Internal footnotes omitted.]

    And, the former Mr. Justice Ian Binne of the Supreme Court of Canada wrote, in the article I quoted from initially:

    Superficially, and misleadingly, the cultures of the lawyer and the scientist seem to have much in common. We all try to act on “evidence” and use comparable indicia of expertise and testing in the search for reliability. Conclusions are drawn from the accumulated evidence through logic, deduction and induction. Yet in truth, we largely inhabit mutually skeptical solitudes, and where these solitudes rub together the result is often not pleasurable for either discipline, especially for the judge. For instance, the idea of “evidence” seems to mean something quite different to scientists than to judges. In science, “evidence” is not equivalent to “proof” For the most part, theories cannot truly be “proved” in the sense that they become indisputable. Almost all scientific conclusions are considered provisional. Theories can be supported by more or less evidence and may eventually come to be accepted as the dominant or most likely explanation of observed phenomena. The phrase “consistent with” to a scientist merely means that something cannot be excluded as an explanation. It does not connote any particular weight, association or commonality. However, to judges, lawyers, and the public this phrase is often interpreted to mean that there is a “match”.

    Binnie, “Science in the Courtroom”, at 308-09 [internal footnotes omitted].

    In any event, there is ample authority for the position that judges, in Canada and in the United States, at least, believe that there is something inherently different as between legal and scientific decision making. In the trial process, the judge has the last word.

    DC