CanLII Citation Analysis Available

According to a study commissioned by CanLII and released today reported cases have varying “life spans” and cease to be important — as measured by their citation in other judgments — somewhere between three and fifteen years. The exception to this are judgments of the Supreme Court of Canada, the average “time to failure” of which is a whopping fifty years.

Citation Analysis of Canadian Case Law by Thom Neale is a full-on informatics study that:

uses simple statistical and functional analysis in conjunction with network analysis algorithms to examine the network of Canadian caselaw using data supplied by the Canadian Legal Information Institute (CanLII). Seeking to explore three basic questions, the study describes the database coverage of CanLII along with that of two commercial vendors and juxtaposes that information with the number of citations to cases decided by courts within each province each year. The study then uses analysis of time-series network rankings for each case to determine 1) the age at which cases in the network typically cease to be important, and 2) what characteristics define those cases that continue to be important despite the passage of time.

canlii_study

Although, as stated above, the average life span of all SCC judgments is fifty years, currently (and perhaps at any given point in time?) only about 19% of all their judgments are still “important” as it were, “where importance is defined as a positively trending pattern of citation for a period of at least 15 years.”

The life spans of decisions of the various courts of appeal vary quite considerably, ranging from just under six years for the Ontario Court of Appeal to twelve years for the British Columbia Court of Appeal and sixteen years for the Northwest Territories Court of Appeal. What is at work here, it seems, is the volume of cases generated and the rate of change in the jurisdictions legislation. For similar reasons, I suspect, a mere 3% of BCCA, 1.6% of ABCA, and 1.5% of all ONCA cases are currently “important.”

CanLII’s brief note on the study says, in part, “The study does not propose a firm target, a clear path or a definitive preference in favour of completeness or tailored approaches.”

Interestingly, the study has an appendix listing the “Top 100 Cases That Continue to be Cited Over Time,” which, as I understand it, is not quite the same things as the hundred most cited cases. Unfortunately the cases are listed using a CanLII identifier (e.g. 1987canlii67) that doesn’t tell you which court gave the judgment and that can’t be simply plugged into a URL structure to call up the case itself. I’d like to see CanLII convert the list to a set of usable hyperlinks.

Retweet information »

Comments

  1. I think the low rank of the Ontario Court of Appeal in the life span chart at 10.2 in the report can be explained, at least in part, by the large number of decisions released as brief endorsements.

  2. I’d hope the life span of an SCC case is longer than that of a lower court appellate decision. If it weren’t, that would suggest a bit of turmoil at the top.

    I also hope that the margin of error, or the size of the samples, is such that the not-infrequent judicial citation of the wrong case (or an older case) just because that’s the case the judge regularly uses isn’t an issue.

    I suspect that some of us are curious about how close quantitative importance tracks qualitative. I’m going to play with the cases listed in section 13.1. I’m hoping that a pdf to text conversion will work adequately enough that I can then use CanLII’s hypertext linking tool on the cites. Failing that, I’ll ask CanLII for an html version of 13.1.

  3. Simon, David –

    A couple comments on the top 100 cases listed in section 13.1 of the report:

    - to read the list in rank order, work your way through column 1, then column 2 and then column 3
    – in accordance with the methodology outlined in section 11.1, the list is limited to cases that presently trend positive AND have been around for at least 15 years. Consequently, many cases that would otherwise figure among the most significant (e.g., Dunsmuir, R. v. W.(D.), [1991] 1 SCR 742) are not present

    In support of CanLII’s upcoming hackathon, the study author Thom Neale has developed an interactive website that will show all the fun statistical details about these cases (including style of cause) as well as cool graphs that show their influence over time and in comparison to other or groups of other cases. A few more tweaks before that site is made public, but certainly by middle of next week at the latest.

    Cheers

    Colin

  4. Colin,

    Thanks. You don’t, by chance, already have a hyperlinked version of the chart? Or will the website have it?

    David

  5. Hi David,

    The site will have all the hyperlinks. From this conversation it’s clear we should make sure a linked chart is also made available. (Suddenly, I want one too!)

    Colin

  6. As mentioned last week, study author Thom Neale has developed an interactive website that shows all the fun statistical details about the top cases (including style of cause). That site is now live:

    cite-fight.com

    Click cases from the lists provided or enter your own examples to generate cool graphs that show the influence of each case over time and in comparison to other cases.

    In addition to the study data, this site makes extensive use of CanLII’s API.

    More about the API and other initiatives at the upcoming “Law, government and open data conference and hackathon” set for this coming Friday and Saturday.

  7. Wow. This is very very nicely done. Congratulations Thom — and to Colin.