The Magic Ingredient

What makes CLEBC publications so popular with BC lawyers? There’s no doubt our talented editorial and production staff make a great contribution, but I believe the magic ingredient is the combined wisdom of our excellent authors and editorial boards.

Once the authors have made their submissions, their work is carefully reviewed—page by page—by the legal editor and the editorial board. Editorial board meetings usually involved spirited discussions of law and practice. The implications of recent decisions, the significance of new statutory requirements, a new practice or understanding that has arisen within the BC legal community—all are flagged and discussed to ensure accuracy. The final product is a reliable guide to the practice of law in BC.

So can that wisdom be captured by the algorithms used for artificial intelligence?

Information and speculation about the use of AI in law has certainly been a constant this past year. Along with everyone else, I wonder about the effect of artificial intelligence on our world of legal information.

Some key themes emerged:

  • The algorithm is key, but it’s not neutral. Tim Knight has made a great contribution to this discussion in his columns here and here. He reminds us that there should be some accountability around the content of the algorithms used in AI. A recent column at 3 Geeks also calls for algorithm accountability. The author reports on research by Susan Nevelow Mart in which different legal research databases were searched using the same search terms and data content. The results were quite surprisingly (or alarmingly) inconsistent, with only about 7% of the cases returned in all six database results. Understanding the bias written into the algorithm is essential to understanding knowing how accurate or effective it is.
  • Understanding the size and quality of the data set is also essential to evaluating the reliability of the results; this is the topic of Robert Parnell’s excellent paper When Big Legal Data Isn’t Big Enough: Limitations in Legal Data Analytics. Omar Ha-Redeye also addresses this problem. For example, is there enough data coming from the BC courts and legislature to support effective AI? CanLII contains just over 20,000 decisions from the BC Court of Appeal (that’s every decision since 1990). The court’s 2015 annual report tells us that the court disposed of 850 appeals in 2015, 230 of which were criminal appeals. The small size of our data sets may means that it will be difficult to get useful results from AI for our jurisdiction.
  • There’s a tension between the trend to uniformity generally and location-specific practice that often develops. We see this in BC when we discuss how law is practiced in downtown Vancouver compared to the smaller and more remote communities. As long as there’s information specific to a particular jurisdiction, there will be a need for information about those practices.
    On the other hand, changes in the way law is used and understood may make local practice less relevant. It’s become very easy to research law from other provinces or other countries. Greater access to information from other jurisdictions means that different ways of thinking about or resolving problems are more readily available. After all, dispute resolution professionals everywhere are looking for tools and ideas that will lead to early resolution, wherever those ideas come from.
  • A couple of columnists have addressed the effect of AI on the work of lawyers and information professionals. To what extent with the human element still be required? Jamie Baker explains the three main levels of natural language processing, predicts that computers will not be able to master high level pragmatic language processing (that is, mastering context) until about 2100, and cautions:
    What we all must be cognizant of is the notion of “premature disruption”—losing valuable players (lawyers and librarians) before the technology is truly ready to replace them. ROSS, for example, is really good at PR, but stakeholders must be careful to avoid premature disruption. ROSS is a great tool that will augment lawyers and librarians, but it will not replace us. And it won’t replace us for the foreseeable future.

    Meanwhile, Louis Mirando asks: what are the possible consequences of AI for law librarians? He speculates that they will need to develop different skills: “All librarians will need to improve their technological literacy, especially coding and text markup, better to understand and assist with the transformation of information into data.”

So will the capabilities of AI replace the insight we find in excellent secondary sources? At some point will the need for wisdom or agreed practice (as reflected in CLEBC publications) be eliminated or become irrelevant?

Of course, it’s too soon to say whether or when AI will replace secondary sources from a small jurisdiction. It may be that by the time high level pragmatic language processing is successful, CLEBC publications will be just a happy memory. On the other hand, our content may well become part of the data that is being examined.

(For a further and very interesting take on these issues, see Colin Lachance’s recent column Racing to Nirvana—Legal Research Edition and the robust discussion in the comments; his column was published just as I was completing this one.)


  1. I would just like to add my observation that in respect of AI, we are about where “the jetsons” were in respect of predicting where electronic technology might lead. It will become what people demand of it and what is technically feasible. There is still a lot of technologically feasible automation yet to be taken up and put into application.
    As to accountability It seems strange to demand greater accountability from the AI than from the data sources it works from.
    Generally, I believe what is necessary for the legal profession is make law more sophisticated and less complicated. The future of our economy depends on that. This is a larger problem of course, and I believe as Susan suggests, that technology will play more of an assistive role.
    I would pose the question : what will Law do for the economy in order to stimulate growth and facilitate new forms of business, and what will Law do to communicate what it needs from technologists for it’s own interests as well as the greater good.

  2. Dear Susan,

    The answer to the CLEBC material is so popular is this: It’s damn good and the price is remarkably low – far less than what CLEBC could reasonably charge – because CLEBC isn’t out to make a profit.

    The AI vs Librarian or other competent “form” of researcher issue? Would the minimally adequately programmed AI have needed to write email (to a firm that I left 6 years ago). The email got to me, soon enough, because, not surprisingly, my friends at that firm know where I am.

    “I have been asked to find the following article which is cited as being on your website but which appears to be no longer available. Cheifetz, David, Causation in Canada in the Third millennium. Nothing is now Enough – Scraping the Surface The Consequences of Resurfice Corp. v Hanke 2007 SCC 7 , June 23, 2008. This is cited, among other places on WestlawNext.”

    10 seconds or less of typing on any of the leading search engines produces the new URL through a search that took the engine significantly less than 1 second.

    A few seconds on the web would have provided the email writer with the information needed to email me if, for some astonishing reason, the writer’s search engine failed to get any hits. (I didn’t check on what would happen if my name was misspelled “Chiefetz”.)

    I wonder how much the client got billed for the work leading up to the email

    Best regards,