Book Review: Litigating Artificial Intelligence

Litigating Artificial Intelligence
Authors: Jesse Beatson, Gerald Chan, Jill Presser
Page count: 368 pages
Publication Date: May 2020
Price: $149 (print) and $99 (e-book)
ISBN 978-1-77255-764-0

In 1962, the renowned science fiction writer Arthur C. Clarke wrote that any sufficiently advanced technology is indistinguishable from magic. As technology progresses with each passing year, more of it may come to feel magical. Yet when that technology has the capacity to impact individuals’ legal rights, it becomes necessary for lawyers to learn the magician’s tricks.

One of the brave new frontiers facing lawyers and judges today is artificial intelligence. AI is deployed in a host of benign ways that make our lives easier. It gives us new content recommendations on Netflix and Spotify. It helps Google Maps or Waze to find the best way home from the office. And it can help translate text quickly from one language to another. In many day-to-day applications, we have no need to understand how that AI works. It works relatively well, and that’s good enough. Yet the application of AI to high stakes issues is a different matter. A cautionary tale is the story of Amazon’s use of machine learning algorithms as part of its hiring process. The use of that tool merely ended up replicating inherent biases already present at Amazon, directing Amazon to hire male workers. The application of AI in the criminal law context, where individuals’ liberty may be at stake, is potentially even more worrisome.

For AI to be used in a manner that gains broad societal acceptance, the answers that AI provides must be open to challenge by lawyers, at least for high stakes decisions. But therein lies the rub. Typical artificial intelligence programs are black box algorithms, meaning the relationship between the inputs and the outputs is horrendously complex, trained on proprietary data that is not available to the public. As a result, it is essentially impossible to understand, let alone explain, why the AI did what it did. And as litigators know, a decision that is impossible to understand or explain is also impossible to challenge.

In this regard, the text Litigating Artificial Intelligence, 2021/2022 edition, edited by Jesse Beatson, Gerald Chan, and now Justice Jill R. Presser of the Ontario Superior Court of Justice, is an indispensable resource for any lawyer who faces artificial intelligence or algorithmic machine learning issues in their practice. And given that more lawyers will face these problems every day, Litigating Artificial Intelligence fills an important niche. There is currently relatively little law in Canada dealing with the use of artificial intelligence in the court room or tribunals, but this text does an excellent job of walking through the problems that may arise and the analytical framework they can use to unpack those problems.

Take for example the chapter by Kate Robertson and Justice Presser entitled “Algorithmic Technology and Criminal Law in Canada”. That chapter masterfully works through the problems of algorithmic risk prediction in the criminal law system, probabilistic genotyping DNA tools, and predictive policing technology. It shows lawyers how to challenge such tools both from a conventional evidentiary perspective and using Charter principles. Taking the issue of algorithmic predictions in the bail context, Robertson and Presser systematically walk through the application of the rules relating to expert evidence to the admission of algorithmic evidence, and highlight the current case law that might help litigators challenge the admissibility of such evidence. Even in the absence of a well-established body of law, their chapter provides a ready-made road map for criminal lawyers facing any of those issues to deal with the problems.

The problems that Robertson and Presser highlight with AI are significant and well-known to experts in machine learning and data science, and they need to be understood by lawyers seeking to challenge the use of decision-making by AI in the courtroom. All machine learning algorithms consist of a statistical algorithm that tries to learn the underlying pattern in historical data (called training data) so that it can then be extrapolated to particular cases (test data). Yet, that process can have several potential problems. For example, the data on which the algorithm is trained may reflect biased historical decisions. If the algorithm is trained on data generated by a criminal justice system that historically made racist decisions, the algorithm will merely replicate racist decisions in its future predictions. Yet, investigating such potential bias requires detailed information about the training data set, which will often be resistant on commercial or other confidentiality grounds.

Moreover, human error can easily creep its way into the algorithm’s code. It is entirely possible for an algorithm to be coded incorrectly in a way that it produces results that in general seem right, but in certain cases are horrifically wrong. The only way to discover this would be to review and carefully analyze the code. But again, the disclosure of the code for the algorithm tends to be heavily resisted by those who own it. Robertson and Presser’s chapter, and many others in that text, give insights for how litigators can confront those very problems.

The criminal justice system is not the only context of legal practice in which AI arises, and Litigating Artificial Intelligence is by no means a text only for practitioners of criminal law. Justice Lorne Sossin of the Court of Appeal for Ontario contributes a chapter on administrative law and artificial intelligence. Petra Molnar provides an overview on the use of artificial intelligence tools in migration decisions. And Ren Bucholz and Andy Yu contribute a chapter on civil liability for AIs causing harm, a relatively new area of practice.

A series of chapters on AI as part of the litigation process round out the text. Civil litigators are used to using AI technology for reviewing large document data basis for relevance and privilege, and the chapter by Maura R. Gossman and Gordon V. Cormack provides an overview of that process and the key points that litigators must understand. Rounding that section are a chapter by Chris Bentley on on-line dispute resolution, while Anthony Niblett contributes a chapter on predictive analytics, both of which are more likely to be used increasingly by lawyers in the future.

The core value of Litigating Artificial Intelligence lies in its breadth. It is unlikely to provide the final word on any issue in this space, given the complexity of issues and the myriad of legal context in which AI will arise. However, what it does provide a road map that lawyers can now use to chart through these issues in all of the different contexts in which AI can arise in legal practice. Essentially, every artificial intelligence issue that a litigator might run into within the next decade is canvassed to some extent or another in that text. As AI finds further applications in various domains, a baseline level of knowledge will be critical for all lawyers. This text provides exactly that baseline level of knowledge.

– Paul-Erik Veel
Lenczner Slaght