Summarize This?

Last week, the Canadian Judicial Council (CJC) released its updated Guidelines for the Use of Artificial Intelligence in Canadian Courts. Credit is due to the CJC for its ongoing thoughtful engagement on this important topic. Similarly, several Quebec courts released their own guidelines on judicial AI use last month. This column looks at one feature that both sets of guidelines have in common: guidance for judges on using AI tools to summarize material filed with the court, including evidence and submissions. This is far from the only aspect of the CJC and Quebec guidelines worthy of discussion, but it is an aspect that risks being overlooked and requires, in my view, careful consideration.

The Quebec guidelines state that judges might reasonably contemplate using generative AI for:

the production of a strictly descriptive summary of a witness statement, a document or a factual sequence, without any assessment, interpretation, normative selection or conclusion.

Similarly, the CJC guidelines describe as an “appropriate” use of AI for judges:

summarizing lengthy affidavits, expert reports, transcripts or written submissions for later verification against the originals.

It is easy to understand why judges may be interested in using generative AI tools to summarize material. Legal proceedings often involve voluminous documents and courts are under tremendous pressure to work faster. A closer look, however, reveals that AI summarization is not the risk-free (or even always low risk) efficiency boon that it is often presented to be.

First, generative AI summaries are not always accurate. This is a basic point but one that is not always fully appreciated. An American lawyer recently discovered the frailties of AI summaries the hard way when “he thought that ChatGPT would provide a ‘bulletproof’ summary when he provided his client’s trial transcripts [to the tool]” and then, relying on ChatGPT’s output, filed an appeal brief with the court that “included references to nonexistent witnesses and fictional testimony from at least one real witness.” While this is an extreme example that could have been avoided with some basic verification (something that the CJC and Quebec guidance expressly advise), accuracy issues in AI-generated summaries are not always straightforward or easy to catch.

AI tools, for example, can generate summaries that include wrong or misleading inferences. A stark example is the following finding from a recent study on AI scribes that are used by doctors to draft clinical notes: when summarizing a telephone consultation, two of the three AI scribes wrongly inferred that a physical examination had taken place and wrote up the history as if it followed that non-existent physical examination. This type of erroneous inference should be easy to catch on a subsequent human review, but more subtle incorrect inferences may be harder to spot. Similar concerns relate to errors of omission. Jennifer Ellis, in her excellent column about the risks lawyers face when using AI to summarize documents and materials, observes:

Summaries, by their nature, leave things out. The things AI leaves out might be important. You cannot assume that AI will understand what it should keep in and what it should leave out. AI is not a lawyer, it does not have the knowledge of a lawyer when it comes to deciding what matters.

Knowing whether a generative AI tool has omitted an important or even critical piece of information in an expert report that is hundreds of pages long can be very difficult unless one carefully reads the whole document (arguably defeating the purpose of using an AI tool in the first place).

Second, AI summaries are not value neutral. Yet, this seems to be the assumption underpinning the Quebec court guidance when it references “strictly descriptive” summaries that do not involve “any assessment, interpretation, normative selection or conclusion.” But again, generative AI summaries, by their very nature involve decisions—assessments made by the AI tool—about what should be included and what can be left out. AI summaries also involve decisions about how the included content should be organized and described. The language used to summarize the material (which can incorporate both subtle or not subtle biases) and the framing of the summary can unconsciously impact how a judge sees a matter, even if the judge themselves subsequently reads the original documentation after reviewing an AI-generated summary.

The CJC Guidelines quite rightly observe that “[o]ne of the less visible challenges posed by generative AI is that each tool embodies an implicit ‘world model’: a set of assumptions, values, priorities, and statistical patterns learned from its training data and reinforced through its design and post-training alignment” and that “[t]hese embedded perspectives may influence how information is framed, summarized, or prioritized, even when responses appear neutral and authoritative.” While a judge using AI-generated summaries of court material to assist them in making a decision is not the same as relying on AI to make the decision entirely, it does risk inserting a worrying machine-driven influence into their decision-making process which the judge may not even be aware of.

Third, it is worth considering what is lost when we “see the forest but not the trees”, to reverse a well-known idiom. In cases where a judge is reading AI summaries, they surely risk losing some of the context that might otherwise inform their decisions. Studies have demonstrated that when individuals learn about something from an AI synthesis rather than parsing the raw material themselves, they develop shallower knowledge. Additionally, judges relying on AI-generated summaries risk creating distance between themselves and the humans impacted by, or otherwise implicated in, their decision-making. This would seem to be a particular risk in cases where AI is used to summarize the testimony of a witness, exchanges between people recorded in transcripts, affidavits or witness statements. Undoubtedly, some remove is necessary for judicial work. But what do we lose when we swap out the actual words of humans for machine compressions? The importance of “being heard” is a long-standing principle of administrative justice. Beyond the technical legal application of this principle, we should ask what acts of respect and humanity are involved in judicial listening. What is lost when the words of the people involved in court proceedings are taken as prompts for AI summaries instead of expressions that deserve, or perhaps even demand, direct engagement?[1]

Of course, these risks and concerns will materialize differently depending on what generative AI tool is being used, how it is being used, and in what context. There are ways, for example, to produce relatively more accurate AI summaries and some tools will produce relatively more accurate summaries than other tools. At the same time, many, if not all, of the concerns above persist in some form or another with any use of AI tools to generate summaries of case materials. No tool can accurately replicate a judge’s professional discretion about what is relevant or important to include in a summary. Likewise, reading an AI-generated summary of someone’s words is never going to be an equivalent experience to reading their actual words in full context.

Whenever questions about AI assistance for judges arise, someone inevitably draws a parallel to judges relying on law clerks and other human staff for summaries, arguing that judges regularly rely on summaries and syntheses produced by others. In response, I’d suggest that rather than seeing the fact of human-generated summaries as a reason to uncritically accept AI-generated summaries, we consider what concerns about judicial use of AI-generated summaries might apply in equal measure to their use of human-generated summaries. It’s possible that, on further review, one might conclude that, in some circumstances, judges ought to be less reliant on human-generated summaries of court materials. Additionally, it is important to recognize that generative AI tools do not, as a general matter, approach tasks in the same way as humans (see my earlier Slaw column for a discussion of this in a different context). Human summarizing a qualitatively different process than AI summarizing. Lastly, those participating in justice processes may have very different responses to learning that a machine-judge team has undertaken key tasks in the decision-making process as opposed to a judge who is supported by other humans. That judging is a human endeavour can be seen as not only a lodestar for those who work in the justice system but also as an expectation for those who use it and as importantly connected to perceptions of fairness and justice.

There is no doubt that our justice system is under enormous strain and risks further erosion of public confidence as it struggles with fewer resources and greater caseloads. We need to explore and embrace innovative solutions; but in doing so we must also be clear-eyed about the risks and the trade-offs of inviting AI tools into the judicial process. While official guidance contemplating judicial use of AI tools to generate summaries may feel quotidian, in my view this use may require more caution than is currently being suggested.

_________

[1] On this point I am indebted to a Professor Jennifer Raso and her thoughtful work on the concepts of hearing and listening in technology-mediated contexts.