Taking Stock: A Practical First Step in AI Governance
Over the past two and a half years of writing about AI governance, risk and compliance, I have covered a range of subjects, from the stalled progress of federal AI legislation and the role of privacy regulators to the first wave of AI litigation in Canadian courts and, most recently, the responsibilities of boards. When I have received responses from readers, they have been thoughtful, and I am grateful for them. One question, however, comes up with such frequency that I thought I should address it directly. It usually arrives by email after a column is posted, or in conversation after a presentation, and commonly takes some version of the following form: this is all very interesting, but what should my organization actually do about it?
It is a fair question. Much of the commentary on AI risk, including a good deal of my own, identifies gaps, describes emerging risks, and encourages caution. It can leave the reader with a clearer understanding of the problem but not the first practical step toward addressing it. For a managing partner, a general counsel, or the executive director of a legal clinic already stretched across competing priorities, that is not a particularly satisfying place to be left.
In this column, I want to suggest a practical starting point for law firms and legal organizations considering how to manage AI risk: a structured examination of how artificial intelligence is already being used within the organization. I will refer to this as an AI governance audit. Policies, training, vendor due diligence, and clear lines of accountability all have their place, but each is more useful when grounded in an accurate picture of current practice. Much of the initial work can be undertaken internally, and its scope can be adapted to the size and circumstances of the organization.
Why Begin with an Audit?
In earlier columns I have suggested that one of the most significant governance problems in this area is a lack of visibility into how AI is being used. Artificial intelligence often enters a legal organization through several small decisions rather than a single, deliberate adoption process. A lawyer experiments with a public generative tool to prepare a first draft. A research or practice management platform adds an AI feature. An assistant discovers that a transcription tool saves several hours each week. Each development is understandable on its own terms, but together they can produce an organization whose actual use of AI differs considerably from what its leadership believes to be the case.
This matters particularly in legal organizations. Duties of competence, confidentiality, and supervision do not relax simply because a new tool has made a task easier. As the Alberta Court of Appeal reminded the profession in Reddy v Saroya, 2025 ABCA 322, a lawyer whose name appears on a document filed with the court bears responsibility for its contents, including where its preparation has been delegated to someone else. The handling of client information by third party vendors and the obligations imposed by privacy legislation require attention alongside those professional duties. A firm cannot meaningfully assess whether it is meeting these obligations without first understanding how its people are using AI.
In my column on boards earlier this year, I observed that it is impossible to govern a risk that has not first been identified. An audit provides a practical means of doing so.
What Kind of Audit?
The phrase “AI audit” can suggest a highly technical examination of an AI system, or a formal process leading to certification. Those are important forms of assessment, but an audit can also examine how an organization governs its use of AI, including what its people are doing, which obligations apply, whether controls are working, and who is accountable. The scope and the criteria against which practice is assessed should be clear from the outset.
Standards can support this work. The International Organization for Standardization has established requirements for AI management systems and for the bodies that audit and certify them. In Canada, the Standards Council of Canada supports this framework through an accreditation program for certification bodies. While these standards are useful, full implementation of an AI management system may be considerably more elaborate than many law firms and legal organizations need as a starting point. Their complexity should not delay the essential first step of understanding how AI is being used and whether appropriate governance is in place.
An AI governance audit provides a systematic, evidence-based assessment of actual use against relevant obligations, organizational policies, and defined responsibilities. It produces findings that can be acted upon and can identify where specialist technical assessment or standards-based certification would be useful. Certification is a separate objective, and an organization need not pursue it to begin this work.
What does the Audit involve?
The shape of an audit will depend on the organization, but five areas provide a useful starting point.
First, an inventory of actual use. The starting point is a plain account of where AI is being used, by whom, for what purposes, and with what information. This should capture approved tools, informal use of public tools, and AI features embedded in software the organization already licenses. Interviews and short, non-judgmental surveys can reveal uses that procurement records will never show. Asking people how they complete their work may be more revealing than asking them whether they use AI, particularly where they do not recognize a familiar software feature as an AI application.
Second, a mapping of that use against existing obligations. Each use can then be considered against confidentiality, competence and supervision requirements, applicable privacy legislation, client engagement terms, and relevant law society guidance and court requirements. The assessment should consider both the information supplied to the tool and what happens to its output. Reformatting a non-confidential template with an approved tool presents different risks from summarizing sensitive client files or preparing material for filing in court.
Third, a comparison of policy against practice. A policy drafted in the early days of public generative AI may no longer reflect the tools available or the ways people use them. The audit asks two questions I have suggested elsewhere are particularly revealing: does the policy correspond to actual practice, and is it being followed? A policy that prohibits uses which have quietly become routine provides little practical protection and may create false confidence. The response may require enforcing an existing restriction, revising an outdated policy, or providing an approved alternative.
Fourth, an assessment of ownership and escalation. The audit should identify who, by name or by role, is responsible for AI risk, who approves new uses, and what circumstances require escalation to leadership or a board. It should also establish whom staff can approach when they are uncertain about a proposed use or concerned about an incident. Where no one holds a clearly defined responsibility, the audit has identified an important gap.
Fifth, a review of vendors and contracts. The terms governing how vendors store, use, retain, and delete client information require careful attention. The audit should consider whether those terms are understood and consistent with the organization’s obligations, including whether information may be used for model training or accessed by other service providers. Someone should also be responsible for monitoring material changes in terms and product features.
The result should be a short, written account of current practice and a prioritized action plan. Each recommendation should identify the concern, the action required, the person responsible, and a timeframe for completion.
Consider a firm that discovers staff are using an unapproved transcription service for client meetings. A useful finding would go beyond recommending a better AI policy. It would identify what information has been supplied, require an assessment of the service’s data practices and any resulting exposure, and assign responsibility for addressing the concern. Depending on the findings, the firm might suspend that use, approve a suitable alternative, and provide guidance for future meetings. The audit becomes useful when it connects an observed practice to a decision and someone accountable for carrying it out.
Scope and Limits
A governance audit should draw on the work of information technology professionals. It does not replace a technical security assessment or the evaluation of data architecture and system integration. As I have argued previously, technical work is indispensable, but AI use also engages professional judgment, legal obligations, and institutional accountability. The audit brings those considerations into a common assessment.
It should also establish a baseline for subsequent audits. Tools change, vendors introduce features, and staff find new uses for systems they already know. An annual update may be sensible for some organizations, but significant changes in tools, contractual terms, or intended uses may warrant an earlier assessment. An incident may also reveal that an existing approval needs to be reconsidered. The frequency should depend on how the organization uses AI and the degree of exposure that use creates.
Proportionality
For a sole practitioner or a small firm with limited AI use, an initial audit might begin with an afternoon spent listing tools, reading vendor terms, and comparing current practice against relevant professional guidance. That may be enough to identify the next steps, even where resolving the findings takes longer. A larger firm, legal department, or public sector legal organization will generally need broader participation and more documentation. In either case, the depth of the exercise should reflect the nature of the use, the information involved, and the potential consequences for clients and others.
In my conversations with organizations over the past year, a few patterns come up often enough that they are worth looking for. Leadership may underestimate informal use. Policies may lag behind available tools. Embedded features may have been enabled without a clear assessment of their implications. Staff may give different answers when asked whom they would consult about appropriate use.
Such findings may simply reflect capable people trying to manage workloads in an environment where organizational guidance has not kept pace with the technology, and should not be taken as evidence of carelessness or bad faith. An audit should leave room for that explanation while still addressing uses that fall short of the organization’s obligations.
The aim is to make useful experimentation easier to govern. Staff who understand which tools they can use, what information they can supply, and when they need approval are better placed to use AI with confidence. Clear guidance also gives them somewhere to take questions that should not be left to individual judgment alone.
Conclusion
An AI governance audit will not resolve every uncertainty or eliminate the possibility of error. It can, however, give an organization an accurate picture of its own practice and provide a sound foundation for policies, training, vendor arrangements, and accountability.
It also contributes to the governance record I discussed in a previous column. If something goes wrong, that record may help explain what the organization knew, what it considered, and how it responded. Its value depends on identifying gaps, assigning responsibility for addressing them, and recording what was done. A report that identifies a significant concern and is then left unattended may reveal knowledge of a risk without any meaningful response to it.
For law firms and legal organizations wondering where to begin, my suggestion is simply this: begin by finding out where you already are. The next decisions become clearer once that picture is in hand. As always, I welcome hearing from readers about their experiences, including approaches that have worked well and those that have proved less useful.
__________
Note: Generative AI was used in the preparation of this article.




Start the discussion!