Ensuring Professional Competence?

In February, it was reported that the UK’s Legal Services Board was moving forward with plans to introduce “continuing competence checks” for lawyers. This could involve the regulator obtaining feedback from consumers, judges and peers; making quality assurance visits; and possibly even requiring formal revalidation of lawyers’ credentials.

In my last column, I discussed how the raison d’être of lawyer regulation is to ensure that anyone providing legal services will meet standards of professional competence and professional conduct. In Ontario, this is codified in s. 4.1(a) of the Law Society Act.

But I have long wondered: Is the self-regulating legal profession, in fact, doing enough to ensure practising lawyers meet standards of competence—not just upon entry to the profession, but throughout their careers?

If not, would alternative or supplementary measures (such as “continuing competence checks”) help us better achieve that goal? Or would it this simply increase the regulatory burden on licensees—which can trickle down as increased costs to consumers—without much practical benefit?

While this column cannot definitively answer this question, I thought it could be interesting to explore the issues and solicit feedback from readers—whether practising lawyers, law students, law teachers, legal regulators, clients, self-represented parties, or others—as a useful starting point for discussion. I understand at least one Canadian legal regulator is actively considering this issue, and I expect others are as well.

Below, I provide an overview of the regulatory measures in place in Ontario to ensure lawyer competence.[1] In so doing, I seek to make two overarching points:

1. Legal regulators tend to focus on “input regulation” intended to ensure entry-level competence, and place relatively little focus on “output regulation” to ensure continuing competence and the provision of quality services after lawyers are called to the bar;[2]
2. There is little or no empirical support for the policy decisions legal regulators have made about what is required to demonstrate competence, whether entry-level or otherwise.[3]

I. Entry requirements, or “input regulation”

The requirements for the issuance of licenses to provide legal services in Ontario are detailed in the Law Society of Ontario’s By-Law 4. A candidate must meet three substantive requirements to be admitted to the bar as a barrister and solicitor:[4]

1. obtain an accredited law degree;
2. complete an experiential training requirement; and
3. pass the bar exam.

1. Law degree

Section 9(1) of By-Law 4 provides that to apply for a license to practise law, a candidate must have one of:

1. A bachelor of laws or juris doctor degree from a law school in Canada that was, at the time the applicant graduated from the law school, an accredited law school.
2. A certificate of qualification issued by the National Committee on Accreditation appointed by the Federation of Law Societies of Canada and the Council of Law Deans.

The Law Society of Ontario has accredited all twenty Canadian law schools that currently offer a common-law program for the purpose of this requirement.[5] Each of these law faculties satisfies the FLSC’s National Requirement, a curriculum adopted by the law societies in 2010 that specifies the educational requirements that applicants for admission to the bar are expected to meet (including substantive knowledge about public and private law and the Canadian legal system).

Individuals who obtain a law degree from outside Canada cannot apply for a license to practise law in Canada until they complete a program prescribed by the National Committee on Accreditation (“NCA”), a standing committee of the FLSC. The NCA individually assesses the credentials of foreign-trained lawyers and determines what further legal education is required for the candidate to demonstrate that their training is comparable to that provided by an accredited Canadian common law faculty (or, in other words, that these candidates also meet the National Requirement). The NCA typically requires candidates to complete certain examinations to demonstrate the requisite substantive knowledge, sometimes after completing coursework offered by an accredited Canadian law faculty, and will issue a certificate of qualification to candidates upon their successful completion of the prescribed requirements.

The National Requirement prescribes some areas of required substantive knowledge: (1) foundational common-law principles; (2) ethics and professionalism (3) Canadian constitutional, criminal, and administrative law; and (4) contracts, torts, and property law.

I note as an aside that, interestingly, there is no requirement for specific training in areas in which lawyers have in recent years sought to protect their “turf” vis-à-vis non-lawyer service providers, such as family law and immigration law.

2. Experiential training

Section 9(1) of By-Law 4 provides:

3. The applicant … must have,

i. experiential training by successfully completing,

A. service under articles of clerkship for a period of time, not to exceed ten months, as determined by the Society and all other requirements, as determined by the Society, that must be completed during the time of service under articles of clerkship, or

B. the law practice program […]

The nature of the experiential training requirement in Ontario has been the subject of much debate. In December 2018, the Law Society of Ontario (“LSO”) decided to continue permitting licensing candidates to satisfy this requirement through by completing either a 10-month articling term or the Law Practice Program (“LPP”).

This decision was made on the basis that there would be “enhancements” implemented by May 2021 to address some of the concerns that had arisen with articling. In October 2020, however, it was announced that “[t]he enhancements are on hold as a result of new priorities that have emerged since the onset of COVID-19”.

This is neither the time nor place to rehash this debate, but I note that neither articling nor the LPP requires evaluation on anything other than a “pass/fail” basis—and, to the best of my knowledge (I am not aware of any data made public in this regard), very few candidates fail on the basis that they cannot demonstrate sufficient competence.

3. Licensing exams

Section 9(1) of By-Law 4 further provides:

2. The applicant must have successfully completed the applicable licensing examination or examinations set by the Society by not later than two years after the end of the licensing cycle into which the applicant was registered.

The LSO administers two licensing exams: the Barrister Examination and the Solicitor Examination.^[6] When administered in person, each has been offered as a full-day examination composed of approximately 240 multiple-choice questions. Since the COVID-19 pandemic, they have been offered online as 4-hour exams with about 160 multiple-choice questions (the LSO now plans to continue offering online licensing exams through 2024, which will permit it to evaluate the effectiveness of this delivery model). Either way, there is no written component, such as short answer or long answer questions.

The bar exam is administered to licensing candidates in an open-book format. The LSO prepares and provides to candidates detailed study materials that speak to all examined “competencies” on the licensing exams (discussed further below). The bar exams are developed with reference to these study materials, and while candidates are permitted to refer to any notes or textbooks to assist them in writing the exam, no external material or information is required to successfully complete it. For the 2019-2020 licensing term, the LSO’s prepared materials were over 2000 pages long.

The bar exams are marked on a pass/fail basis. When it develops and sets the examination, the LSO determines the passing mark for the licensing examination, and scores equal to or higher than the established passing mark receive a “pass” result, and scores lower than the passing mark receive a “fail” result. A candidate’s performance is assessed only with reference to the pre-determined passing mark; it is not assessed in comparison to the performance of other candidates, by using a curve or otherwise, and there is no pre-determined rate for the proportion of candidates who will pass the bar exams.

The LSO has made some bold claims about the effectiveness of the bar exam in measuring competence; for example, in the consultation materials it prepared for 2017’s “Dialogue on Licensing”, it stated that the bar exam permitted it “to ascertain, with evidence, that individuals who score below the passing mark are not competent and individuals who score above the passing mark are competent”.^[7]

Without wading into the grounds for this assertion today, it suffices to say that many law students and practising lawyers would disagree that passing an open-book, multiple choice exam can, in fact, demonstrate that the test-taker is competent to practise law.

II. Post-entry competence regulation – is there “output regulation”?

Everyone licensed to practice law must demonstrate they meet entry-level competence standards through the above-noted regulatory steps.

In its working paper, “Overview of the Lawyer Licensing Process in Ontario” (part of a series prepared for the 2017 “Dialogue on Licensing”), the Law Society of Ontario stated that the lawyer licensing process, which is directed toward assessing entry-level competence, “is one of many regulatory evaluations and requirements that will occur during a lawyer’s career”.[8]

But I suggest that once licensed, it is entirely possible that a lawyer will not have any meaningful evaluation of their competence.

What are the regulatory measures in place to regulate post-licensure lawyer competence?

1. Competence standard prescribed in the Rules of Professional Conduct
2. Mandatory liability insurance
3. Mandatory continuing professional development (“CPD”)
4. Regulatory evaluations: annual reports, spot audits, and practice reviews

1. Competence in the Rules of Professional Conduct

Ontario’s Rules of Professional Conduct provide a lengthy but general definition of a “competent lawyer” at Rule 3.1-1 (“competent lawyer” means a lawyer who has and applies relevant knowledge, skills and attributes in a manner appropriate to each matter undertaken on behalf of a client including…), then provide at Rule 3.1-2 that “A lawyer shall perform any legal services undertaken on a client’s behalf to the standard of a competent lawyer”.

The Commentary provides general factors about how to decide whether a lawyer has employed the requisite degree of knowledge and skill in a particular matter. Also relevant is Rule 3.2-1, which states that “A lawyer has a duty to provide courteous, thorough and prompt service to clients. The quality of service required of a lawyer is service that is competent, timely, conscientious, diligent, efficient and civil.”

In her excellent article “From Colleague to Cop to Coach: Contemporary Regulation of Lawyer Competence”,[9] Amy Salyzyn discussed how lawyer incompetence and quality of service are infrequently the subject of lawyer discipline proceedings. She conducted an informal study that concluded that in 2015, fewer than 7% of discipline cases involved allegations relating to violation of the competence rule, and less than 1% dealt with standalone allegations of incompetence. She noted that these cases “tended to involve very clear failures on the part of the lawyer”, such as failing to complete the work for which the lawyer had been retained; taking actions contrary to the client’s express instructions; or doing nothing or close to nothing on a file.

Although further study would be useful to confirm Professor Salyzyn’s preliminary findings, I would suggest it is fair to state that discipline proceedings are not being heavily relied upon by law societies to regulate post-licensure competence (perhaps for good reason).

I also note that the Commentary to the Ontario competence rule continues:

[5] A lawyer should not undertake a matter without honestly feeling competent to handle it, or being able to become competent without undue delay, risk, or expense to the client. This is an ethical consideration and is distinct from the standard of care that a tribunal would invoke for purposes of determining negligence.

[6] A lawyer must recognize a task for which the lawyer lacks competence and the disservice that would be done to the client by undertaking that task. If consulted about such a task, the lawyer should

(a) decline to act;

(b) obtain the client’s instructions to retain, consult, or collaborate with a licensee who is competent for that task; or

(c) obtain the client’s consent for the lawyer to become competent without undue delay, risk or expense to the client. [Emphasis added.]

I would suggest that the main purpose of the competence rule is not to be enforced through discipline, but rather to (a) set and inform lawyers of a standard to be met and aspired to (and perhaps enforced through non-disciplinary means, such as practice reviews and associated recommendations, described below), and (b) require lawyers to “self-regulate” by declining to act in circumstances where they cannot meet the standard required of them.

2. Mandatory liability insurance

Generally speaking, regulators across Canada require lawyers in private practice to hold professional liability insurance as a condition of maintaining their license in good standing (in Ontario, it is required by By-Law 6).

In my view, this is good policy—but it does not do anything to ensure or even promote professional competence or quality services.

Rather, it assists in ensuring that a meaningful civil remedy is available to clients in the event a lawyer (whether by reasons of incompetence otherwise) has not provided quality services and harmed the client’s interests though an error, omission, or mistake. A client who has concerns about their lawyer’s competence or quality of services must bring a civil negligence action to access that remedy, which presents a number of practical challenges (including the need for the client to identify that they have received poor quality services; the cost of hiring counsel to advance a negligence action; and the delays inherent in the civil justice system).

3. Mandatory CPD

In Ontario, By-Law 6.1 requires lawyers to complete 12 hours of continuing professional development activities each year (at least 3 hours of which must be accredited as covering ethics, professionalism or practice management topics) and to report to the Society identifying the eligible CPD activities completed.

Much has been written questioning or supporting the value of this requirement, including in Slaw (by Jordan Furlong, Omar Ha-Redeye, and Alice Woolley, for example). I would also commend a 2017 article by Professor Rima Sirota of Georgetown Law, “Making CLE Voluntary and Pro Bono Mandatory”, in which she makes an empirical argument that mandatory CPD does not improve lawyer competence.

I will not repeat the points made by these other commentators, except to quote this helpful and, in my view, balanced proposition offered by Justice Woolley in her Slaw piece:

…law societies must be modest in their claims about CPD. No, CPD will not ensure lawyer competence. It will not make an incompetent lawyer competent. Mandatory CPD creates an opportunity for a lawyer to enhance their competence, but it does nothing more than that.

4. Regulatory evaluations

The LSO has a few other tools available to it to review lawyers’ practices, but their practical value in promoting and ensuring professional competence is unknown.

By-Law 8 requires lawyers to complete an annual report in respect of the licensee’s professional business in the year. This involves reporting on the types of work a lawyer does, but does not purport to evaluate or measure professional competence in any way.

The LSO is also empowered by s. 49.2 of the Law Society Act to perform a Spot Audit to assess and ensure compliance with record keeping requirements and identify misconduct relating to financial matters. In 2019, the LSO performed 1,309 spot audits, and identified that 14% of law firms had “serious books and records deficiencies” (54% had either minor or no deficiencies, and 32% had deficiencies that were readily remediated to the LSO’s satisfaction) (see p. 10 of this report). It is not clear what happened with the 14% of firms that had serious deficiencies.

Generally speaking, spot audits for this purpose do not bear upon competence in the sense of ensuring the quality of legal services provided to clients. That being said, a failure to maintain record-keeping standards identified during a spot audit may suggest other deficiencies in the lawyer’s practice—indeed, deficiencies noted in spot audits may form the basis of reasonable and probable grounds for a focused practice review.

The “focused practice review” is perhaps the most interesting regulatory measure available to and exercised by the LSO to ensure professional competence. Section 42 of the Law Society Act and section 27 of By-Law 11 permit the Society to conduct a review of a licensee’s practice if “there are reasonable grounds for believing that the licensee may be failing or may have failed to meet standards of professional competence”, considering, among other things:

• The nature and number of complaints made to the Society in respect of the licensee;
• Any orders made against the licensee or undertakings given by the licensee;
• Any information coming to the attention of the Society in the course of considering a complaint, investigation, or proceeding that suggests the licensee may be failing to meet standards of professional competence;
• The result of a spot audit that suggests the licensee is in default of financial and other administrative and record-keeping requirements.

A focused practice review will lead to a final report by the Society, which may include recommendations for improvement, and can lead to an order against the licensee requiring remedial action to ensure the lawyer meets standards of competence.

The LSO also conducts “practice management reviews” on lawyers who are in their first eight years of practice, using a “risk based random selection process”. These reviews consider basic practice management systems, such as file management, financial management, client communication, and technology (as opposed to competence in substantive or procedural law), but is designed to identify practice management issues that could have an adverse effect on the quality of legal services. Law Society data from 2019 states that 473 lawyer practices underwent a practice review; 66% of initial reviews met standards of professional competence; 34% of initial reviews required a follow up review; and (presumably after the follow-up review) there was a 99% compliance rate with Law Society competence standards (see p. 10 of this report). It is unclear what happened to the 1% that did not meet competence standards.

To the best of my knowledge (and I welcome the opportunity to be corrected), there is no data available to the public (or interested columnists) about how many focused practice reviews are conducted; on what basis these practice reviews are conducted; or the outcomes of focused practice reviews.

Both focused practice reviews and random/risk-based practice management reviews seem to me to be useful tools to work with licensees to identify and remediate issues of professional competence—but it is unclear to the outside observer how they are being used, and if they are in fact effective.

III. Issues to consider going forward

In my introduction to this column, I made two propositions:

1. Legal regulators tend to focus on “input regulation” intended to ensure entry-level competence, and place relatively little focus on “output regulation” to ensure continuing competence and the provision of quality services after lawyers are called to the bar;
2. There is little or no empirical support for the policy decisions legal regulators have made about what is required to demonstrate competence, whether entry-level or otherwise.

I would suggest that the above summary of Ontario’s approach illustrates my first point. I acknowledge, however, that some law societies are moving forward with law firm regulation, which is in part a competency-based approach (see, e.g., Saskatchewan’s approach, by which law firms register with the Law Society and become actively involved in assessing and addressing practice management issues).

On the second point, I recognize that it is difficult, and indeed may not be practically feasible, for regulators to gather and rely upon rigorous empirical evidence[10] to support all policy decisions. Perhaps the best our regulators can do is rely on their and their members’ and benchers’ knowledge and experience and exercise their best judgment about what is most likely to promote competence.

I make this point not to suggest that empirical evidence is necessary in this regard, or that the measures in place are not valid, but to put them in context. By and large, these decisions are made based on what the lawyers employed by and responsible for governing the Law Society have judged as appropriate based on their experience and common sense.

I would propose, as Justice Woolley did regarding CPD, that law societies should be modest in their claims regarding whether we are “ensuring” professional competence when the measures are not supported by evidence.

That being said, gathering data about the effectiveness of our professional competence measures is far from impossible. I would be interested, for example, in a study considering whether there is any correlation between bar exam scores/number of bar exam attempts/law school grades and later complaints or reports about lawyer competence. Such data could not only validate regulators’ claims about whether measures serve to ensure professional competence, but could provide useful insight on how competence can be better promoted and regulated.

Before going down the road of introducing new regulatory measures along the lines of the “continuing competence checks” proposed in the UK, I would be interested in more information being made public about the effectiveness of our existing regulatory actions intended to ensure post-licensure competence. I expect the Law Society of Ontario keeps track of more information internally than what is published, and suggest it would be useful for public and licensee confidence in the profession and its regulation to understand more about the ways we seek to ensure continuing competence post-licensure. For example:

• How many lawyers are the subject of complaints about competence and quality of service? [11] What proportion of these complaints proceed to discipline, and what proportion are diverted to be addressed through remedial measures such as practice reviews and subsequent recommendations?
• How many focused practice reviews are run in a given year? What happens when deficiencies are identified in spot audits or focused practice reviews that are not easily remediated to the Society’s satisfaction?
• Are practice reviews effective, in the sense that after implementing recommendations the identified issues are not only resolved but do not recur?
• On what specific bases do competence issues arise? Are they predominantly issues of substantive legal knowledge (or lack thereof), or of a more “customer service” nature such as poor communication, file management, and missed deadlines?[12]

Finally, I’m curious about what Slaw readers think about the existing regulatory regime for “ensuring” competence, and what else could or should be done to achieve this important objective. In the comments, please feel free to share your thoughts, including on:

• Do we even have a competence problem? Or is the status quo entirely adequate and not worth revisiting?
• What other post-licensure or “output” measures may be taken to promote lawyer competence? What do you think about regulators obtaining feedback from consumers, judges and peers, or making more frequent (or random) quality assurance visits?
• Is there any basis to explore requiring formal revalidation of lawyers’ credentials at any point? If so, on what basis? (Complaints? Years in practice? Adverse outcomes?)
• Should regulators gather or rely on data to make decisions about competence measures, or is it sufficient to make decisions based on information obtained through complaints and anecdotal evidence from benchers’ experience?

Addendum: Just before publication of this article, I came across the forthcoming article “Do We Need a Bar Exam… for Experienced Lawyers?” by Professor David Adam Friedman, who observes (in the American context) that “The data show that lawyers tend to confront disciplinary problems later in their career, not earlier, yet for some reason, the profession focuses heavily on screening new lawyers”, and recommends “rethinking the entire chain of the licensure admission and retention process”. For those interested in the regulation of continuing competence, his article is worth a read.

___________________________

[1] I recognize that other Canadian law societies’ approaches may vary; in particular, there is a growing trend towards practice readiness courses as a prerequisite to bar admission (e.g. in Alberta, Saskatchewan, Manitoba, and Nova Scotia), which seems to me to be a productive step towards promoting competence in practice, and I understand that the measures in place to monitor practising lawyers’ competence can vary widely by jurisdiction. But for the purposes of this particular column, I believe my overarching points hold true regardless of the details of the approaches taken across the country, I hope it is sufficient to focus on the high-level similarities. I note as well that the various law societies across Canada collaborate on standards of admission to the profession through a national coordinating body, the Federation of Law Societies of Canada (FLSC), and that under the FLSC’s National Mobility Agreement the law societies of all nine common-law provinces will recognize the credentials of each other’s licensees, regardless of where the licensee was initially admitted to practice law.

[2] This point has been made before, of course; see, e.g., Professor Michael J. Trebilcock, “Regulating Legal Competence”, Canadian Business Law Journal, Vol. 34, p. 444, 2001. While the specific regulatory measures taken have been adjusted over the past twenty years, I would respectfully suggest that this overarching theme remains true.

[3] As will be discussed below, I recognize that it is difficult and may not be practically feasible for regulators to gather and rely upon rigorous empirical evidence to support all policy decisions. I make this point not to suggest that the measures in place are necessarily invalid, but to put them in context, and suggest that regulators and licensees should critically examine whether the measures in place are in fact the best available options.

[4] For the purposes of this column I am ignoring administrative requirements for licensure and the “good character” requirement, which goes to professional conduct rather than competence (if anything–but the controversy regarding good character requirements will have to wait for another column).

[5] Dalhousie University, Lakehead University, McGill University, Osgoode Hall Law School, Queen’s University, Thompson Rivers University, Université de Montréal, Université de Sherbrooke, University of Alberta, University of British Columbia, University of Calgary, University of Manitoba, University of Moncton, University of New Brunswick, University of Ottawa, University of Saskatchewan, University of Toronto, University of Victoria, University of Western Ontario, University of Windsor.

[6] Law Society of Ontario, “Guide to Licensing Examinations”, available online: https://lso.ca/becoming-licensed/lawyer-licensing-process/licensing-examinations/guide-to-licensing-examinations. All details about the licensing examination methodology in this section are derived or quoted from the Law Society of Ontario’s official explanation on this website.

[7] LSO, “Licensing Examinations”, Dialogue on Licensing, p. 20. While these materials were previously available at http://www.lsodialogue.ca/, it appears that the webpage was taken down sometime after the consultation process closed and a decision was made in December 2018.

[8] LSO, “Overview of the Lawyer Licensing Process”, Dialogue on Licensing, p. 5. As above, while these materials were previously available at http://www.lsodialogue.ca/, it appears that the webpage was taken down sometime after the consultation process closed and a decision was made in December 2018.

[9] Amy Salyzyn, From Colleague to Cop to Coach: Contemporary Regulation of Lawyer Competence, (2017) 95 Can. Bar. Rev. 489 (2017) at pp. 504-507.

[10] Empirical evidence refers to data obtained through observation and documentation of certain behavior and patterns or through an experiment, used to validate or disprove a hypothesis or claim as part of the scientific method. It is to be distinguished from the application of hunches, “common sense”, or ideology. I am grateful to Noel Semple for observing, however, that there are intermediate options for “evidence” to form the basis for decision-making, such as observations from experience that, while not based in social-science research methods, are more rigorous and useful than a “hunch”.

[11] The LSO identifies and breaks down the “types of allegations raised in complaints received on p. 34 of its 2019 annual report, but unfortunately the types of complaints are aggregated at such a high level that it is difficult to understand to what extent complaints raise true issues of competence; the data states that 48% of complaints received were about “Service issues (examples: fail to account; fail to communicate; fail to serve client)”.

[12] In Ontario, LawPRO (the provider of lawyer licensees’ mandatory errors and omissions insurance) has published fact sheets highlighting the basis for legal malpractice claims, broken down by practice area. These fact sheets suggest that, by and large, many more claims are based on practice management concerns such as communication concerns, clerical errors, and time management than on “errors of law”. These fact sheets provide some useful data and it would be useful to see how claims data aligns with competence complaints and reports made to regulators, and consider how such data can be used to address any mismatch between what clients need in terms of “competence” and what law societies are focusing on in the regulatory measures imposed.