December 8, 2008
Jordan
Furlong
Re-Engineering Law Schools
by Jordan Furlong
December 8, 2008
Considering that it has the potential to profoundly reshape the nature of American legal education, I’m a little surprised that the Interim Report of the Outcome Measures Committee of the ABA’s Section of Legal Education and Admissions to the Bar hasn’t received more attention since its release in June. Aside from brief mentions at the places you’d expect — Best Practices for Legal Education and the Law Professors Blog Network — I haven’t seen the report and its implications discussed in much detail. So I thought I might take a crack at it.
What follows isn’t really a summary of the report, which is 65 pages long — neither you nor I have that kind of time. And there's a lot of content, such as assessment methodologies and other professional education regimes, that's important but not germane to my present interest: the practicing bar’s long-overdue willingness to play a role in deciding what a legal education should actually accomplish. So with those caveats issued, here we go.
The ABA, for those who don’t know, is in charge of accrediting law schools in the U.S. About 200 schools have received the ABA’s imprimatur so far, and all states recognize that graduating from such schools qualifies a person to take the state bar examination. (Though ABA accreditation is apparently not always necessary for a school's graduate to write a bar exam, as a recent case in Massachusetts decided (HT to Joseph Dang.))
Anyway, in May 2007, the Section’s Accreditation Policy Task Force recommended appointing a committee to consider revising the accreditation process, in order to rely more heavily on “output measures” than on “input measures.” The afore-mentioned committee was struck in October 2007 and issued its preliminary report this past summer.
What do we mean by input and output measures? The report gets right to that in its very first paragraph:
In recent years, there has been a growing sentiment within the legal education community that the Accreditation Standards of the ABA Section of Legal Education and Admissions to the Bar should be reframed to focus more heavily on “outcome measures” – accreditation criteria that concentrate on whether the law school has fulfilled its goals of imparting certain types of knowledge and enabling students to attain certain types of capacities, as well as achieving whatever other specific mission(s) the law school has adopted.
Those who advocate this change maintain that the current Accreditation Standards rely too heavily on “input measures” – accreditation criteria that concentrate on whether law schools are investing the right types and amounts of resources (such as physical plant, number of faculty, and budget) to achieve the goals identified in the accreditation standards and the school’s missions.
This represents a major shift in how the ABA assesses schools’ fitness for accreditation. It’s also — to my mind, anyway — a pretty clearly good one: a school is as good as its graduates, not its infrastructure, especially when they're graduating into a professional environment. And in fact, outcome measures are specifically and approvingly cited in the preamble to the ABA’s Standards and Rules of Procedure for Approval of Law Schools. But in practice, thanks to written standards at odds with this emphasis, input measures have tended to win the day. The ABA clearly is ready to change that, and the report, in its conclusion, paves the way there:
An outcome-oriented approach to accreditation would call for reframing Standard 302 so that, instead of focusing on the areas and types of instruction that the law school should provide, the Standard would instead focus on the types of lessons the students should have learned … by the time of graduation from law school.
But the committee was also instructed to think about output measures themselves, and to look beyond traditional measures like bar passage and employment rates. It was encouraged, in so many words, to rethink what should qualify as meaningful outcome measures of a law school’s effectiveness. To do that, the report relied heavily on two recent watershed works in legal education: Educating Lawyers: Preparation For The Profession Of Law (Carnegie Foundation for the Advancement of Teaching, 2007) and Best Practices For Legal Education: A Vision And A Road Map (Clinical Legal Education Association 2007) [PDF].
The committee turned to the Carnegie Foundation (CF) book to answer the key question: “What outcomes may be regarded as central to the legal education field as a whole?”
To prepare students to be competent professionals, CF ascribes three apprenticeships that should make up their education. The first apprenticeship is the cognitive or intellectual, which provides students with the academic knowledge base. The second apprenticeship is the forms of expert practice shared by practitioners. The third is the apprenticeship of identity and purposes, which introduces the student to the values required of the professional community. … In shorthand, CF describes these three apprenticeships as “knowledge, skills, and attitude.”
You could substitute “values” for “attitude” and end up pretty much at the same destination. A law graduate should possess knowledge of the law, the skills with which to apply that knowledge, and the values or attitudes or judgment to use the first two traits ethically and for the public good. The committee acknowledged that U.S. law schools have over-emphasized the first trait to the detriment of the other two.
From that point, the committee dug in for a great deal of research: looking at accreditation standards in other fields of professional education, studying the work of regional and national accreditation commissions outside legal education, and reviewing submissions by groups such as the Society for the Advancement of Law Teaching. At the end of that process, the committee, as expected, recommended that "the Section of Legal Education and Admissions to the Bar re-examine the current ABA Accreditation Standards and reframe them, as needed, to reduce their reliance on input measures and instead adopt a greater and more overt reliance on outcome measures."
Now, if you’re like me, you’re probably wondering: “So what exactly are the outcome measures that law schools should be measured against?” The committee wondered that too, and asked itself whether the ABA should set out detailed outcome measures when designing those accreditation standards.
But in the end, the committee decided that a law school should be given “considerable flexibility … to determine the outcomes the school seeks to effect (presumably within broad contours established by the Standards and Interpretations) and the mechanisms by which to measure those outcomes.” This is in pursuit of greater innovation and to allow schools more breadth in cost allocation decisions. But it's also in support of the idea that every school should be allowed to define its own “mission,” so long as it’s then held strictly to account for its efforts in achieving it:
[I]n order to provide law schools with the information they need for making decisions about what outcomes to set and how to measure them, the Committee recommends that the Standards Review Committee and the Council adopt Standards and Interpretations that would set very general parameters regarding outcome measures and then flesh those out with commentary setting forth models that have proven successful and that a school could choose to use if it wishes.
Okay, then: what are those parameters? What should those commentaries be? What are the “broad contours” of the Standards and Interpretations within which schools must define their outcomes? The committee wasn’t sure if its mandate included coming up with proposed wording for new Standards and Interpretations. It would like to do so, though, judging from this paragraph at the end of the report:
The committee believes that its presentation of potential wording for any such new Standards and Interpretations might help to clarify the committee’s vision of the new outcome measures and how they would operate in practice. Accordingly, the committee is inclined to present such potential wording in its final report, with the understanding that any such draft Standards and Interpretations would be merely illustrations, offered to the Standards Review Committee as one of many possible approaches to the subject. Barring instructions to the contrary from the Section Chair, the committee intends to present draft language of this sort in its final report.
Nonetheless, over the course of the report, you can glean some of the factors to which the committee refers favourably and some of the principles that it would like to see schools follow. For instance, here’s the committee quoting from Best Practices’ “seven principles for developing outcome assessments”:
• faculty should formulate outcomes in collaboration with the bench and bar;
• outcomes should serve the law school’s mission;
• outcomes should be adopted only after consensus is reached;
• outcomes should be measurable;
• outcomes should be clear, straightforward;
• faculty should choose a reasonable amount of outcomes in terms of resources available
• outcomes should be reasonable in light of the abilities of the students and faculty.
I don’t think it’s a coincidence that the first factor listed — collaborate with the bench and bar — is the first factor listed. There are numerous other excerpts from the report, either quoted from sources or drafted by the committee. Sift through them and note the underlying theme:
- [L]aw schools should shift in assessment from the conceptual knowledge accumulated by students … to the assessment of practical competencies (professional skills) and the development of professional identity.
- Most lawyers spend most of their time trying to solve problems. Those problems consist of raw facts … presented by clients, along with some questions like “Legally speaking, how do I get myself out of this mess?” or “How do I plan my affairs to avoid getting into a mess like this in the first place?” If our job is to teach students how to “think like lawyers,” then we should train them to solve such a problem.
- Direct evidence of student learning outcomes may include: faculty testing, capstone performances and courses; professional and clinical performances; and third-party testing, including licensing examinations.… Indirect evidence of effectiveness include: work samples; graduate surveys; employer ratings for performance; and self-reported growth by graduates.
- The Law Society [of England & Wales] is currently “developing new forms of examination and assessment of those values, skills, and knowledge [that a solicitor should possess on his first day of practice],” which are “intended ‘to ensure that qualification to practice law is based on an individual’s knowledge and understanding of law and legal practice and their ability to deliver legal services to a high quality, rather than on their ability to complete a particular course or courses of study.’”
- The Australian Law Reform Commission … urged an Australian curriculum re-orientation away from the traditional content focus towards skills and values acquisition and training – towards “what lawyers need to be able to do [rather than] anchored around outmoded notions of what lawyers need to know.”
- [T]raditional legal education does not give enough emphasis to other practical skills such as working with clients, managing a file, the business of law practice, negotiations, etc.
- General Motors has developed an assessment instrument to evaluate the performance of its lawyers, using the Lominger Competencies, with respect to [, among other things ...] knowledge (such as, for example, possessing the appropriate legal knowledge, knowing how businesses work and knowledge about current and possible future policies, practices, trends, technology and information affecting the business and the organization; knowing the competition; and awareness of how strategies and tactics work in the marketplace);
Now, this is just an interim report, and on its face, it recommends only what its commissioners asked it to do: shift the ABA’s accreditation focus from input to output measures. But it seems to me that the committee is also cautiously laying the groundwork for a major rethinking of the purpose of U.S. law schools.
This isn't just about shifting from input to output criteria; it's about reframing those output criteria in practical, client-facing ways. The Legal Education and Admissions Section is being asked to think about law school, at least in part, in terms of producing lawyers with knowledge, skills and values geared towards the practical demands of a professional life of client service. I’ll be the first to say that that’s not all a law school should do. As I’ve said before, a law degree is and should remain an intellectual pursuit, “higher learning” in the best sense of the term. But it's also preparation for using that knowledge in the business of law and the service of clients.
The Interim Report of the Outcome Measures Committee is a first step in that direction and is immensely important for that reason. If the committee continues along this path with its final report, recommendations and implementation, it could end up triggering some of the most significant changes ever to the American legal education system and, shortly thereafter, to the American legal profession itself.
Considering that it has the potential to profoundly reshape the nature of American legal education, I’m a little surprised that the Interim Report of the Outcome Measures Committee of the ABA’s Section of Legal Education and Admissions to the Bar hasn’t received more attention since its release in June. Aside from brief mentions at the places you’d expect — Best Practices for Legal Education and the Law Professors Blog Network — I haven’t seen the report and its implications discussed in much detail. So I thought I might take a crack at it.
What follows isn’t really a summary of the report, which is 65 pages long — neither you nor I have that kind of time. And there's a lot of content, such as assessment methodologies and other professional education regimes, that's important but not germane to my present interest: the practicing bar’s long-overdue willingness to play a role in deciding what a legal education should actually accomplish. So with those caveats issued, here we go.
The ABA, for those who don’t know, is in charge of accrediting law schools in the U.S. About 200 schools have received the ABA’s imprimatur so far, and all states recognize that graduating from such schools qualifies a person to take the state bar examination. (Though ABA accreditation is apparently not always necessary for a school's graduate to write a bar exam, as a recent case in Massachusetts decided (HT to Joseph Dang.))
Anyway, in May 2007, the Section’s Accreditation Policy Task Force recommended appointing a committee to consider revising the accreditation process, in order to rely more heavily on “output measures” than on “input measures.” The afore-mentioned committee was struck in October 2007 and issued its preliminary report this past summer.
What do we mean by input and output measures? The report gets right to that in its very first paragraph:
In recent years, there has been a growing sentiment within the legal education community that the Accreditation Standards of the ABA Section of Legal Education and Admissions to the Bar should be reframed to focus more heavily on “outcome measures” – accreditation criteria that concentrate on whether the law school has fulfilled its goals of imparting certain types of knowledge and enabling students to attain certain types of capacities, as well as achieving whatever other specific mission(s) the law school has adopted.
Those who advocate this change maintain that the current Accreditation Standards rely too heavily on “input measures” – accreditation criteria that concentrate on whether law schools are investing the right types and amounts of resources (such as physical plant, number of faculty, and budget) to achieve the goals identified in the accreditation standards and the school’s missions.
This represents a major shift in how the ABA assesses schools’ fitness for accreditation. It’s also — to my mind, anyway — a pretty clearly good one: a school is as good as its graduates, not its infrastructure, especially when they're graduating into a professional environment. And in fact, outcome measures are specifically and approvingly cited in the preamble to the ABA’s Standards and Rules of Procedure for Approval of Law Schools. But in practice, thanks to written standards at odds with this emphasis, input measures have tended to win the day. The ABA clearly is ready to change that, and the report, in its conclusion, paves the way there:
An outcome-oriented approach to accreditation would call for reframing Standard 302 so that, instead of focusing on the areas and types of instruction that the law school should provide, the Standard would instead focus on the types of lessons the students should have learned … by the time of graduation from law school.
But the committee was also instructed to think about output measures themselves, and to look beyond traditional measures like bar passage and employment rates. It was encouraged, in so many words, to rethink what should qualify as meaningful outcome measures of a law school’s effectiveness. To do that, the report relied heavily on two recent watershed works in legal education: Educating Lawyers: Preparation For The Profession Of Law (Carnegie Foundation for the Advancement of Teaching, 2007) and Best Practices For Legal Education: A Vision And A Road Map (Clinical Legal Education Association 2007) [PDF].
The committee turned to the Carnegie Foundation (CF) book to answer the key question: “What outcomes may be regarded as central to the legal education field as a whole?”
To prepare students to be competent professionals, CF ascribes three apprenticeships that should make up their education. The first apprenticeship is the cognitive or intellectual, which provides students with the academic knowledge base. The second apprenticeship is the forms of expert practice shared by practitioners. The third is the apprenticeship of identity and purposes, which introduces the student to the values required of the professional community. … In shorthand, CF describes these three apprenticeships as “knowledge, skills, and attitude.”
You could substitute “values” for “attitude” and end up pretty much at the same destination. A law graduate should possess knowledge of the law, the skills with which to apply that knowledge, and the values or attitudes or judgment to use the first two traits ethically and for the public good. The committee acknowledged that U.S. law schools have over-emphasized the first trait to the detriment of the other two.
From that point, the committee dug in for a great deal of research: looking at accreditation standards in other fields of professional education, studying the work of regional and national accreditation commissions outside legal education, and reviewing submissions by groups such as the Society for the Advancement of Law Teaching. At the end of that process, the committee, as expected, recommended that "the Section of Legal Education and Admissions to the Bar re-examine the current ABA Accreditation Standards and reframe them, as needed, to reduce their reliance on input measures and instead adopt a greater and more overt reliance on outcome measures."
Now, if you’re like me, you’re probably wondering: “So what exactly are the outcome measures that law schools should be measured against?” The committee wondered that too, and asked itself whether the ABA should set out detailed outcome measures when designing those accreditation standards.
But in the end, the committee decided that a law school should be given “considerable flexibility … to determine the outcomes the school seeks to effect (presumably within broad contours established by the Standards and Interpretations) and the mechanisms by which to measure those outcomes.” This is in pursuit of greater innovation and to allow schools more breadth in cost allocation decisions. But it's also in support of the idea that every school should be allowed to define its own “mission,” so long as it’s then held strictly to account for its efforts in achieving it:
[I]n order to provide law schools with the information they need for making decisions about what outcomes to set and how to measure them, the Committee recommends that the Standards Review Committee and the Council adopt Standards and Interpretations that would set very general parameters regarding outcome measures and then flesh those out with commentary setting forth models that have proven successful and that a school could choose to use if it wishes.
Okay, then: what are those parameters? What should those commentaries be? What are the “broad contours” of the Standards and Interpretations within which schools must define their outcomes? The committee wasn’t sure if its mandate included coming up with proposed wording for new Standards and Interpretations. It would like to do so, though, judging from this paragraph at the end of the report:
The committee believes that its presentation of potential wording for any such new Standards and Interpretations might help to clarify the committee’s vision of the new outcome measures and how they would operate in practice. Accordingly, the committee is inclined to present such potential wording in its final report, with the understanding that any such draft Standards and Interpretations would be merely illustrations, offered to the Standards Review Committee as one of many possible approaches to the subject. Barring instructions to the contrary from the Section Chair, the committee intends to present draft language of this sort in its final report.
Nonetheless, over the course of the report, you can glean some of the factors to which the committee refers favourably and some of the principles that it would like to see schools follow. For instance, here’s the committee quoting from Best Practices’ “seven principles for developing outcome assessments”:
• faculty should formulate outcomes in collaboration with the bench and bar;
• outcomes should serve the law school’s mission;
• outcomes should be adopted only after consensus is reached;
• outcomes should be measurable;
• outcomes should be clear, straightforward;
• faculty should choose a reasonable amount of outcomes in terms of resources available
• outcomes should be reasonable in light of the abilities of the students and faculty.
I don’t think it’s a coincidence that the first factor listed — collaborate with the bench and bar — is the first factor listed. There are numerous other excerpts from the report, either quoted from sources or drafted by the committee. Sift through them and note the underlying theme:
- [L]aw schools should shift in assessment from the conceptual knowledge accumulated by students … to the assessment of practical competencies (professional skills) and the development of professional identity.
- Most lawyers spend most of their time trying to solve problems. Those problems consist of raw facts … presented by clients, along with some questions like “Legally speaking, how do I get myself out of this mess?” or “How do I plan my affairs to avoid getting into a mess like this in the first place?” If our job is to teach students how to “think like lawyers,” then we should train them to solve such a problem.
- Direct evidence of student learning outcomes may include: faculty testing, capstone performances and courses; professional and clinical performances; and third-party testing, including licensing examinations.… Indirect evidence of effectiveness include: work samples; graduate surveys; employer ratings for performance; and self-reported growth by graduates.
- The Law Society [of England & Wales] is currently “developing new forms of examination and assessment of those values, skills, and knowledge [that a solicitor should possess on his first day of practice],” which are “intended ‘to ensure that qualification to practice law is based on an individual’s knowledge and understanding of law and legal practice and their ability to deliver legal services to a high quality, rather than on their ability to complete a particular course or courses of study.’”
- The Australian Law Reform Commission … urged an Australian curriculum re-orientation away from the traditional content focus towards skills and values acquisition and training – towards “what lawyers need to be able to do [rather than] anchored around outmoded notions of what lawyers need to know.”
- [T]raditional legal education does not give enough emphasis to other practical skills such as working with clients, managing a file, the business of law practice, negotiations, etc.
- General Motors has developed an assessment instrument to evaluate the performance of its lawyers, using the Lominger Competencies, with respect to [, among other things ...] knowledge (such as, for example, possessing the appropriate legal knowledge, knowing how businesses work and knowledge about current and possible future policies, practices, trends, technology and information affecting the business and the organization; knowing the competition; and awareness of how strategies and tactics work in the marketplace);
Now, this is just an interim report, and on its face, it recommends only what its commissioners asked it to do: shift the ABA’s accreditation focus from input to output measures. But it seems to me that the committee is also cautiously laying the groundwork for a major rethinking of the purpose of U.S. law schools.
This isn't just about shifting from input to output criteria; it's about reframing those output criteria in practical, client-facing ways. The Legal Education and Admissions Section is being asked to think about law school, at least in part, in terms of producing lawyers with knowledge, skills and values geared towards the practical demands of a professional life of client service. I’ll be the first to say that that’s not all a law school should do. As I’ve said before, a law degree is and should remain an intellectual pursuit, “higher learning” in the best sense of the term. But it's also preparation for using that knowledge in the business of law and the service of clients.
The Interim Report of the Outcome Measures Committee is a first step in that direction and is immensely important for that reason. If the committee continues along this path with its final report, recommendations and implementation, it could end up triggering some of the most significant changes ever to the American legal education system and, shortly thereafter, to the American legal profession itself.
Respond: make a comment | read the 1 comment
Share: Email | Save as PDF | Print
| Bookmark & Share |
| 
More columns on Columns: Practice of Law | from Jordan Furlong
|
the count:
8226 posts | 11407 comments
recent comments  
There is good leagal content that doesn’t necessarily come in the neat packages that we usually look in. Though our commercial legal database subscriptions have linked, vetted, edited, and easily. […] »»Research When you need to collaborate on a document displayed on your screen, it’s great to have a colleague from down the hall come into your office and look over your … »»Technology You should assess whether you can accept the financial risks associated with taking the matter, just as clients will assess whether they can (and will) pay your fee. Spend time at the beginning of the. […] »»Practice
-
Available online today are four new chapters of the publication Women in Canada: A Gender-based Statistical Report, which explores the socio-demographic and economic circumstances of Canadian women in general.
-
The bill amends the Constitution Act, 1867 by readjusting the number of members and the representation of the provinces in the House of Commons.
-
-
Blueseed plans to buy a ship and turn it into a floating incubator anchored in international waters off the coast of California.
-
Under Prime Minister Stephen Harper, the flow of information out of Ottawa has slowed to a trickle.
-
-
"…the IPC has exclusive jurisdiction to decide whether a record is in the custody or control of a university in the context of an access request…"
-
-
John J.L. Hunter, Q.C. of Vancouver has been elected President for 2011-2012
-
Detailed results from 321 members.
These summaries of selected recent cases are provided each week to Slaw by Maritime Law Book.
More information.
-
Banks and Banking - Liability of banks to third parties - Negligence - General
The plaintiffs were the former shareholders of a company that failed. They sued the defendant bank alleging that it breached its contract with the company and the plaintiffs and breached a duty ...
-
Actions - Cause of action - General principles - New or extended cause of action - Opening of floodgates
The plaintiff and defendant worked at different branches of the same bank. The defendant’s common-law husband was the plaintiff’s ex-husband. Over a four year period, the defendant ...
-
Aliens - Definitions and general principles - Immigration consultants
The Canadian Society of Immigration Consultants (CSIC) had been designated as the sole regulatory body of immigration consultants in Canada from 2004 until June 2011. On June 30, 2011, Bill C-35 came into force, which significantly amended ...
-
Criminal Law - Sexual offences, public morals and disorderly conduct - Public morals - Obscenity - Possession of child pornography
The accused was convicted of making child pornography available and two counts of possession of child pornography (see [2010] Sask.R. Uned. 197). Subsequently, he was sentenced ...
-
Criminal Law - Procedure - Charge or directions - Jury or judge alone - Directions regarding pleas or evidence of witnesses, co-accused and accomplices
Rowe was convicted by a jury of five offences. He appealed.
The Ontario Court of Appeal allowed ...
-
Narcotic Control - Offences - Possession - General
The accused wished to access marijuana for medicinal purposes but did not have an authorization to possess marijuana issued under the Marihuana Medical Access Regulations. He was notified that a package of marihuana addressed to him had been ...
-
Narcotic Control - General - Legislation - Exemptions - Medicinal marijuana
McCrady, who had an application pending under the Marihuana Medical Access Regulations (MMAR) to possess and grow marijuana, was convicted of possession of marijuana (Controlled Drugs and Substances Act (CDSA), s. 4(1)). Hearn pleaded guilty ...
-
Criminal Law - Sentence - Trafficking in hashish or marijuana (incl. possession for purposes of trafficking)
The accused pleaded guilty to one count of possession of marijuana for the purpose of trafficking. He was sentenced to 30 days’ imprisonment to be served intermittently and 11 months’ ...
-
Municipal Law - Powers of municipalities - Particular powers - Imposition and collection of taxes or fees
Catalyst Paper Corp. operated a paper mill in the District of North Cowichan. Catalyst objected to the tax rate that it paid compared to residential ratepayers. In 2009, the ...
This is a listing of a few upcoming events in Canada of interest to lawyers, law students, legal librarians, and others involved in the practice of law.
Clicking on any event in the list below will give you access to more information and to links allowing you to see the full entry and to add the event to your own calendar.
Click this link for a fuller version of the TalkLaw/ParLoi calendar of events and for instructions as to how to add events and calendars to your own calendar.
|
I should add, by the way, that I had intended to also look at a consultation paper released by the Federation of Law Societies on the Canadian common-law degree in this article, but I couldn't manage it in time. It makes for an interesting compare-and-contrast with the ABA piece, but it's not yet an actual report, just a consultation paper at this stage. Definitely worth a close read, though.