Surprising Survey Results

Last November, Lexum proceeded to survey the users of its Supreme Court of Canada decisions web site (scc.lexum.org/en). The results of such surveys are rarely shared; they can indeed have high commercial value or simply be embarrassing. The ones communicated here are not embarrassing and they are, at times, surprising. At least, they were not the ones Lexum was expecting. For instance, respondents are less eager to access legal content on mobile platforms than Lexum had initially assumed and their appetite for social web-related services appears much more limited than anticipated. In a more foreseeable way, respondents confirmed how the Canadian legal information market is today dominated by the free access to law movement. Here are some highlights from the results. Those interested in methodology will find some details at the end of this post.

Smart Phones and Tablets

Everybody knows that tablets – iPads I mean – are hot, very hot. At the Montreal Auto Show last weekend, all those staffing kiosks had one in hand. In few words, this year, iPads overcame cars at the show. Furthermore, we all enjoy our iPhones. They are cool. Recently, apps available for answering every imaginable need reached half a million. Clearly, if you are in the business of moving content, mobility is the way to go.

The Lexum 2011 survey data reveals a much less mobility-enthralled user base. The most popular demands for improvement were (1) comprehensiveness (all SCC decisions since 1876), (2) addition of hypertext link to references and (3) the addition of PDF versions faithful to the appearance of the Supreme Court Reports (since paragraphs were not numbered in older decisions, users need the page numbers to be able to cite). One must look towards the least chosen options to find the port to mobile platforms. Indeed, amazingly, “accessibility via mobile device” is at the ninth rank out of ten.

Even if we filter the results to consider only the younger respondents, those under 40, the priority for improvements remains essentially unaltered: the “kids” love more than anything else comprehensiveness, hyperlinking and PDFs. With this group of respondents, the mobile access option moves up only two ranks going from the 9^th to the 7^th priority. This comes as an agreeable surprise to Lexum, for it means that it can still discreetly jump on the mobility bandwagon.

Twitter and Facebook

Without a doubt, Lexum’s survival throughout the years does not owe much to its tweeting or cleaver uses of a Facebook page. In this world of hectic twittering and philandering expressions of “Like”-ing, Lexum has been at best a diffident participant. One of Lexum’s intentions with this survey was to give it the opportunity to take stock of its lateness and to help it catch up and find ways to start leveraging new media.

Again, respondents surprised Lexum with their answers; social media integration is the lowest of their priorities. Case in point, less than 4% of respondents chose “social media integration” despite the three chances they had to vote for it.

Slaw is not the forum to downplay the importance of collaboration and user-generated content. How then does one conciliate the usefulness of such forums, which appears so evident to many like me, with the weak demand for integrating social media to the Supreme Court of Canada decisions web site?

Could the notion of cognitive authority (introduced to our field by Professor Robert C. Berring) help explain this lack of interest? Respondents may have felt that the most authoritative content in law, the Supreme Court of Canada’s decisions, does not mix well with the musings one often reads on Twitter, Facebook and even more serious forums. It could be that for our users these other types of content do not register in the same spectrum of cognitive authority and they wisely prefer avoiding the intermingling of highly dissimilar genres.

Most of us, and probably most of Lexum’s users as well, would accept the idea that well designed social media-based environments can support the creation of valuable crowd-produced commentaries about legal issues. Perhaps the lukewarm reception of social media integration on the Lexum Supreme Court of Canada decisions web site can simply be explained by the lack of common knowledge about how social media can serve access to legal information. Nevertheless, the survey results invite the exercise of caution when introducing user-produced content to a site branded as authoritative.

A Distinct Way to Search for Legal Information

For reasons easy to understand, the survey was prepared in both languages and users were offered the choice of linguistic version. Nothing was expected from this very pragmatic arrangement. However, some differences were observed between the responses of the English and the French groups.

French speakers care more for the Supreme Court of Canada’s press releases than their English speaking equivalents (72% of French respondents say they consult them v. 49% of the English respondents). French speaking respondents are also three times more likely to appreciate SMS as a way to be informed of the arrival of new decisions on the Lexum site (11% v. 4%).

Proof that English and French minds don’t think alike, the approaches taken to find a case varies according to the language of the respondent. English respondents are especially fond of searching by case name where their French peers prefer using the reference to the official report. English users also seem more familiar with neutral citations than the French users.

(click on image to enlarge)

Fig 1: To the question “How do you search for information on Lexum’s Supreme Court of Canada decisions web site?” Each respondent picked three out of eight ways to find information on the Lexum site. English respondents’ answers are in blue, French respondents’ in red.

CanLII Leads

However you interpret them, the answers given by the respondents reveal that CanLII tops the market when it comes to accessing primary legal information. In one of its questions the survey asked users to identify their 3 favorite legal information sources out of 7 (5 out of 10 in Quebec, for there is a richer offer of legal information sources in Quebec). CanLII dominates the results across the board.

Both French (81%) and English (80%) respondents concur, making CanLII their favorite web site when looking for legal information. Filtering these results to consider only the lawyers among the respondents, the expressed preference for CanLII grows (E: 83%, F: 81%). Similarly, the younger clientele is slightly fonder of CanLII than the more general group of respondents (E: 83%, F: 84%). Even when one considers the respondents who indicate that they are users of the main commercial publisher of legal information outside of Quebec, respondents keep CanLII as their top favorite (79%) and say they visit it more often than their commercial provider. Similarly, in Quebec, when limiting the respondents to those who are users of the main commercial service here, they say they visit CanLII more often than the commercial provider and CanLII remains their favorite choice to the sound of 80%.

Best Search Tools According to Users

A well-crafted post must be somewhat self-serving; why, I ask you, would anyone participate in social web media otherwise? I am therefore pleased to say that survey respondents found CanLII’s search engine, designed by Lexum, the best on the market.

When bragging, a modest man must thrive to be brief. So, let’s go for it: overall, both English and French respondents, comparing Lexum’s search engine to all others legal information services (6 other choices in the survey for outside of Quebec, and 9 others for Quebec) chose the CanLII search tool as the best of the lot.

Final Thoughts

Lexum’s 2011 Supreme Court of Canada decisions web site survey brought with it some surprising results. It also confirms the place taken by the free access to law legal information sources in our country. It was a survey limited in scope. Many legal publishers and other actors in the arena of legal information survey the field regularly. Let’s hope that they will more regularly share their successes and also some of the facts collected. Our respondents merit as much.

Methodology

The invitation to participate in the survey was found on Lexum’s Supreme Court of Canada’s decisions web site (scc.lexum.org/en) and sent through the related e-mail distribution list, so it could be that the respondents come out of a group more interested in free access to law than the general body of legal information users. The survey itself was very short with only 20 questions. The questions first collected demographic information about respondents. The next group of questions related directly to the Lexum Supreme Court of Canada’s decisions web site: frequency of visits, web site uses, awareness tools preference and opinions on possible improvements. In the last four questions, the survey addressed the more general opinion about the whole of the Canadian legal information system: the respondents’ favourite sites for legal information, how frequently they visit them and which ones offer the best search tool. Finally, respondents were invited to share what they “dislike” about today’s legal information systems.

The survey was offered in both English (574 respondents) and French (224 respondents). In both groups, half of the respondents were lawyers and less than 5% defined themselves as being members of the ‘general public’. The other respondents identified themselves as being other legal professionals (15%), students (10%), academics, and so on. A little more than half were men. The age of respondents seems to be representative of the legal profession, 20% under 30, 25% between 30 and 40, 20% in their forties and 30% over 50. The others were over 65 or refused to answer. The jurisdictional distribution is also well spread out with 42% of the respondents from Ontario, 31% from Quebec, 10% from British Columbia and so on.

Lexum has been publishing the Supreme Court of Canada’s judgments since 1993. This initiative started free access to law almost twenty years ago in Canada. The service is now operated as a free access service by Lexum informatique juridique inc. It will be updated shortly with Lexum’s new Decisia publishing system.

Disclosure: Lexum operates the CanLII web site for the Federation of Law Societies of Canada under a service contract.

Daniel Poulin

Emma Elliott has contributed to the design of the survey.