Hostname: page-component-78c5997874-dh8gc Total loading time: 0 Render date: 2024-11-15T04:58:01.889Z Has data issue: false hasContentIssue false
  • English
  • Français

Sense and non-sense of a European ranking of law schools and law journals

Published online by Cambridge University Press:  02 January 2018

Rob van Gestel
Rob van Gestel*
Affiliation:
Tilburg University
*
Rob van Gestel, Tilburg University, Graduate School, Warandelaan 2, PO Box 90153, 5000 LE Tilburg, The Netherlands. Email: r.a.j.vangestel@tilburguniversity.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

Rankings of law schools and law journals are part of a trend towards more emphasis in academia on transparency and accountability with regard to the quality of research and education. Globalisation increases the need to compare law schools and law journals across borders, but this raises complicated questions due to differences in language, education systems, publishing style and so on. In this contribution, it is argued that ranking of law schools and law reviews runs the risk of driving us away from quality based on substance towards proceduralisation and quality assessment based on proxies favoured by managers of law schools, funding bodies and government agencies, instead of by the forum of legal scholars.

Type
Research Article
Information
Legal Studies , Volume 35 , Issue 1 , March 2015 , pp. 165 - 185
Copyright
Copyright © Society of Legal Scholars 2015

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

*

Rob van Gestel is professor of theory and methods of regulation at Tilburg Law School (The Netherlands), professor of methodology of legal research at Leuven Law School (Belgium) and Braudel fellow at the European University Institute. The author would like to thank Han Somsen, Mathias Siems, Jan Vranken and the faculty members of the Law Department of the EUI in Florence for their valuable input. The usual disclaimer applies.

References

1. Posner, R ‘Law school rankings’ (2006) 81 Ind L J 13 at 13.Google Scholar

2. See the World Bank Doing Business Reports, available at http://www.doingbusiness.org/rankings (accessed 17 March 2014).

3. Hart, M The 100: A Ranking of the Most Influential People in History (New York: Citadel Press, 2000).Google Scholar

4. The BBC, for instance, organised the 100 greatest Britons in 2002 and in Germany the ZDF organised ‘Unsere Besten’ according to a similar concept. Similar initiatives can be found in a host of other countries.

5. See http://en.wikipedia.org/wiki/William_the_Silent (accessed 17 March 2014).

6. See http://news.bbc.co.uk/2/hi/europe/4015173.stm (accessed 17 March 2014).

7. Comparative lawyers use the term ‘tertium comparationis’ to point to the fact that legal systems that one wants to compare need to be comparative by a third unit apart from the two legal comparanda. This third unit is supposed to be the common denominator. Örücü, AE ‘Methodology of comparative law’ in Smits, J (ed) Elgar Encyclopedia of Comparative Law (Cheltenham: Edward Elgar, 2nd edn, 2012) p 561.Google Scholar

8. Film-maker Theo van Gogh was killed by a Muslim extremist less than 2 weeks before the television show was broadcast.

9. See eg http://www.ratemyprofessors.com/index.jsp and www.myedu.com/sign-up/ (formerly known as ‘pick-a-prof’) (both accessed 17 March 2014).

10. This partly explains the criticism levelled at popular websites such as http://www.ratemyprofessors.com, which are closer to a popularity contest than a serious evaluation of professors' performance in terms of teaching qualities. See http://sites.bio.indiana.edu/~bender/resources/Assessment/fractalevals10.pdf (accessed 17 March 2014).

11. See http://www.theguardian.com/education/2011/jun/27/journals-index-angers-european-academics (accessed 17 March 2014).

12. See the special issue on globalisation and legal education: Chiesa, N, de Luca, A and Maheandiranthe, B (eds) ‘Following the call of the wild: the promises and perils of transnationalizing legal education’ (2009) 10 German L J 629–1168.Google Scholar

13. See the report by the Cohen Committee, Sustainable Humanities (Amsterdam: Amsterdam University Press, 2008) p 41, explaining that in the humanities there is an ‘inadequate range of tools for objectively defining and differentiating quality’.Google ScholarPubMed

14. See B de Witte ‘European Union law: a unified academic discipline?’ (2008) Eui Working Papers Rscas no 34, available at http://cadmus.eui.eu/dspace/handle/1814/10028 (accessed 17 March 2014).

15. See H Morris ‘University ranking shows boom in global student mobility’ International Herald Tribune, 11 September 2012, referring to data gathered by QS, World University Ranking: http://www.topuniversities.com/university-rankings/world-university-rankings/new-look-rankings-show-unstoppable-rise-student-mobili (accessed 17 March 2014).

16. No particular figures seem to be available for legal scholars. For the general trend in all fields of research, see Idea Consult ‘Study on mobility patterns and career paths of Eu researchers’, Final report (Brussels, June 2010), available at http://ec.europa.eu/euraxess/pdf/research_policies/MORE_final_report_final_version.pdf (accessed 17 March 2014).

17. Technically it is possible to generate a European law schools ranking from the QS world ranking, but the credibility of this ranking would be seriously affected by the limited data set and reliance on reputational surveys. It is also possible to use this ranking to rank schools by citation counts on Scopus, but that would also offer a very limited perspective of law school quality. See http://www.topuniversities.com/university-rankings/university-subject-rankings/2013/law-and-legal-studies (accessed 17 March 2014).

18. One could come to an informal ranking for European law journals by using Google Scholar Metrics. See http://scholar.google.com/citations?view_op=top_venues (accessed 17 March 2014). Google Scholar Metrics is a recent initiative, which currently only covers articles published between 2008 and 2012. The metrics are based on citations from all articles that were indexed in Google Scholar in July 2013. It excludes all sorts of citations that would be relevant for law, such as books and dissertations but also court opinions and so on.

19. See http://lawlib.wlu.edu/LJ/ (accessed 17 March 2014).

20. This does not imply that US student-edited journals necessarily publish lower-quality papers than European peer-reviewed journals. Some of the top law journals in the US are student-edited, although some have also introduced (un)official peer review practices for certain types of articles (eg empirical legal research). For a typical example, see http://www.professorbainbridge.com/professorbainbridgecom/2011/08/chicago-law-review-chutzpah.html (accessed 17 March 2014).

21. More generally, it is not always clear why a journal should receive a certain ranking. It can be the number of prescriptions, in which case general interest journals in bigger jurisdictions will always have an advantage over specialist journals in smaller jurisdictions; it can be done on the basis of the rejection rate, which tells little about what is rejected and why; or it can be the number of downloads or citations, which may also have different reasons than quality alone, such as accessibility. The German Law Journal, for instance, became – over a period of 10 years – the world's leading online peer-reviewed law journal in terms of the numbers of citations. See http://germanlawjournal.com/index.php?pageID=1 (accessed 17 March 2014).

22. See http://lawlib.wlu.edu/LJ/index.aspx (accessed 17 March 2014).

23. Weiler, J ‘Impact factor – the food is bad and what's more there is not enough of it’ (2012) 23 (3) Eur J Int'l L 607.Google Scholar

24. Koelman, J and Venniker, R ‘Public funding of academic research: the Research Assessment Exercise of the UK’ in Canton, E et al Higher Education Reform: Getting the Incentives Right (The Hague: Sdu Uitgevers, 2001).Google Scholar

25. See http://www.shanghairanking.com (accessed 17 March 2014).

26. See http://www.topuniversities.com/university-rankings/world-university-rankings (accessed 17 March 2014).

27. See http://www.timeshighereducation.co.uk/world-university-rankings (accessed 17 March 2014).

28. See http://www.leidenranking.com (accessed 17 March 2014).

29. For an interesting comparison of the quite different outcomes of Times and Sunday Times University League Tables 2014 (one of the UK rankings) and the Times Higher Education (THE) World University Rankings 2013–2014 on UK law schools, see Matthias Siems' legal blog of 6 October 2013, available at http://siemslegal.blogspot.nl/2013/10/the-discrepancy-between-uk-and-global.html (accessed 17 March 2014).

30. Expert Group on Assessment of University-Based Research ‘Assessing Europe's university-based research’ (Brussels: Directorate-General for Research, 2010) at 9–10.Google Scholar

31. Even the products of individual legal scholars are getting ranked now. For an interesting example of ranking of individual legal researchers, see Shapiro, F and Pearse, M ‘The most-cited law review articles of all time’ (2012) 110 Mich L Rev 1483.Google Scholar

32. Saunders, N ‘Student-edited law reviews: reflections and responses of an inmate’ (2000) 49 Duke L J 1663 at 1666 with reference to other sources.CrossRefGoogle Scholar

33. Brophy, A ‘The emerging importance of law review rankings for law school rankings, 2003–2007’ (2007) 78 U Colo L Rev 35.Google Scholar

34. What is confusing, however, is that the concept of peer review is used in many different ways. Some reserve the use of the term for the process of subjecting an author's scholarly research to the scrutiny of independent experts in the same field, before a paper is published in a journal. Others see scrutiny by a publisher or an editorial board also as peer review, or include in the definition the refereeing of research proposals for funding.

35. See F van Vught and F Ziegele (eds) ‘U-Multirank: design and testing the feasibility of a multidimensional global university ranking’ (Consortium for Higher Education and Research Performance Assessment, June 2011) at 37.

36. See Siems, M ‘Legal originality’ (2008) 28 (1) Oxford J Legal Stud 147.CrossRefGoogle Scholar

37. Robert, RM Serendipity: Accidental Discoveries in Science (New York: Wiley, 1989).Google Scholar

38. See Smits, J The Mind and Method of the Legal Academic (Cheltenham: Edward Elgar, 2012) p 130.CrossRefGoogle Scholar

39. Numerous examples can be found in: Thornton, M Privatising the Public University: The Case of Law (New York: Routledge, 2012);Google Scholar Bok, D Universities in the Market Place: The Commercialization of Higher Education (Princeton, NJ: Princeton University Press, 2004);Google Scholar Lorenz, C (ed) If You're So Smart, Why Aren't You Rich? Universiteit, markt en management (Amsterdam: Boom, 2008).Google Scholar

40. Osterloh, M and Frey, B ‘Research governance in academia: are there alternatives to academic rankings?’ (2009) CESIFO Working Paper no 2797, at 19–20.Google Scholar

41. The quality of legal research is, for example, probably much strongly determined by the selection and training of young legal scholars than by the monitoring of their output, which of course does not mean that it cannot be meaningful to do this monitoring for other purposes.

42. Pen, M Prestatiemeting van wetenschappelijk onderzoek (Utrecht: Lemma, 2009) pp 206–207.Google Scholar

43. See eg http://vpf-web.harvard.edu/annualfinancial/ (accessed 17 March 2014).

44. European University Association ‘Global university rankings and their impact’, report (Brussels: European University Association, 2011) at 20.Google Scholar

45. Eg Thomson-Reuters Web of Science and Elsevier Scopus.

46. European University Association, above n 44, at 15.

47. See eg Newton, B ‘Preaching what they don't practice: why law faculties' preoccupation with impractical scholarship and devaluation of practical competencies obstruct reform in the legal academy’ (2010) 62 S C L Rev 105.Google Scholar

48. Collier, R ‘We're all socio-legal now?’ Legal education, scholarship and the ‘global knowledge economy’ – reflections on the Uk experience (2004) 26 Sydney L Rev 503;Google Scholar Berard, TJ ‘The relevance of the social sciences for legal education’ (2009) 19 (1/2) Legal Educ Rev 189;Google Scholar Hutchinson, T and Burns, K ‘Kylie, the impact of ‘empirical facts’ on legal scholarship and legal research training’ (2009) 43(2) Law Teacher 153.Google Scholar

49. Smits, J ‘European legal education, or: How to prepare students for global citizenship?’ (2011) 45 Law Teacher 163;CrossRefGoogle Scholar Ulen, T ‘The impending train wreck in current legal education: how we might teach law as the scientific study of social governance’ (2009) 6 U St Thomas L J 302.Google Scholar

50. For an intelligent attempt to integrate different views: Ribstein, L ‘Practicing theory: legal education for the twenty-first century’ (2011) 96 Iowa L Rev 1649.Google Scholar

51. Bartie, S ‘The lingering core of legal scholarship’ (2010) 30 Legal Stud 345 at 367.CrossRefGoogle Scholar

52. However, the nature of doctrinal-legal research is also changing. van Gestel, R and Micklitz, H ‘Revitalising doctrinal legal research in Europe: what about methodology?’ in Neergaard, U, Nielsen, R and Roseberry, L (eds) European Legal Method (Copenhagen: DJOF, 2011) pp 25–75 (also available at http://cadmus.eui.eu/handle/1814/16825, accessed 17 March 2014).Google Scholar

53. Expert Group on Assessment of University-Based Research, above n 30, at 43.

54. See Spitzer, R Saving the Constitution from Lawyers (Cambridge: Cambridge University Press, 2008) pp 22–23.CrossRefGoogle Scholar

55. Thomas, DA ‘The law school rankings are harmful deceptions: a response to those who praise the rankings and suggestions for a better approach to evaluating law schools’ (2003) 40(2) Hous L Rev 419 at 456;Google Scholar Van Alstyne, S ‘Ranking the law schools: the reality of illusion?’ (1982) 7(3) Am Bar Found'n Res J 649 at 679.Google Scholar

56. Ciolli, A and Garber, J ‘How to rank law school rankings?’ (2009) 7(2) Dartmouth L J 132 at 146.Google Scholar

57. Newton, B ‘Law review scholarship in the eyes of the twenty-first century Supreme Court Justices: an empirical analysis’ (2012) 4 Drexel L Rev 399.Google Scholar

58. In the US, many students were misled by law school rankings and made expensive choices to go to study law at certain schools, so that many of them now have severe debts, which will be hard to repay. See inter alia Tamanaha, B Failing Law Schools (Chicago: University of Chicago Press, 2012).CrossRefGoogle Scholar

59. Berger, M ‘Why the Us News & World Report law school rankings are both useful and important’ (2001) 51 J Legal Educ 487 at 498.Google Scholar

60. Korobkin, R ‘Harnessing the positive power of rankings: a response to Posner and Sunstein’ (2006) 81 Ind L J 35 at 45.Google Scholar

61. Posner, above n 1, at 13.

62. For an overview of the problems with the ranking of universities in general, see van Vught and Ziegele, above n 35, at 27–34.

63. Murray, J ‘Do Us law schools that report false or misleading employment statistics violate consumer protection laws?’ (2012) 15(3) J Consumer & Com L 97.Google Scholar See also Kuncel, NR, Credé, M and Thomas, LL ‘The validity of self-reported grade point averages, class ranks, and test scores: a meta-analysis and review of the literature’ (2005) 75(1) Rev Educ Res 63.CrossRefGoogle Scholar

64. Leiter, B ‘How to rank law schools’ (2006) 81 Ind L J 47 at 51.Google Scholar

65. Barton, B ‘Is there a correlation between law professor publication counts, law review citations counts, and teaching evaluations? an empirical study’ (2008) 5(3) J Empirical Legal Stud 619.CrossRefGoogle Scholar

66. Teaching and learning, research, knowledge transfer, international orientation and regional engagement.

67. Instead, the U-Multirank tool uses sunburst charts and field charts. See Van Vugth and Ziegele, above n 35, at 18–19.

68. See House of Lords ‘The modernisation of higher education in Europe’ (London, March 2012) at 24–25.

69. For an overview, see http://www.llm-guide.com/law-school-rankings (accessed 17 March 2014).

70. It seems more likely that student choices will be influenced by a number of issues, including customer service from the administration, tuition fees, the availability of scholarships, discussions on blogs, location, language and so on.

71. Thornton, M ‘The law school, the market and the new knowledge economy’ (2009) 10(7) German L J 641.Google Scholar

72. See http://siemslegal.blogspot.nl/2012/10/uk-law-journal-ranking-rae-2008-data.html (accessed 17 March 2014).

73. See http://research.unsw.edu.au/excellence-research-australia-era-outlet-ranking (accessed 17 March 2014); Ramsay, I and Stapledon, GP ‘A citation analysis of Australian law journals’ (1997) 21 Melb U L Rev 676;Google Scholar Warren, D ‘Australian law journals: an analysis of citation patterns’ (1996) 27 Austl Acad & Res Libr 261.Google Scholar

74. Perry, R ‘Ranking Hebrew law reviews: theoretical foundations and a preliminary empirical study’ (2004) 1 Haifa L Rev 401.Google Scholar

75. George, T and Guthrie, C ‘An empirical evaluation of specialized law reviews’ (1999) 26 Fla St U L Rev 813 at 813;Google Scholar Jarvis, R and Coleman, P ‘Ranking law reviews: an empirical analysis based on author prominence’ (1997) 39 Ariz L Rev 15; R Jarvis and P Coleman ‘Ranking law reviews by author prominence – ten years later’ (2007) 99(3) Law Libr J 573.Google Scholar

76. For an overview, see Korobkin, R ‘Ranking journals: some thoughts on theory and methodology’ (1998–1999) 26 Fla St U L Rev 851.Google Scholar

77. See http://www.theaustralian.com.au/higher-education/end-of-an-era-journal-rankings-dropped/story-e6frgcjx-1226065864847 (accessed 17 March 2014). One example was that research managers immediately started to set targets for publication in A and A* journals.

78. For an example, see http://siemslegal.blogspot.co.uk/2011/03/world-law-journal-ranking-2011.html (accessed 17 March 2014).

79. Campbell, K, Goodacre, A and Little, G ‘Ranking of United Kingdom law journals: an analysis of the Research Assessment Exercise 2001. Submissions and results’ (2006) 33 J L Soc'y 335.CrossRefGoogle Scholar

80. An elaborate list of advantages can be found in Perry, R ‘The relative value of American law reviews: a critical appraisal of ranking methods’ (2006) 11(1) Va J L & Tech 1 at 4–6.Google Scholar

81. Svantesson, J ‘International ranking of law journals – can it be done and at what cost?’ (2009) 29(4) Legal Stud 678 at 685.CrossRefGoogle Scholar

82. Ibid, at 683.

83. Korobkin, above n 76, at 866.

84. In the case of empirical-legal research, how will the data be double-checked?

85. See van Gestel, R and Micklitz, HW ‘Why methodology matters’ Eur L Rev, forthcoming (2014). See http://onlinelibrary.wiley.com/doi/10.1111/eulj.12049/abstract (accessed 17 March 2014). In Europe for a long period throughout history, law as a discipline was a role model for other sciences. SeeGoogle Scholar van Caenegem, R Judges, Legislators and Professors, Chapters in European Legal History (Cambridge: Cambridge University Press, 1987).CrossRefGoogle Scholar

86. See Tamanaha, above n 58, p 71 and further. See also Lorenz, above n 39.

87. With regard to the index, see http://thomsonreuters.com/arts-humanities-citation-index/ (accessed 17 March 2014).

88. Council for the Humanities and the Social Sciences Council ‘Judging research on its merits. an advisory report by the Council for the Humanities and the Social Sciences Council’ (Amsterdam: Royal Academy of Arts and Sciences, 2005) at 17, available at https://www.knaw.nl/shared/resources/actueel/publicaties/pdf/20051029.pdf (accessed 17 March 2014).

89. See http://pf.ujep.cz/user_files/ERIH_zakladni%20informace.pdf (accessed 17 March 2014).

90. Cook, H et al ‘Journals under threat: a joint response from history of science, technology and medicine editors’ (2009) 53(1) Med Hist 1 at 1.Google Scholar

91. Ochsner, M, Hug, S and Daniel, H-D ‘Indicators for research quality for evaluation of humanities research: opportunities and limitations’ (2012) 1 Bibliometrie – Praxis und Forschung at 4.4–4.5.Google Scholar

92. For an overview, see https://www.knaw.nl/shared/resources/actueel/publicaties/pdf/20111024.pdf (accessed 17 March 2014).

93. See Royal Academy of Sciences ‘Quality indicators for research in the humanities’ (Amsterdam, 2011), available at http://www.knaw.nl/shared/resources/actueel/publicaties/pdf/20111024.pdf (accessed 17 March 2014).

94. Hicks, D and Wang, J ‘Towards a bibliometric database for the social sciences and humanities – a European scoping project’ (School of Public Policy, Georgia Institute of Technology, 2009).Google Scholar

95. See the ERiC project, which stands for Evaluating Research in Context: http://www.rathenau.nl/uploads/tx_tferathenau/ERiC_guide.pdf (accessed 17 March 2014).

96. A recent survey among 1325 law professors, 338 student-editors, 215 attorneys and 156 judges in the US shows that all of these groups agree on the fact that important reforms are desperately needed for law reviews. Surprisingly, the vast majority agreed that the introduction of some type of blind peer review would increase the quality of the papers being published and diminish the current biases in the selection process. Wise, R et al ‘Do law reviews need reform? a survey of law professors, student-editors, attorneys and judges’ (2013) 59 Loy L Rev 1.Google Scholar

97. McCormack, N ‘Peer review and legal publishing: what law librarians need to know about open, single-blind, and double-blind reviewing’ (2009) 101:1 Law Libr J 59.Google Scholar

98. Van Hoecke, M (ed) Methodologies of Legal Research: Which Kind of Method for What Kind of Discipline? (Oxford: Hart Publishing, 2011).Google Scholar

99. See Mertz, E The Language of Law School: Learning to Think Like a Lawyer (Oxford: Oxford University Press, 2007).CrossRefGoogle Scholar

100. Spitzer, above n 54, p 23.

101. Ibid, pp 24–25.

102. See http://www.scienceguide.nl/201306/multirank-milestone-reached.aspx (accessed 17 March 2014).

103. See B Tamanaha Failing Law Schools (Chicago: University of Chicago Press, 2012). See also http://lawprofessors.typepad.com/legal_skills/2013/03/the-pernicious-influence-of-the-us-news-law-school-rankings.html (accessed 17 March 2014).

104. Crittenden, J ‘Building better ranking’ (2013) 22(5) Nat'l Jurist 24–27. See http://www.nxtbook.com/nxtbooks/cypress/nationaljurist0213/#/22 (accessed 17 March 2014).Google Scholar

105. Think of issues such as using a (PhD) student work in one's own publications without reference, plagiarism or selling out to sponsors of contract research.

106. In Switzerland, for example, where the Conference of the Swiss Universities (CRUS) has started a project ‘Research evaluation in law’. See http://www.rechtswissenschaft.unibe.ch/unibe/rw_neu/content/e1984/e373454/e373467/e386328/e386329/files386330/Lienhard_Amschwand_HerrmannLeGes2013_2_ger.pdf (accessed 17 March 2014). In The Netherlands, the deans of the joint law schools have established a committee on classification of law journals, chaired by the dean of Maastricht law school. See Du Perron, E ‘De kwalificatie van rechtswetenschappelijke tijdschrift, in van Dijck, G et al (eds) Cirkels: een terugblik op een vooruitziende blik (Deventer: Kluwer, 2013) pp 191–199.Google Scholar