Hostname: page-component-78c5997874-8bhkd Total loading time: 0 Render date: 2024-11-10T17:10:33.927Z Has data issue: false hasContentIssue false
  • English
  • Français

On the limited interchangeability of rule of law measures

Published online by Cambridge University Press:  01 April 2011

Jørgen Møller  and
Svend-Erik Skaaning
Jørgen Møller*
Affiliation:
Associate Professor, PhD, Department of Political Science, Aarhus University, Bartholins Allé, Århus C, Denmark
Svend-Erik Skaaning*
Affiliation:
Associate Professor, PhD, Department of Political Science, Aarhus University, Bartholins Allé, Århus C, Denmark
*
* E-mails: jm@ps.au.dk, skaaning@ps.au.dk
* E-mails: jm@ps.au.dk, skaaning@ps.au.dk
Get access Rights & Permissions [Opens in a new window]

Abstract

During the latest decade, empirical research on the causes and consequences of the rule of law has expanded and, in the process, become extremely influential. However, we show that a number of widely used indices of the rule of law are not interchangeable. This lack of interchangeability is reflected in the fact that they are based on different defining attributes, to some extent cover distinct empirical scopes, do not correlate highly with each other, and support different explanatory factors. Until a consensus has been established with respect to the conceptualization of the rule of law, scholars are thus not free to opt for the measure that fits their data requirements best regarding spatial and/or temporal scope. Instead, they must carefully assess the content validity vis-à-vis their stipulated definition of the rule of law. Given the amount of money and time poured into the rule of law agenda, the problems identified reflect the lack of maturity of ‘good governance’ research.

Keywords

rule of lawindicesinterchangeabilitymissingnesscontent validity
Type
Research Article
Information
European Political Science Review , Volume 3 , Issue 3 , November 2011 , pp. 371 - 394
Copyright
Copyright © European Consortium for Political Research 2011

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Adcock, R. Collier, D. (2001), ‘Measurement validity: a shared standard for qualitative and quantitative research’, American Political Science Review 95: 529–546.CrossRefGoogle Scholar
Alesina, A., Devleeschauwer, A., Easterly, W., Kurlat, S. Wacziarg, R. (2003), ‘Fractionalization’, Journal of Economic Growth 8: 155–194.CrossRefGoogle Scholar
Ali, A.M. (2003), ‘Institutional differences as sources of growth differences’, Atlantic Economic Journal 31: 348–362.CrossRefGoogle Scholar
Alvarez, M., Cheibub, J.A., Limongi, F. Przeworski, A. (1996), ‘Classifying political regime types’, Studies in Comparative International Development 31: 3–36.CrossRefGoogle Scholar
Andrews, J.T. Montinola, G.R. (2004), ‘Veto players and the rule of law in emerging democracies’, Comparative Political Studies 37: 55–87.CrossRefGoogle Scholar
Arndt, C. Oman, C. (2006), The Uses and Abuses of Governance Indicators, OECD: OECD Publishing.Google Scholar
Ashby, N.J. Sobel, R.S. (2008), ‘Income inequality and economic freedom in the U.S. States’, Public Choice 134: 329–346.CrossRefGoogle Scholar
Barahona, D. (2007), ‘The freedom house files’. Monthly Review, March.Google Scholar
Barro, R. (2000), ‘Rule of law, democracy, and economic performance’, in O'Driscoll G.P. Jr., Holmes K.R. and Kirkpatrick M. (eds), 2000 Index of Economic Freedom, Washington and New York: Heritage Foundation and Wall Street Journal, pp. 31–49.Google Scholar
Belton, R.K. (2005), ‘Competing definitions of the rule of law’, Carnegie Endowment. Carnegie Paper no. 55.Google Scholar
Bertelsmann Transformation Index (2006), ‘Bertelsmann Transformation Index’. Retrieved 15 March 2008 from http://www.bertelsmann-transformation-index.de/11.0.html?&L=1Google Scholar
Bleaney, M., Dimico, A. (2008), ‘Biogeographical conditions, the transition to agriculture and long-run growth’. CREDIT Research Paper no. 08/15.Google Scholar
Bolaky, B., Freund, C. (2006). ‘Trade, regulations, and growth’. Paper presented at the IMF Trade and Growth Conference, January 9, 2006, Washington, DC.Google Scholar
Bollen, K.A. (1992), ‘Political rights and political liberties’, in T.B. Jabine and R.P. Claude (eds), Human Rights and Statistics: Getting the Record Straight, Philadelphia: University of Pennsylvania Press,, pp. 188–213.CrossRefGoogle Scholar
Bollen, K. Paxton, P. (2000), ‘Subjective measures of liberal democracy’, Comparative Political Studies 31: 58–86.CrossRefGoogle Scholar
Burnside, C., Dollar, D. (2004), ‘Aid, policies, and growth’. World Bank Policy Research Working Paper no. 3251.Google Scholar
Butkiewicz, J.L. Yanikkaya, H. (2006), ‘Institutional quality and economic growth’, Economic Modelling 23: 648–661.CrossRefGoogle Scholar
Card, D., Freeman, R. (2002), ‘What have two decades of British economic reform delivered?’. Retrieved 7 September 2010 from http://www.nber.org/papers/w8801CrossRefGoogle Scholar
Casper, G. Tufis, C. (2003), ‘Correlation versus interchangeability: the limited robustness of empirical findings on democracy using highly correlated data sets’, Political Analysis 11: 196–203.CrossRefGoogle Scholar
Chomsky, N. Herman, E.S. (1988), Manufacturing Consent. New York, Scottsville: Pantheon Books.Google Scholar
Coppedge, M. Reinicke, W. (1990), ‘Measuring polyarchy’, Studies in Comparative International Development 25: 51–72.CrossRefGoogle Scholar
Dahl, R.A. (1971), Polyarchy: Participation and Opposition, New Haven: Yale University Press.Google Scholar
Doornbos, M. (2003), ‘ “Good governance”: the metamorphosis of a policy metaphor’, Journal of International Affairs 57: 3–17.Google Scholar
Eisenberg, M.A. (1988), The Nature of the Common Law, Cambridge: Harvard University Press.Google Scholar
Fish, M.S. (2002), ‘Islam and authoritarianism’, World Politics 55: 4–37.CrossRefGoogle Scholar
Forewaker, J. Krznaric, R. (2001), ‘How to construct a database of liberal democratic performance’, Democratization 8: 1–25.CrossRefGoogle Scholar
Fraser Institute (2009), ‘Economic freedom of the world’. Retrieved 15 November 2009 from http://www.freetheworld.comGoogle Scholar
Freedom House (2006), ‘Freedom in the world’. Retrieved 15 March 2008 from http://www.freedomhouse.org/template.cfm?page=15Google Scholar
Gerring, J. (1999), ‘What makes a concept good?’, Polity 31: 357–393.CrossRefGoogle Scholar
Giannone, D. (2010), ‘Political and ideological aspects of the measurement of democracy’, Democratization 17: 68–97.CrossRefGoogle Scholar
Goertz, G. (2005), Social Science Concepts: A Users’ Guide, Princeton: Princeton University Press.Google Scholar
Hadenius, A., Teorell, J. (2005). ‘Assessing alternative indices of democracy’. IPSA Concepts & Methods Working Papers 6.Google Scholar
Hansson, G., Olsson, O. (2006), ‘Country size and the rule of law’. Working Paper 200, School of Economics: University of Gothenburg.Google Scholar
Hawken, A., Munck, G.L. (2009), ‘Do you know your data? Measurement validity in corruption research’, Manuscript.Google Scholar
Hayek, F.A. (1973), Law, Legislation and Liberty, Chicago: University of Chicago Press.Google Scholar
Hayo, B., Voigt, S. (2005), ‘Explaining de facto judicial independence’. Marburg Working Papers on Economics 200507, Philipps-Universität Marburg: Department of Economics.Google Scholar
Heritage Foundation (2009), ‘Index of economic freedom’. Retrieved 15 November 2009 from http://www.heritage.org/indexGoogle Scholar
Hoff, K. Stiglitz, J. (2004), ‘After the big bang? Obstacles to the emergence of rule of law in post-communist societies’, American Economic Review 94: 753–763.CrossRefGoogle Scholar
IMF (2007), ‘Guide on resource transparency’. Retrieved 15 March 2008 from http://www.imf.org/external/np/pp/2007/eng/051507g.pdfGoogle Scholar
Joireman, S.F. (2001), ‘Inherited legal systems and effective rule of law’, Journal of Modern African Studies 39: 571–596.CrossRefGoogle Scholar
Joireman, S.F. (2004), ‘Colonization and the rule of law’, Constitutional Political Economy 15: 315–338.CrossRefGoogle Scholar
Kaufman, D., Kray, A., Mastruzzi, M. (2007), ‘Governance matters VI: governance indicators for 1996–2006’. Retrieved 15 March 2008 from http://papers.ssrn.com/sol3/papers.cfm?abstract_id=999979Google Scholar
Knack, S. (1996), ‘Institutions and the convergence hypothesis’, Public Choice 87: 207–228.CrossRefGoogle Scholar
Knack, S. (2002), ‘Governance and growth’. The IRIS Discussion Papers on Institutions & Development no. 2/05.Google Scholar
Koelble, T.A. Lipuma, E. (2008), ‘Democratizing democracy’, Democratization 15: 1–28.CrossRefGoogle Scholar
La Porta, R., Lopez-de-Silanes, F., Shleifer, A. Vishny, R. (1999), ‘The quality of government’, Journal of Law, Economics, and Organization 15: 222–279.CrossRefGoogle Scholar
Mo Ibrahim Foundation (2010), ‘The Ibrahim Index of African Governance’. Retrieved 7 September 2010 from http://www.moibrahimfoundation.org/en/section/the-ibrahim-indexGoogle Scholar
Møller, J., Skaaning, S.-E. (2010), ‘Systematizing thin and thick conceptions of the rule of law’. Paper presented at the APSA 2010 Annual Meeting Paper. from SSRN: http://ssrn.com/abstract=1643367Google Scholar
Moore, D. (ed.) (2007), The World Bank. KwaZulu-Natal University Press.Google Scholar
Mudde, C. Schedler, A. (2010), ‘Introduction: rational data choice’, Political Research Quarterly 63: 410–416.CrossRefGoogle Scholar
Munck, G.L. (2003), ‘Measures of democracy, governance and rule of law’. Paper prepared for the World Bank Workshop on Understanding Growth and Freedom from the Bottom Up, July 15–17, Washington, DC.Google Scholar
Munck, G. (2009), Measuring Democracy, Baltimore: The Johns Hopkins University Press.CrossRefGoogle Scholar
Munck, G. (2010), ‘Comparative politics: taking stock and looking forward’. Paper prepared for presentation at the 2010 APSA Annual Meeting, September 2–5, Washington, D.C.Google Scholar
Munck, G.L. Verkuilen, J. (2002), ‘Conceptualizing and measuring democracy’, Comparative Political Studies 35: 5–34.Google Scholar
Nanda, P. (2006), ‘The “Good Governance” concept revisited’, The ANNALS of the American Academy of Political Science 603: 269–283.CrossRefGoogle Scholar
Norton, S.W. (1998), ‘Poverty, property rights, and human well-being’, Cato Journal 18: 233–245.Google Scholar
Penn World Tables (2007), ‘PWT 6.2’. Retrieved 15 March 2008 from http://pwt.econ.upenn.edu/php_site/pwt_index.phpGoogle Scholar
Political Risk Services (2007), ‘ICRG’. Retrieved 15 March 2008 from http://www.prsgroup.com/ICRG.aspxGoogle Scholar
Ríos-Figueroa, J., Staton, J. (2008), ‘Unpacking the rule of law’. Committee of Concepts and Methods Working Paper no. 21.Google Scholar
Sandholz, W. Taagepera, R. (2005), ‘Coruption, culture, and communism’, International Review of Sociology 15: 109–131.CrossRefGoogle Scholar
Skaaning, S.-E. (2010), ‘Measuring the rule of law’, Political Research Quarterly 63: 449–460.CrossRefGoogle Scholar
Sunde, U., Cervallati, M. Fortunato, P. (2008), ‘Are all democracies equally good?’, Economic Letters 99(3): 552–556.CrossRefGoogle Scholar
Tamanaha, B. (2004), On the Rule of Law, Cambridge: Cambridge University Press.CrossRefGoogle Scholar
The Economist (2008), Economics and the Rule of Law. March 13. http://www.economist.com/node/10849115?story_id=10849115Google Scholar
Thomas, M. (2007), ‘What do the worldwide governance indicators measure?’. Retrieved 1 May 2009 from http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1007527Google Scholar
Uvin, P. (2002), ‘On high moral grounds: the incorporation of human rights by the development enterprise’, PRAXIS 17: 1–11.Google Scholar
Weingast, B.R. (1997), ‘The political foundations of democracy and the rule of law’, American Political Science Review 91: 245–263.CrossRefGoogle Scholar
Williams, A. Siddique, A. (2008), ‘The use (and Abuse) of governance indicators in economics’, Economics of Governance 9: 131–175.CrossRefGoogle Scholar
World Bank (2008), ‘World Development Indicators’. Retrieved 15 March 2008 from http://publications.worldbank.org/onlineGoogle Scholar