The hierarchy in economics and its implications

Abstract

This paper argues for two propositions. (I) Large asymmetries of power, status and influence exist between economists. These asymmetries constitute a hierarchy that is steeper than it could be and steeper than hierarchies in other disciplines. (II) This situation has potentially significant epistemic consequences. I collect data on the social organization of economics to show (I). I then argue that the hierarchy in economics heightens conservative selection biases, restricts criticism between economists and disincentivizes the development of novel research. These factors together constrain economics’ capacity to develop new beliefs and reduce the likelihood that its outputs will be true.

Introduction

Is the way that economics is organized conducive to the production of economic knowledge?

James Heckman and Sidharth Moktan (2020) recently highlighted the dominance of economics’ ‘Top 5’ journals. Others have noted the outsize representation of economists from top-ranked departments among the authors and editors of those journals (Fourcade et al. 2015; Colussi 2018). I collect these issues together with others to highlight the many asymmetries of power, status and influence that exist between economists. In addition to (i) the dominance of the Top 5 and the concentration of (ii) authors and (iii) editors from a few universities in those journals, the top-ranked departments also train most of the discipline’s (iv) governors and (v) awardees, (vi) individual star economists dominate networks of coauthorship and (vii) the discipline exhibits a strong prestige factor in hiring. Together these asymmetries constitute the hierarchy in economics.

I give reasons to believe that the hierarchy in economics is both steeper – the asymmetries are greater – than it could be and steeper than hierarchies in other fields. I then highlight four reasons to worry about this increased degree of hierarchy in economics. Through (a) reinforcing conservative selection biases and (b) disincentivizing innovation, the steeper hierarchy in economics constrains the development of new beliefs from the discipline. By (c) restricting the exploration of alternatives, the steeper hierarchy reduces the justification we have for believing the outputs of economics. By (d) discouraging criticism, the steeper hierarchy makes it less likely that errors and faulty reasoning will be spotted. This reduces the likelihood that the outputs of economics will be true and so further reduces the justification we have for believing them. My descriptions of (a–d) will be qualitative. I will describe how the present organization of economics leads to (a–d) and describe the negative impact (a-d) have on the production of economic knowledge. I will not measure the effect size of (a–d) or weigh them off against trade-offs. My argument will, consequently, not constitute an all-things-considered judgement on the health of economics. The point is rather to describe the asymmetries that exist between economists (i–vii) and to spell out the mechanism by which these social features of economics impact the epistemic virtues of its outputs (a–d).

This way of proceeding achieves two things. First, by triangulating data from a range of sources, my overview of the asymmetries that exist between economists provides a wide-angle view on the issue of hierarchy in the discipline. Large disparities of power, status or influence due to social positioning are a general feature of economics that are manifested in many different factors, and not just an artefact of the Top 5. Those considering how the organization of economics can be improved should, consequently, cast their eyes beyond narrowly focusing on the dominance of the discipline’s top journals, and consider the many ways in which asymmetries of power, status and influence are manifested together.

Second, although many social scientists have studied the social features of economic research, they have not discussed the normative implications of their findings in much detail. At the same time, philosophers of economics – the subfield that tends to discuss the practices in economics from a normative perspective – have been surprisingly mute on the social organization of the discipline. Philosophers have developed detailed arguments on particular methods, assumptions and measures used by economists (Mireles-Flores 2018). This is good work. But the absence of debate on the discipline-wide social determinants of economic knowledge is a mistake (Alexandrova et al. 2021). Sociologists, historians and economists have mapped out social features of economics that impact issues philosophers can and should comment on – political representation, justification, bias and reliability, to name just a few. This paper rectifies that omission by offering a starting point for discussing what, if anything, is problematic about the hierarchy in economics.

The paper proceeds as follows. In section 1, I describe the asymmetries of power, status and influence between economists (i–vii). In section 2, I offer four reasons to worry about the impact these asymmetries have on the production of economic knowledge (a–d). In section 3, I conclude by noting that although (i–vii) and (a–d) provide reasons to worry about the present state of hierarchy in economics, they do not provide an all-things-considered judgement on the health of the discipline. I also discuss how (i–vii) may be changed to mitigate (a–d).

1. Economics is hierarchical

An organizational hierarchy is typically defined as a partial ordering of individuals, along one or more socially important dimensions, which effects the distribution of power, status and/or influence among those individuals (Magee and Galinsky 2008; Anderson and Brown 2010). The steepness of a hierarchy is normally defined as the degree of asymmetry between the individuals at its different rungs. The main claim I want to make in this paper is that the present steepness of economics’ hierarchy enables a collection of epistemically problematic incentives and social dynamics. To make this argument, I first describe the ways in which the distribution of power, status and/or influence among economists is highly asymmetrical.

1.1. The Top 5

Publications (or lack thereof) in top journals play a central role in career success in most academic disciplines. In economics this is more acute, with five journals particularly dominant – they are the American Economic Review (AER), Quarterly Journal of Economics (QJE), Econometrica (ECMA), Journal of Political Economy (JPE) and Review of Economic Studies (ReStud).¹ Many economics departments in the USA set targets for junior academics to publish in the ‘Top 5’, with tenure often resting on little else.^{2, 3} Heckman and Moktan (2020) show that at the top 15 US News World report ranked economics departments, success in tenure decisions is strongly linked to Top 5 publications, with non-Top 5 papers playing a much less significant role. ⁴ Non-Top 5 journals are valued more highly outside the best ranked departments. But even then, Top 5 articles have much more impact. Controlling for total publications, gender, number of co-authors, ranking of graduate school and citations, Heckman and Moktan find that “it is more valuable to have a mediocre publication portfolio with T5 [Top 5] publications than an outstanding portfolio without any T5s” (2020: 429).

The importance of the Top 5 does not end at tenure. Top 5 papers are also instrumental in securing pay rises, research grants, requests for professional advice and speaking invitations. One would be hard pressed to find an economist who is not aware of the significance of the Top 5. This is highlighted in survey data. When asked to rank the influence of different factors in tenure and promotion decisions, economists choose Top 5 publications as the most important factor for success in all key career milestones – above letters or recommendation, non-Top 5 publications, citations, teaching evaluations, grants, book chapters and books (Heckman and Moktan 2020).

1.2. Authors and editors

Given the significance of the Top 5, it is striking that the authors in these journals are concentrated at a small number of universities. Colussi (2018) finds that, between 2000 and 2006, 25% of the authors in the top four economics journals – the Top 5 minus ReStud – were employed at just six universities (Harvard, Chicago, MIT, Stanford, Berkeley and Princeton), while 47% got their PhDs from the same universities. Baghestanian and Popov (2017) report a similar result and note that alma mater rankings are statistically significant predictors of the likelihood of publishing in the Top 5.

The cases of individual journals and universities are particularly striking. Wu (2007) shows that, between 2000 and 2003, 14% of the pages of the JPE were authored by economists from the University of Chicago (where the journal is based) and that 15% and 13% of the pages of QJE were authored by economists from MIT and Harvard, respectively (both in Cambridge, MA, where the journal is based). The eight departments that provide the most authors to these journals account for 40% of the pages of JPE and 58% of the pages of QJE. Even more striking, Colussi (2018) finds that those with PhDs from Harvard and MIT accounted for just under 50% of the authors in QJE in a similar period (2000–06). Updating these findings, Heckman and Moktan (2020) find that, between 2000 and 2016, 11.9% of the papers in the AER and 24.7% of the papers in QJE had at least one author affiliated with Harvard. They also found 14.3% of JPE papers had authors affiliated with Chicago.

Colussi (2018) suggests that this concentration of authorship is caused by connections to editors. The same six universities (Harvard, Chicago, MIT, Stanford, Berkeley and Princeton) provided 56% of the editors of the top four journals between 2000 and 2006. Moreover, PhDs from those universities accounted for 64% of the editors of the top four journals in the same period. The roles of MIT and Harvard PhDs are particularly striking, given that they provide 31% and 13% of the editors of these journals, respectively. That means that 44% of the editors of the top four journals got their PhDs from just two institutions based in the same town and thought to have similar philosophies. Even if Colussi’s claim that this concentration of editors plays a causal role in the concentration of authors is incorrect, a picture emerges in which those that publish in and edit the most significant economics journals mainly come from the top departments. Even if there were no unfair advantages gained, that is even if all articles submitted were judged according to the same standards, those connected to just a few US departments seem to be afforded larger voices in economics’ premier journals. Fourcade et al. (2015) note that, although top sociology departments provide the most authors to the top sociology journals, the degree of centralization in economics is greater. ⁵

1.3. Governance

The influence of the few universities mentioned already is also evident in the governance of economics. Hoover and Svorenčík (2020) show that economists with their highest degrees from and/or employed at Harvard, MIT, Chicago, Columbia, Stanford and Princeton have dominated the ‘electoral pool’⁶ of the American Economic Association (AEA) since its inception. Moreover, they dominate more now than they did in the middle of the 20th century. Economists educated at Harvard, MIT, Chicago and Stanford, for example, accounted for just over 58% of the electoral pool between 1985 and 2019. Between 1950 and 1984 the four universities that provided the most candidates accounted for just over 50%.

Given that it is those that make it on to the Executive Committee that are the most influential, it is worth noting that between 1985 and 2019 just over 54% of the Executive Committee were employed at six universities, with just over 75% educated at the same six (Hoover and Svorenčík 2020: 85). ⁷ Fourcade et al. (2015) compare similar findings to the professional associations of other social sciences. Between 2010–14 they find that 72% of the AEA’s Executive Committee came from the top five economics departments. This compares with 12% for the American Political Science Association (APSA) and 20% for the America Sociological Association (ASA).⁸

This centralization of governance is not without consequence. The AEA controls several of the most important economics journals, including the AER, Journal of Economic Literature (JEL) and Journal of Economic Perspectives (JEP). The AER is particularly notable because it accounts for a large chunk of the Top 5 papers published – between 35% and 41% for each year between 2011 and 2017 (Card and DellaVigna 2018). The association also plays a significant role in the job market, runs the annual meeting of the Allied Social Science Associations (ASSA) and hands out the prestigious John Bates Clark Medal. ⁹ After election, the AEA’s President-Elect also appoints the Nomination Committee for the following year’s executive committee elections. These factors together give the AEA leadership a large say in the individuals, themes and research that get attention within economics (Fourcade et al. 2015).

1.4. Individuals and awards

Famous individuals play a big role in economics. Goyal et al. (2006) highlight this by showing that the network of co-authorship in the discipline is highly unequal and dominated by ‘interlinked stars’. The 100 most connected economists average 25 co-authors across a five-year period, whereas the average economist has 1.67 co-authors (a difference of 10 times the standard deviation). Well-connected stars score low on ‘clustering’, meaning that their collaborators tend to not collaborate with one another. They also score highly on ‘inbetweenness’, meaning that they frequently provide the shortest links between others in the network. These factors have two likely effects. Firstly, the role star economists play in connecting others gives them outsize influence on the research questions, frameworks and methodologies in the discipline. Secondly, the output of stars likely benefits from their networks and reinforces their influence. ¹⁰

One way that individuals gain outsize influence in economics is via prizes. Offer and Söderberg (2016) show that those that are awarded Nobel prizes in economics are typically afforded greater voice and authority in the discipline, and that laureates receive significant citation boosts. Cherrier and Svorenčík (2020) note that defining figureheads who “could speak with authority on behalf of economists” ¹¹ was an important part of the reasoning for the creation of the John Bates Clark Medal.

Given the extra voice afforded to winners, it is instructive to note that those that receive the top prizes typically come from a small collection of universities. Of the 40 John Bates Clark Medallists (up to 2018), one quarter got their undergraduate degrees from Harvard, half got their doctorates from Harvard or MIT (10 each) and almost 90% were employed at one of six universities at the time of their awards (Harvard (9), MIT (9), Chicago (7), Princeton (4), Stanford (3), Berkeley (3)) (Cherrier and Svorenčík 2020). ¹² Nobel prize winners have more varied alma maters – with 29 institutions training the 86 laureates (up to 2020). This is in part because the winners of the early prizes were trained before the discipline was as dominated by US departments as it is now (even then Harvard, MIT and Chicago account for 34 of the 86). Given this and given that Nobels are typically awarded late career, it is instructive to look at the doctoral institutions of recent laureates to get an idea of the present concentration of power in economics. Focussing on 20 years of awards from 2001, just under half of the 40 laureates got their doctorates from Harvard (10) or MIT (9). By comparison, in the same period 54 Nobels in physics were awarded, with the two universities that trained the most winners accounting for just nine of them (Harvard (5) and Berkeley or Nagoya (both 4)).¹³ The Gini coefficient for the spread of where economics laureates were trained between 2001 and 2020 is 0.50, for physics it is 0.30.¹⁴

The domination of a small section of individual economists is not isolated to the academy. It also plays a role in who is heard in policy. Economists from the big universities dominate political appointments and policy forums. Rubinstein (2016), for example, draws attention to the recently created Booth Initiative on Global Markets (IGM) – a panel that is occasionally asked for an opinion on specific policy matters. The IGM declares that it includes distinguished scholars familiar to economists and the media. As Rubinstein notes “[A]ll fifty-one experts (yes, all of them) come from six universities (and you guessed them correctly: Harvard, MIT, Stanford, Yale, Princeton, and Chicago)” (2016: 166).

1.5. The job market

Hierarchy also plays a significant role in hiring within economics. Han (2003) shows that a ‘prestige principle’ – where departments hire from those similarly ranked or above – exists in many disciplines, but notes that economics is unique in two ways. First, the main ‘faction’ is more dominant in economics. There is one ordering, whereas other disciplines contain parallel orderings that may interpret prestige differently. Second, the prestige principle is stronger in economics.¹⁵ Top economics departments are more likely to hire students from other top departments than are top departments in sociology, political science, history, psychology, English and mathematics.

For those that do not train at the top departments, the prestige factor in hiring is compounded by Oyer’s (2006) claim that the ranking of an economist’s first job has a causal effect on career success. He finds that candidates that initially place into tenure track jobs are 55% more likely to secure tenure in a later year. And candidates that initially place into a top 50 department are 60% more likely to be at a top 50 department in a later year. ¹⁶

Oyer suggests that this first job effect is in part caused by initial placements impacting productivity. He estimates that being placed in a school ranked 13th rather than 16th equates to one extra paper in 10 years for an economist who would normally publish 10–15 papers in 10 years. He also estimates that being placed at a top 50 school increases the probability of publishing in the Top 5 by 50%. But he notes that this explanation is not fully satisfactory. Indeed, Kim et al. (2009) have contested the idea that elite universities provide productivity gains. (They argue that most of the productivity gains that elite universities historically provided have dissipated due to changes in communication technology.) It is plausible that placing into a stronger department causes candidates to settle on more successful work patterns and gives them access to more fruitful ideas and collaborations. But it is also plausible that later search committees do not consider the possibility that a candidate’s first placement may be in part due to how strong the job market was when they graduated, and instead see the higher prestige of their initial placement as a sign of underlying quality. An alternative explanation might be that other factors that make one attractive to search committees are increased by getting a higher ranked first job. A candidate that places into a higher ranked job is, for example, more likely to have ties to a Top 5 editor or someone involved in the governance of the AEA.

2. Reasons to worry

(i) Career advancement in economics is dominated by five journals. Economists connected to a small set of universities are much more likely to (ii) publish in or (iii) edit those journals. (iv) Economics’ premier governing body, the AEA, is dominated by those trained at the same universities. (v) Awards that single out individuals to speak for the discipline tend to go to those trained at the same small group of universities. ¹⁷ (vi) A few star economists benefit from being central in networks of co-authorship. (vii) Economics exhibits a strong prestige factor in hiring, based on a single hierarchy, and the success of an economist’s career is affected by the rank of the institution that first hires them. Collectively these factors paint a picture in which some economists, some institutions and some journals are afforded more significance than others. Those at higher rungs of economics’ hierarchy are more likely to engage in productive networks, more likely to be afforded larger platforms, more likely to win awards, more likely publish in and edit the big journals and more likely to have their influence felt in the running of the AEA. When combined with the prestige factor in hiring and the fact that an economist’s first job has an impact on their later career progression, a pattern emerges in which those that get PhDs from the top universities – and likely degrees from the top universities, and come from privileged backgrounds before that (Chetty et al. 2017) – find it easier to secure influential roles, have their voices heard and reshape the discipline.¹⁸

What should we conclude from this? What, if anything, is wrong with the hierarchy in economics? Many of those cited above present their data with the clear implication that the issue they describe is a problem, but without a detailed discussion of what the problem is (see, for example, Fourcade et al. 2015). That is the gap I intend to fill. But before doing so, it is helpful to be clear on exactly what the phenomenon we are evaluating is.

Any partially ordered set can be a hierarchy. Given that no academic disciplines have perfectly equal distributions of power, status and influence, all are hierarchical to some degree. It should not be surprising that economics is also. I suggest that the data outlined above is interesting not because it shows that there is a hierarchy in economics, but because it shows that said hierarchy is notably steep. I mean two things by this. First, the hierarchy in economics seems to be steeper than in other disciplines. Where comparative data are available, it shows there are greater asymmetries between economists than between academics in other disciplines (this is the case for the asymmetries in publishing, governance, awards and the job market). Secondly, economics is more hierarchical than it could plausibly be. The discipline could be much less hierarchical without being perfectly equal. Tenure committees could do more to look beyond the Top 5, without completely jettisoning the idea that some journals are better than others. A wider distribution of talent could be represented in editorial roles and the AEA’s governing body, without assuming that all universities are equal. And economists could partly relinquish faith in rankings and diminish the significance of their role in the job market. ¹⁹ In what follows, I will assess the potential effects of this increased degree of hierarchy – that is, I will describe a collection of incentives and social dynamics that are encouraged by steepening academic hierarchies and how they play out in economics. I will do this from the perspective of an uncontroversial goal: developing economic knowledge.

2.1. New beliefs

Hierarchy restricts the production of new beliefs from economics, and in doing so, restricts the creation of new economic knowledge. There are two mechanisms by which this happens: reinforcing conservative bias and disincentivizing innovation.

2.1.1. Reinforcing conservative bias

Lamont (2009) notes that at many levels – evaluation of student work, admission to post graduate studies, hiring, acceptance of papers, awards of grants, invitations, etc. – academics favour work similar to their own.²⁰ This could be framed in terms of maximizing their own chances of success – it makes sense for academics to favour work that conforms to positions they have previously defended (Holst and Christensen 2018) – but it might also have other causes. Social psychologists, sociologists and economists have all noted that it is common for people to favour those that are like themselves.²¹

It is, thus, plausible to assume that selectors in economics – those who sit on tenure committees, those who determine conference programmes, those that edit journals, etc. – exhibit at least weak biases in favour of research similar to their own.²² Unchecked, such biases can skew journals, departments, conferences and whole disciplines in a conservative direction with papers, job (and tenure) applicants, conference programmes and award and committee candidates with research profiles similar to selectors given an increased likelihood of prevailing. This can make it less likely that new ideas, theories and models without cheerleaders among existing selectors will emerge.²³

This need not happen. Selection biases can be mitigated by the existence of reliable and unambiguous criteria for selection – a clear criteria for selection leaves less room for selector discretion, meaning that their biases are less likely to bite. Such criteria do not, however, seem to be readily available in economics. Take papers. Although some factors can be used to identify obviously flawed papers (internal contradictions, incomplete data, etc.), decisions over publication become more complicated when borderline work needs to be evaluated. In such cases, it is not clear what the criteria of acceptance should be. Creating an argument for the relevance of a specific methodology in a given context is important and there is often disagreement on which factors should be emphasized. Is it important that a paper modelling behavioural responses to the COVID-19 lockdowns presents a model that is simple? Explains a lot? Is calibrated by data to a particular degree? Even if there were an unambiguous scale for evaluating papers in economics, with some threshold for publication, a reliable test for determining whether or not papers pass that threshold would still be required.²⁴ It is not clear that this is the case in economics.²⁵ The discipline is of course not alone in this situation. Sociology, anthropology, philosophy and many branches of the natural sciences do not seem to admit both a clear scale for selecting good work/people and a reliable test for scoring candidates on that scale. In the absence of strong selection criteria the conservative effect of biased selectors can be mitigated if their influence is limited or if their biases are diversified. In economics, both of these things are undermined by the steepness of the discipline’s hierarchy.

Starting with publishing, there are three facts worth noting. First, given the impact the Top 5 has on career success, Top 5 editors have a disproportionate impact on selections in economics. ²⁶ This amplifies any biases they may have. Second, the significance of just five journals means that the pool of influential editors is small. Third, editors in this already small pool are mostly trained at a small group of elite universities – as a reminder, six universities trained 64% of the editors of the top four journals. Whilst these three facts do not categorically show too little diversity among Top 5 selectors, they do at least indicate constraints on how diverse they might be. If the Top 5 were not so significant in other selections, then the preferences of their editors would not have as much influence. If the Top 5 were extended to a top 10 or 20, the pool of editors holding power would widen. If the Top 5 journals were less dominated by economists educated at a small subsection of universities, they would also likely bring more diverse intellectual perspectives.

A similar story can be told about other aspects of economics’ hierarchy. The constraints on the diversity of Top 5 editors are compounded by the facts that those in governing positions, those that become figureheads through big prizes, and those whose expertise and opinion is sought after are often also Top 5 editors or at least come from the same universities. These factors all increase the influence of a small group of selectors and increase the likelihood that the preferences of those in that group will be similar. The hierarchy in economics, thus, heightens the consequences of conservative selection biases and undermines one way of mitigating them: diversification. In doing so it enhances the influence of the research preferences and styles common in the top universities and decreases the likelihood that new ideas will be able to find cheerleaders among key selectors.²⁷

2.1.2. Disincentivizing innovation

The awareness among economists of a clear and consequential hierarchy serves to make competition in the discipline intense and stressful (Conley and Önder 2014; Akerlof 2020). A number of studies have offered reasons to believe that high-stakes competitive environments heighten risk-averse behaviour (Slovic et al. 1979; Loewenstein et al. 2001; Weber et al. 2002). Given this, it is reasonable to believe that many economists are discouraged by their environment from following paths with big chances of failure. Rather than following risky innovative paths, they are more likely to reason instrumentally and conform.

This risk-aversion is compounded by a number of perverse incentives. Starting with publishing again, the dominance of the Top 5 creates at least four incentives that pull away from novelty. First, the importance of the Top 5 incentivizes economists to follow known paths to Top 5 success. According to a former editor of two Top 5 journals (JPE and ECMA), Lars Hansen, this makes “high-quality follow up papers”, rather than original research projects, more common (at ∼55–56 minutes of Heckman et al. 2017). Second, the importance of the Top 5 encourages the herd-following behaviour that creates fads, since economists must think about what is popular when deciding what to work on. Third, the dominance of the Top 5 incentivizes spending time polishing papers to satisfy reviewers over taking those papers elsewhere and moving on to new projects.²⁸ Fourth, the importance of the Top 5 incentivizes developing lines of research and writing papers on topics, in styles and using methods preferred by editors of the top journals.

If economists act on these incentives then they are less likely to explore new topics, more likely to follow others, likely to spend more time on each paper and more likely to tailor their work in the direction of already existing research. There are reasons to believe that many economists do act with such incentives in mind. Long review times (Ellison 2002, 2011), low acceptance rates,²⁹ and short tenure clocks combine with the dominance of the Top 5 to encourage individual economists to think strategically about the work they pursue. Many young economists will tell you that whether or not a piece of work would get into the Top 5 is at the forefront of their mind when starting a project.

Other aspects of economics’ hierarchy deepen these issues. Given how significant first jobs are, it is important to choose doctoral research that is popular among Top 5 editors and those likely to be on hiring committees. Given how influential those at the top of the discipline are – as governors, spokespeople, editors or selectors – it makes sense to think carefully about how one’s work can be tailored in a direction they might approve of, rather than follow one’s own thoughts on what is most important. This is particularly problematic given that the need to think and act instrumentally is likely to be stronger for early career economists, who might otherwise be hoped to be the most creative.

The hierarchy in economics also potentially dampens the supply of ideas from outside of the elite. The voices and preferences of those at the top dominate, and because of the prestige factor in hiring and the fact that those outside the top find it more difficult to publish in the top journals, there are fewer avenues available for those from lower down to rise up. This may discourage some from entering or continuing in economics.³⁰

Thus, as well as constraining avenues for the uptake of new ideas, the hierarchy in economics constrains the production of such ideas. Together these constraints against the creation and uptake of new beliefs in economics suggest unseen absences – research and potentially whole topics that might have come to light in another environment may be missing because economics is too hierarchical.³¹

2.2. Truth and justification

One way of countering the argument above is to note that the development of scientific knowledge is not always helped by newness. Productive sciences tend to balance novelty and more workmanlike problem solving (Kuhn 1962; Weisberg and Muldoon 2009). The constraints on novelty in economics may be part of what is on aggregate an efficient system. A second counter-point could be that new beliefs are not in themselves sufficient for new knowledge. Truth and justification are an important part of knowledge in addition to belief.

I have two responses to these points. First, even though the development of scientific knowledge may require a balance between scientists that strike out in new directions and those that solve existing problems, and even though sciences require mechanisms for selecting the right propositions among those that are put forth, there is no evidence that biases in favour of the preferences of a select few and incentives against developing new ideas for the rest provide an adequate version of either. Second, the likelihood of truth and thus the justification of economic knowledge is also negatively affected by the steepness of the discipline’s hierarchy. There are two ways in which this is the case.

2.2.1. Unconceived alternatives

The faith one places in the outputs of any science should be a function of whether or not we think that relevant alternatives have been considered. If we have reason to believe that superior theories, mechanisms, models or explanations were missed because they were never conceived then we have less reason to believe the outputs we are presented with (Stanford 2001, 2019).

Establishing that superior alternatives exist requires finding them and showing that they are superior. But given that unconceived alternatives are just that – unconceived – they cannot be shown to be superior. Whether the outputs of a science suffer from regularly failing to conceive of superior alternatives, thus, cannot be assessed in this way. There are, however, other ways of making such assessments. Different ways of practicing and institutionalizing science can affect the likelihood that alternatives will be considered by incentivizing certain behaviours and by elevating certain forms of research. It is plausible that the present degree of hierarchy in economics does just this. An environment that disincentivizes innovation and encourages sticking close to the research of lauded seniors likely affects how much of the space of possible theories is explored, increasing the chances that alternatives to what is eventually settled on will be missed.

2.2.2. Constraining critical feedback

As well as considering suitable alternatives, it is important that sciences develop checks for their outputs. All enquiry is fallible. Be they through reasoning mistakes, faulty data, incorrect auxiliary assumptions, or other issues, errors occur. No science is safe. This need not lead to some deep scepticism about scientific knowledge. But it does mean that the faith we place in specific knowledge claims should be a function of how likely we think that any errors in them have been picked up.

This is a key starting point for methodological debate. Methodologists regularly discuss how the techniques used by economists justify inferring the truth of the work they produce (see, for example, Mäki 2011). These are important discussions. But the degree to which economics as a whole produces justified or true propositions is not only a function of method. Social factors can alter the aggregate spread of true outputs from any science by shifting incentives and altering the ways in which candidates for knowledge are challenged and critiqued. One way that the latter can occur is through peer engagement. Peers can press each other’s reasoning, question each other’s background assumptions and suggest new ways of testing each other’s claims against reality. Because these mechanisms of what we might call critical feedback make it more likely that falsehoods, inappropriate assumptions, biases and other failings will be corrected, propositions that are exposed to them are less likely to be false and thus better justified than those that are not (Longino 2002). Scientific knowledge may never be infallible, but at least its robustness can be increased by ensuring it is regularly challenged and critiqued.

This is not a new point, philosophers from Mill (1859), to Peirce (1998), to Longino (2002) have highlighted the importance of something like critical feedback to science. But it is a point that has so far been under-emphasized in social epistemology discussions on the optimal organization of science; and it is a point that is completely absent in discussions about economics. While methods for testing and standardizing individual pieces of research – common topics of economic methodology – are important, the effectiveness with which a science evaluates its outputs does not only rest on such methods. Via affecting the critical feedback within a scientific community, social factors can also aid or inhibit the proper evaluation of knowledge. It is this that the hierarchy in economics negatively affects.

The constraints on innovation discussed in section 2.1. are one way in which this happens. Innovative ideas, forms of measurement, observations, etc. can provide new ways of testing and evaluating existing knowledge. But constraints on innovation are not the only issue. The steepness of economics’ hierarchy also has a direct effect on criticism. As already noted, stressful competitive environments like that found in economics tend to lead to risk-averse behaviour. As well as lowering the propensity of those outside the top of economics to work on risky new topics, this is also likely to make them less willing to criticize those higher up.

There is evidence that hierarchical organizations have this effect – steeper hierarchies discourage those with dissenting opinions from articulating them (Milanovich et al. 1998; Kish-Gephart et al. 2009). Romer (2016) points to something like this in economics. In delivering a wide-ranging critique of macroeconomics, Romer noted that although some of his colleagues agree with his criticisms in private, they would never do so in public for fear of reprisals. The intense competition in economics and the pressures and incentives internalized by economists from a young age, makes them unlikely to rock the boat.³² To highlight this Romer noted that:

After I criticized a paper by Lucas, I had a chance encounter with someone who was so angry that at first he could not speak. Eventually, he told me, “You are killing Bob.” … Several economists I know seem to have assimilated a norm that … it is an extremely serious violation of some honor code for anyone to criticize openly a revered authority figure. (Romer 2016: 21)

Rodrik and Rubinstein have noted similar points – it is typically frowned upon to publicly criticize the work of other economists (WEA 2013; Rubinstein 2016).

Given the role that critical feedback plays in avoiding errors, the hierarchy in economics means that what is taken as knowledge in the discipline is less robustly checked and tested than it could have been. In combination with the fact that the present degree of hierarchy in economics plausibly makes it more likely that alternative theories/mechanisms/models will be missed, this makes the outputs of the discipline less likely to be true, and so less justified.

3. Conclusion: evaluating the social organization of economics

Economics is more hierarchical than other disciplines and more hierarchical than it could be. Via (a) reinforcing conservative selection biases and (b) disincentivizing innovation, the steeper degree of hierarchy in economics lowers the uptake and supply of new beliefs in the discipline. Via (c) discouraging looking for alternatives and (d) curtailing critical feedback, the steeper hierarchy makes it less likely that the discipline’s products will be true, reducing the justification we have for believing them. ³³ The steeper hierarchy in economics, thus, affects all of the key components of knowledge: justification, truth and belief creation. Despite being a social feature, the increased hierarchy in economics has epistemic consequences.

One way of responding to this is to argue that the hierarchical relations I have described are justified by merit. I have two counterpoints. First, the limited data available on whether positions in economics’ hierarchy are earned or based on a combination of luck and initial position points to the latter playing an important role. This is highlighted by the strong prestige factor in hiring and Oyer’s work on first jobs. Second, (a–d) apply regardless. A small selection of economists are better heard in debates, more influential in decisions on what gets published, what gets discussed at conferences, how the job market is structured and how the AEA is run. Even if that subsection of economists were the best on some scale, the fact that they have such a large influence heightens conservative selection biases, incentivizes following their lead, reduces the likelihood that alternative theories and explanations will be explored and disincentivizes critical feedback.

A second way of responding to the points I have raised could be to suggest that I have been too one-sided. Are there not circumstances in which steep hierarchies can be beneficial? To this I offer a clarification. The issues I describe should be considered pro tanto reasons to worry about the present degree of hierarchy in economics. I have argued that the steeper hierarchy in economics encourages four mechanisms that lower the uptake and supply of new beliefs in and the justification of the outputs of the discipline. I have not argued that the hierarchy in economics has no other effects. Thus, although (a–d) should give us reason to worry about the present organization of economics, they do not constitute an all-things-considered judgement on the health of the discipline. (a–d) are best thought of as tendencies worth paying attention to in discussions of how economics should be organized. An all-things-considered judgement on the organization of the discipline should consider (a–d) in conjugation with calculations of their effect sizes and also consider trade-offs from changing the present situation – including any beneficial effects of the hierarchy in economics.

With that caveat aside, there are reasons to believe that the benefits steeper hierarchies are typically thought to offer may not be applicable to economics. Steeper hierarchies are thought to motivate their members to work towards the interests of the group by rewarding those that successfully climb them (Cartwright and Zander 1953; Levine and Moreland 1990). The problem with applying this to the present case is that successful hierarchy climbing by individual economists does not seem to contribute to the wider community’s goals. As argued above, the incentives created by economics’ hierarchy run counter to both innovation and critical engagement. The hierarchy does not motivate economists to produce innovative and useful work or to carefully check the work of their colleagues, it motivates them to think instrumentally and keep their heads down. ³⁴ Steeper hierarchies are also thought to make group decision-making more efficient by giving disproportionate control to members with superior abilities and reining in conflict. But it is questionable whether the goals of economics, or any science, are efficient and conflict-free decision-making. There is no evidence to suggest that efficient decision-making leads to more or better knowledge. Moreover, even if efficient and conflict-free decision-making were the goal of economics, studies that find steep hierarchies to be efficient tend to involve relatively simple, stable, unambiguous tasks. Whereas studies that look at more complicated group tasks, or tasks that require creativity, tend to find that steep hierarchies hinder rather than help performance.³⁵ These points do not mean that an all-things-considered judgement on the health of economics can now be made. Estimates of the magnitudes of all the different mechanisms involved – a large task – would be required for that. But it does give us some reason to believe that economics may, at present, be too hierarchical. ³⁶

If economics is too hierarchical, what might be done? High-ranking economists have started to discuss some of the issues described above. This is welcome. Although the discussion to date has been too narrowly focused on the Top 5. Hansen suggests that editors should be bossier, look for good ideas and rely less on referee reports (Heckman et al. 2017). This proposal may lower the amount of time it takes to publish in the Top 5 and may lower the effect that review has on homogenizing papers towards commonly held ideas. But it would do nothing about other problematic incentives, would not address the pressures against criticism and may even heighten the effects of conservative selection biases by giving even more power to journal editors. Heckman and Moktan (2020) suggest shifting away from the Top 5 to a pre-publication arXiv- or hybrid PLoS ONE-like model of review and publication. This would likely speed up review times and enable innovative ideas that might not make it through Top 5 review to impact the field – thus mitigating some of the problematic incentives described in section 2. However, without equal attention to the other aspects of economics’ hierarchy, such a model may increase rather than decrease asymmetries. Like most academics, economists use proxies to determine which papers deserve their attention. Journals provide one such proxy. Without journals, they are likely to lean on other proxies. Given the steepness of economics’ hierarchy and the widespread belief that it is meritocratic, a pre-publication or hybrid model of publishing may lead to even greater asymmetries of attention, with the work of those at higher ranked universities (or those with big reputations) more likely to be read. This would likely be compounded by the abilities of higher ranked universities to market the ideas of their employees. There are problems with peer review, but that all papers are in principle read and evaluated by someone (even in the case of desk rejections) means that everyone gets a shot. This is preferable to a situation in which the only people read at all are those that dominate by virtue of their reputation and institutional backing. Heckman and Moktan also suggest a “shift from the current publications-based system of deciding tenure, to a system that emphasizes departmental peer review of a candidate’s work” and that “the profession should deemphasize crass careerism and promote creative activity” (2020: 463). It is hard to disagree with either of these points. The former is a good idea but given time pressures it is unlikely to apply to job and grant applicants, who will still likely be judged on the rankings of their publications and previous institutions. The latter is not a solution but suggests support for the broad point I want to emphasize: the issue of hierarchy in economics runs much deeper than the Top 5 and requires more wide-ranging solutions than just tinkering with publication models.

So, what might be done? Readers hoping for a grand solution will be disappointed. Organizing science is hard. Despite criticizing the incompleteness of the solutions suggested by others, I have no complete alternative. The points I have made in this paper do, however, contribute to the search for a solution. My goal has been to highlight the various hierarchical practices in economics and describe why they are problematic from the perspective of developing economic knowledge. The issues I describe – conservative selection biases and incentives against innovation, alternative ideas and critical feedback – can be used to evaluate potential solutions. I offer three further comments. First, the consciousness raising that has begun is a helpful first step. The fact that economists tend to believe that hierarchy is a good thing and based on merit prevents change. But consciousness raising is limited and cannot be the solution alone. Second, in all of the issues discussed above it is the degree of asymmetry rather than the existence of an asymmetric ordering that matters. Other things being equal, if the degree of asymmetry in any factor were lessened then the overall steepness of the hierarchy would be reduced and the negative implications would likely lessen. A weaker prestige factor in hiring would make it easier for ideas to move from lower ranked universities to those higher up, likely diversifying the knowledge base in economics. A wider selection of important journals would diversify the pool of influential selectors. And so on. Small changes here and there can make a difference. Third, changes should, however, be made with care because the issues I have described are not easily separable. Other things are not likely to be equal. The hierarchy in economics has multiple features that feed off one another. Solutions that act on only one dimension, such as the dominance of the Top 5, may lead to the issues popping up in another form. Hansen’s suggestion for mitigating some of the problematic incentives caused by the Top 5 and Heckman’s suggestion of an arXiv-like system are not useless. The issue is taking them in isolation. It is not only the Top 5 that is an issue, but the wider hierarchical practices in economics that have manifested themselves in obsession over the Top 5. The different aspects of economics’ hierarchy serve to reinforce each other. Solutions to the issues I describe should look to act on many factors at once. The key in doing this would be to ensure that the different features of economics’ hierarchy, and any solutions to the issues caused by them, do not pull in the same direction and empower the same individuals or universities.