John et al. make a remarkable articulation of the phenomenon of proxy failure although many components of it have been already well known. However, I am surprised that they do not adequately cover proxy failure in academia apart from a passing mention. I feel it is necessary to deal elaborately with proxy failure in academia for two reasons. One is that proxy failure has reached unprecedented and unparalleled levels in academia leading to bad incentives (Rammohan & Rela, Reference Rammohan and Rela2021; Roy & Edwards, Reference Roy and Edwards2023; Stephan, Reference Stephan2012) so that we can easily identify phenomena much ahead of what John et al. describe. Apart from the three consequences of proxy failure, namely proxy treadmill, proxy cascade, and proxy appropriation (John et al.) at least three more levels are observed in academia that might be difficult to find in other fields.
Proxy complimentarity: In this, more than one type of actors benefit in different ways from a proxy and therefore they reinforce each others' dependence on the proxy resulting in a rapidly deteriorating vicious cycle. Because prestige of a journal is decided by the citations its papers receive and the impressiveness of the curriculum vitae (CV) of a researcher is decided by the impact factors of the journals, the two selfish motives complement each other in citation manipulation (Fong & Wilhite, Reference Fong and Wilhite2017). The reviewers also may pressurize the authors to cite their papers (Teixeira da Silva, Reference Teixeira da Silva2017) and the authors may agree in return of paper acceptance (Chawla, Reference Chawla2023). Institutions and funding agencies are benefited because bibliographic indices lead to a pretense of evaluation saving the cost of in-depth reading of a candidate's research (Watve, Reference Watve2023). Such mutually convenient positive feedback cycles can potentially drive rapid deterioration of the goal.
Proxy exploitation: This is a phenomenon in which apart from the agents in the game optimizing their own cost benefits, a party external to the field achieves selfish benefits by exploiting proxies in the field. In academic publishing profit-making publishers of journals thrive almost entirely on journal prestige as the proxy. Editorial boards appear to strive more for journal prestige than the soundness and transparency of science (Abbott, Reference Abbott2023). More prestigious journals often have higher author charges (Triggle, MacDonald, Triggle, & Grierson, Reference Triggle, MacDonald, Triggle and Grierson2022) and make larger profits with little contribution to the original goals.
Predatory proxy: This might be the most advanced and disastrous form of proxy failure where the proxy devours the goal itself. The process of proxy appropriation where the higher-level goal does a corrective hacking of lower-level proxies (John et al.) does not work in epistemology because the goal itself is difficult to define. In business, the higher-level player directly monitors the goal of profit making and accordingly controls proxies at lower level. This does not work in academia because the higher-level organizations themselves do not have an objective perspective of the goal. As a result not only the proxies are used to evaluate individual researcher, but also they might often be confused with the progress of science itself.
In many fields of science highly complex work involving huge amounts of data and sophisticated methods of analysis are being published in prestigious journals adding little real insights to the field. For example in diseases such as type 2 diabetes, in spite of huge amount of research being published and funds being allocated, there is no success in preventing, curing, reducing mortality (ACCORD study group et al., Reference Gerstein, Miller, Byington, Goff, Bigger and Friedewald2008; Lee et al., Reference Lee, Heckman-Stoddard, Dabelea, Gadde, Ehrmann and Ford2021; Ojha, Vidwans, & Watve, Reference Ojha, Vidwans and Watve2022; The NICE-SUGAR Study Investigators, 2009) or even addressing the accumulating anomalies (Diwekar-Joshi & Watve, Reference Diwekar-Joshi and Watve2020) in the underlying theory. Nevertheless large numbers of papers continue to get published, huge amount of funding is allotted which by itself is viewed as “success” in the field. Failure of achieving the goal is not a crime in science, but quite often the failure is disguised as “success” and researchers receive life-time “achievement” awards. No scientist receiving any such awards appears to have admitted that they have actually failed to “achieve” the real goals. Efforts of a researcher, failed by this definition, should still be appreciated but it should not be called “success” or “achievement” just based on proxies. The worst outcome of proxy failure in academia is the failure to identify research failure as a failure. Many other fields including theoretical particle physics or string theory have received similar criticism (Castelvecchi & Nature magazine, Reference Castelvecchi2015; Hossenfelder, Reference Hossenfelder2022).
The three outcomes of proxy failure might be the reason why the creativity and disruptive content in published science is deteriorating even when it is measured by proxy itself (Chu & Evans, Reference Chu and Evans2021; Park, Leahey, & Funk, Reference Park, Leahey and Funk2023). Simultaneously the frequency of research misconduct, data fabrication, reproducibility crisis, paper mills, predatory journals, citation manipulations, peer-review biases, and paper retractions are alarming (Fanelli, Reference Fanelli2009; Ioannidis, Reference Ioannidis2005; Smith, Reference Smith2006; Xie, Wang, & Kong, Reference Xie, Wang and Kong2021) and appear to be rising (de Vrieze, Reference de Vrieze2021).
Interestingly, many researchers working on aspects of human behavior, system design, or behavior-based policy that are potentially relevant to academia avoid talking about the academic systems (Chater & Loewenstein, Reference Chater and Loewenstein2022; Cushman, Reference Cushman2019). The academic system is the nearest, most accessible and most relevant system to be studied. This is the second important reason why studying proxy failure in academia needs to be prioritized. However, research addressing behavioral aspects of academia is scanty and fragmentary (Chapman et al. Reference Chapman, Bicca-Marques, Calvignac-Spencer, Fan, Fashing, Gogarten and Chr Stenseth2019; Clark & Winegard, Reference Clark and Winegard2020; Watve, Reference Watve2017, Reference Watve2023) and not yet even close to addressing the haunting questions at a system design or s-frame level (Chater & Loewenstein, Reference Chater and Loewenstein2022). Unless researchers address the issues in their own field and come out with sound solutions; unless they design their own systems that are behaviorally sound and little prone to proxy failure; unless they are able to minimize flaws and make the system work smoothly toward the goals, why should other fields follow their advice to redesign their systems? The natural first reaction of a citizen like me working outside mainstream academia is “Academics, mend your house first!!”
You have
Access
John et al. make a remarkable articulation of the phenomenon of proxy failure although many components of it have been already well known. However, I am surprised that they do not adequately cover proxy failure in academia apart from a passing mention. I feel it is necessary to deal elaborately with proxy failure in academia for two reasons. One is that proxy failure has reached unprecedented and unparalleled levels in academia leading to bad incentives (Rammohan & Rela, Reference Rammohan and Rela2021; Roy & Edwards, Reference Roy and Edwards2023; Stephan, Reference Stephan2012) so that we can easily identify phenomena much ahead of what John et al. describe. Apart from the three consequences of proxy failure, namely proxy treadmill, proxy cascade, and proxy appropriation (John et al.) at least three more levels are observed in academia that might be difficult to find in other fields.
Proxy complimentarity: In this, more than one type of actors benefit in different ways from a proxy and therefore they reinforce each others' dependence on the proxy resulting in a rapidly deteriorating vicious cycle. Because prestige of a journal is decided by the citations its papers receive and the impressiveness of the curriculum vitae (CV) of a researcher is decided by the impact factors of the journals, the two selfish motives complement each other in citation manipulation (Fong & Wilhite, Reference Fong and Wilhite2017). The reviewers also may pressurize the authors to cite their papers (Teixeira da Silva, Reference Teixeira da Silva2017) and the authors may agree in return of paper acceptance (Chawla, Reference Chawla2023). Institutions and funding agencies are benefited because bibliographic indices lead to a pretense of evaluation saving the cost of in-depth reading of a candidate's research (Watve, Reference Watve2023). Such mutually convenient positive feedback cycles can potentially drive rapid deterioration of the goal.
Proxy exploitation: This is a phenomenon in which apart from the agents in the game optimizing their own cost benefits, a party external to the field achieves selfish benefits by exploiting proxies in the field. In academic publishing profit-making publishers of journals thrive almost entirely on journal prestige as the proxy. Editorial boards appear to strive more for journal prestige than the soundness and transparency of science (Abbott, Reference Abbott2023). More prestigious journals often have higher author charges (Triggle, MacDonald, Triggle, & Grierson, Reference Triggle, MacDonald, Triggle and Grierson2022) and make larger profits with little contribution to the original goals.
Predatory proxy: This might be the most advanced and disastrous form of proxy failure where the proxy devours the goal itself. The process of proxy appropriation where the higher-level goal does a corrective hacking of lower-level proxies (John et al.) does not work in epistemology because the goal itself is difficult to define. In business, the higher-level player directly monitors the goal of profit making and accordingly controls proxies at lower level. This does not work in academia because the higher-level organizations themselves do not have an objective perspective of the goal. As a result not only the proxies are used to evaluate individual researcher, but also they might often be confused with the progress of science itself.
In many fields of science highly complex work involving huge amounts of data and sophisticated methods of analysis are being published in prestigious journals adding little real insights to the field. For example in diseases such as type 2 diabetes, in spite of huge amount of research being published and funds being allocated, there is no success in preventing, curing, reducing mortality (ACCORD study group et al., Reference Gerstein, Miller, Byington, Goff, Bigger and Friedewald2008; Lee et al., Reference Lee, Heckman-Stoddard, Dabelea, Gadde, Ehrmann and Ford2021; Ojha, Vidwans, & Watve, Reference Ojha, Vidwans and Watve2022; The NICE-SUGAR Study Investigators, 2009) or even addressing the accumulating anomalies (Diwekar-Joshi & Watve, Reference Diwekar-Joshi and Watve2020) in the underlying theory. Nevertheless large numbers of papers continue to get published, huge amount of funding is allotted which by itself is viewed as “success” in the field. Failure of achieving the goal is not a crime in science, but quite often the failure is disguised as “success” and researchers receive life-time “achievement” awards. No scientist receiving any such awards appears to have admitted that they have actually failed to “achieve” the real goals. Efforts of a researcher, failed by this definition, should still be appreciated but it should not be called “success” or “achievement” just based on proxies. The worst outcome of proxy failure in academia is the failure to identify research failure as a failure. Many other fields including theoretical particle physics or string theory have received similar criticism (Castelvecchi & Nature magazine, Reference Castelvecchi2015; Hossenfelder, Reference Hossenfelder2022).
The three outcomes of proxy failure might be the reason why the creativity and disruptive content in published science is deteriorating even when it is measured by proxy itself (Chu & Evans, Reference Chu and Evans2021; Park, Leahey, & Funk, Reference Park, Leahey and Funk2023). Simultaneously the frequency of research misconduct, data fabrication, reproducibility crisis, paper mills, predatory journals, citation manipulations, peer-review biases, and paper retractions are alarming (Fanelli, Reference Fanelli2009; Ioannidis, Reference Ioannidis2005; Smith, Reference Smith2006; Xie, Wang, & Kong, Reference Xie, Wang and Kong2021) and appear to be rising (de Vrieze, Reference de Vrieze2021).
Interestingly, many researchers working on aspects of human behavior, system design, or behavior-based policy that are potentially relevant to academia avoid talking about the academic systems (Chater & Loewenstein, Reference Chater and Loewenstein2022; Cushman, Reference Cushman2019). The academic system is the nearest, most accessible and most relevant system to be studied. This is the second important reason why studying proxy failure in academia needs to be prioritized. However, research addressing behavioral aspects of academia is scanty and fragmentary (Chapman et al. Reference Chapman, Bicca-Marques, Calvignac-Spencer, Fan, Fashing, Gogarten and Chr Stenseth2019; Clark & Winegard, Reference Clark and Winegard2020; Watve, Reference Watve2017, Reference Watve2023) and not yet even close to addressing the haunting questions at a system design or s-frame level (Chater & Loewenstein, Reference Chater and Loewenstein2022). Unless researchers address the issues in their own field and come out with sound solutions; unless they design their own systems that are behaviorally sound and little prone to proxy failure; unless they are able to minimize flaws and make the system work smoothly toward the goals, why should other fields follow their advice to redesign their systems? The natural first reaction of a citizen like me working outside mainstream academia is “Academics, mend your house first!!”
Acknowledgments
I thank Preetee O. for useful discussions in a conceptualization stage.
Financial support
There was no funding for this work.
Competing interest
None.