Hostname: page-component-78c5997874-lj6df Total loading time: 0 Render date: 2024-11-10T16:34:47.809Z Has data issue: false hasContentIssue false
  • English
  • Français

The Impact Factor Fetishism

Published online by Cambridge University Press:  18 October 2013

Christian Fleck
Christian Fleck*
Affiliation:
Karl-Franzens-Universität, Graz, Institut für Soziologie [christian.fleck@uni-graz.at].
Get access

Abstract

One of the most popular indicators is the Impact Factor. This paper examines the coming into being of this highly influential figure. It is the offspring of Eugene Garfield’s experimentation with the huge amounts of data available at his Institute for Scientific Information and the result of a number of attempts to find appropriate measurements for the success (“impact”) of articles and journals. The completely inductive procedure was initially adjusted by examining the data thoughtfully and by consulting with experts from different scientific disciplines. Later, its calculation modes were imposed on other disciplines without further consideration. The paper demonstrates in detail the inopportune consequences of this, in particular for sociology. Neither the definition of disciplines, nor the selection of journals for the Web of Science/Social Science Citation Index follows any comprehensible rationale. The procedures for calculating the impact factor are inappropriate. Despite its obvious unsuitability, the impact factor is used by editors of sociological journals for marketing and impression management purposes. Fetishism!

Résumé

L’un des indicateurs bibliométriques les plus utilisés est le facteur d’impact. On retrace ici l’ascension de ce chiffre si influent. Il est le produit d’une expérimentation d’Eugène Garfield à partir de l’énorme masse de données de son « Institute for Scientific Information » après de nombreux essais pour trouver des mesures convenables de succès (impact) d’articles ou de revues. La démarche, totalement inductive a été d’abord affinée à la fois par l’examen attentif des données et l’appel à des experts scientifiques de quelques disciplines. Par la suite les modalités de calcul ont été étendues aux autres, sans plus de précaution. L’article expose dans le détail les conséquences catastrophiques en particulier pour la sociologie. Ni le contenu des intitulés disciplinaires, ni la sélection des revues dans le Web of Science/Social Science Citation Index ne répondent à des critères rationnels. Il en va de même de la procédure de calcul des facteurs d’impact, qui, en dépit de ces défauts évidents est utilisé par les éditeurs de revues à des fins de marketing et de pilotage. Fétichisme !

Zusammenfassung

Der Beitrag untersucht Entstehung und Verwendung des "Impact Faktors". Er war ein Nebenprodukt von Eugene Garfields Experimentieren mit den riesigen Datenmengen, die seinem Institute for Scientific Information zur Verfügung standen, um den Erfolg („impact“) von Artikeln und Zeitschriften zu messen. Der Impact Faktor wurde völlig induktive anhand der Daten einiger weniger naturwissenschaftlicher Disziplinen und nach Konsultationen mit Experten aus diesem Bereich entwickelt und danach ohne weitere Prüfung auf andere Disziplinen übertragen. Weder die Definition der Disziplinen, noch die Auswahl der Zeitschriften, die in das Web of Science bzw. den Social Science Citation Index aufgenommen wurden, folgen einer nachvollziehbaren Begründung. Trotz offensichtlicher Ungeeignetheit für die Soziologie wird der Impact Faktor von Herausgebern und Verlagen für Werbe- und Marketingzwecke verwendet und beeinflusst das Leseverhalten und die Veröffentlichungspraktiken von Soziologen. Ihr Verhalten gleicht dem Fetischismus, der vor langem als Merkmal der kapitalistischer Produktionsweise identifiziert wurde.

Keywords

Impact FactorWeb of Science/Social Science Citation IndexEugene GarfieldjournalsEuropean sociologyMeasurementScientometricsFacteur d’impactWeb of Science/Social Science Citation IndexEugene GarfieldRevuesSociologie européenneMesuresBibliométrieImpact FactorWeb of Science/Social Science Citation IndexEugene GarfieldZeitschriftenEuropäische SociologyoeMessungScientometrie
Type
Research Articles
Information
European Journal of Sociology / Archives Européennes de Sociologie , Volume 54 , Issue 2 , August 2013 , pp. 327 - 356
Copyright
Copyright © A.E.S. 2013 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

BIBLIOGRAPHY

Aaltojärvi, Inari, Arminen, Ilkka, Auranen, Otto and Pasanen, Hanna-Mari, 2008. “Scientific productivity, web visibility and citation patterns in sixteen nordic sociology departments”, Acta Sociologica, 51: 5-22.Google Scholar
Abbott, Andrew, 2011. “Library Research Infrastructure for Humanistic and Social Scientific Scholarship in America in the Twentieth Century” in Camic, Charles, Lamont, Michèle and Gross, Neill, eds., Social knowledge in the making (Chicago, University of Chicago Press: 43-87).Google Scholar
Archambault, Éric and Larivière, Vincent, 2009. “History of the Journal Impact Factor: Contingencies and Consequences”, Scientometrics, 79: 635-649.Google Scholar
Archambault, Éric, Gingras, Yves and Larivière, Vincent, eds., 2012. Proceedings of 17th International Conference on Science and Technology Indicators (Montréal, Science-Metrix and ost).Google Scholar
Ashton, Susan V. O. C., 1978. “A Method of Predicting Nobel Prizewinners in Chemistry”, Social Studies of Science, 8: 341-348.Google Scholar
Baldi, Stéphane, 1998. “Normative versus Social Constructivist Processes in the Allocation of Citations: A Network-Analytic Model”, American Sociological Review, 63: 829-846.CrossRefGoogle Scholar
Baum, Joel A. C., 2011. “Free-Riding on Power Laws: questioning the validity of the Impact Factor as a measure of research quality in organization studies”, Organization, 18: 449-466.Google Scholar
Becker, Joseph, 1968. “Information Storage and Retrieval” in Sills, David L., ed., International encyclopedia of the social sciences, vol. 7 (New York, Macmillan: 301-304).Google Scholar
Bensman, Stephen J., 2007. “Garfield and the Impact Factor”, Annual Review of Information Science and Technology, 41: 93-155.CrossRefGoogle Scholar
Bjarnason, Thoroddur and Sigfusdottir, Inga D., 2002. “Nordic impact: Article productivity and citation patterns in sixteen Nordic Sociology departments”, Acta Sociologica, 45: 253-267.CrossRefGoogle Scholar
Bulmer, Martin and Solomos, John, 2010. “Introduction: developments and plans for the future of Ethnic and Racial Studies”, Ethnic and Racial Studies, 33: 1689.Google Scholar
Chi, Pei-Shan, 2012. “Characteristics of Publications in Political Science” in Archambault, Éric et al. ., eds., Proceedings of 17th International Conference on Science and Technology Indicators (Montréal, Science-Metrix and ost: 441-442).Google Scholar
Collini, Stefan, 2012. What are universities for? (London, Penguin).Google Scholar
Drori, Gili S., Meyer, John W., Ramirez, Francisco O. and Schofer, Evan, 2003. Science in the modern world polity. Institutionalization and globalization (Stanford, Stanford University Press).Google Scholar
Elkana, Yehuda, Lederberg, Joshua, Merton, Robert K., Thackray, Arnold and Zuckerman, Harriet, eds., 1978. Toward a Metric of Science: The Advent of Science Indicators (New York, Wiley).Google Scholar
Engels, Tim, Ossenblok, Truyken and Spruyt, Eric, 2012. “Changing Publication Patterns in the Social Sciences and Humanities, 2000–2009”, Scientometrics, 93: 373-390.Google Scholar
European Commission, 2011. The Ideas Work Programme. European Research Council Work Programme 2012 (Brussels, European Commission), (http://erc.europa.eu/sites/default/files/document/file/erc_2012_work%20programme.pdf).Google Scholar
The Europa world of learning 2012 (London, Routledge).Google Scholar
Fischer, Klaus, 2008. “Science and its malfunctions”, Human architecture: Journal of the sociology of self-knowledge, 6: 1-22.Google Scholar
Fleck, Christian, 2010. „Wie statistische und andere Zahlen unser Vertrauen gewinnen und enttäuschen“ in Aulenbacher, Brigitte and Ziegler, Meinrad, Hrsg., In Wahrheit … Herstellung, Nutzen und Gebrauch von „Wahrheit“ in Wissenschaft und Alltag (Innsbruck, Studienverlag: 131-146).Google Scholar
Fox, R. et al. ., 2009: “Journals under Threat: A Joint Response from History of Science, Technology and Medicine Editors”, Notes and Records of the Royal Society, 63: 1-3.Google Scholar
Fuyuno, Ichiko and Cyranoski, David, 2006. “Cash for Papers: Putting a Premium on Publication”, Nature, 441: 792.Google Scholar
Garfield, Eugene, 1963. “Citation Indexes in Sociological and Historical Research”, American Documentation, 14: 289-291.Google Scholar
Garfield, Eugene, 1972. “Citation Analysis as a Tool in Journal Evaluation”, Science Citation Index 1961, 178: 471-479.Google Scholar
Garfield, Eugene, 1973. “More on Forecasting Nobel Prizes and the Most Cited Scientists of 197”, Current Contents, 40: 5-7.Google Scholar
Garfield, Eugene, 1977a. “Citation Analysis as a Tool in Journal Evaluation” in Garfield Eugene, Essays of an Information Scientist: 1962-1973, vol. 1 (Philadelphia, ISI-Press: 527-544).Google Scholar
Garfield, Eugene, 1977b [1970]. “Citation Indexing, Historio-bibliography, and the Sociology of Science” in Garfield, Eugene, Essays of an Information Scientist: 1962-1973, vol. 1 (Philadelphia, ISI-Press: 158-174).Google Scholar
Garfield, Eugene, 1977c. ““Citations-to” divided by “Items-published” gives Journal Impact Factor; ISI Lists the Top Fifty High-impact Journals of Science” in Garfield, Eugene, Essays of an Information Scientist: 1962-1973, vol. 1 (Philadelphia, ISI-Press: 270-273).Google Scholar
Garfield, Eugene, 1988. “Derek Price and the Practical World of Scientometrics”, Science, Technology, & Human Values, 13: 349-350.Google Scholar
Garfield, Eugene, 1990. “How ISI Selects Journals for Coverage: Quantitative and Qualitative Considerations”, Current Contents, #22.Google Scholar
Garfield, Eugene, 2005. “The Agony and the Ecstacy – The History and Meaning of the Journal Impact Factor”, International Congress on Peer Review and Biomedical Publication, Chicago (http://garfield.library.upenn.edu/papers/jifchicago2005.pdf.)Google Scholar
Garfield, Eugene and Sher, Irving H., 1963. “New Factors in the Evaluation of Scientific Literature Through Citation Indexing”, American Documentation, 14: 195-201.Google Scholar
Gingras, Yves and Mosbah-Natanson, Sébastien, 2010. “Where are Social Sciences Produced?” in unesco and issc, eds., World Social Science Report 2010: Knowledge Divides (Paris, unesco Publishing: 149-153).Google Scholar
Glänzel, Wolfgang and Moed, Henk, 2002. “Journal impact measures in bibliometric research”, Scientometrics, 53: 171-193.Google Scholar
Glänzel, Wolfgang and Moed, Henk, 2012. “Thoughts and Facts on Bibliometric Indicators” in Archambault, Éric et al. ., eds., Proceedings of 17th International Conference on Science and Technology Indicators (Montréal, Science-Metrix and ost: 305-318).Google Scholar
González-Pereira, Borja, Vicente Guerrero-Bote Felix and Moya-Anegón (o. J.): The SJR indicator: A new indicator of journals scientific prestige. Available online athttp://arxiv.org/ftp/arxiv/papers/0912/0912.4141.pdf.Google Scholar
Hargens, Lowell L, 2000. “Using the Literature: Reference Networks, Reference Contexts, and the Social Structure of Scholarship”, American Sociological Review, 65: 846-865.Google Scholar
Holmwood, John, 2010. “Sociology’s Misfortune: Disciplines, Interdisciplinarity and the Impact of Audit Culture”, British Journal of Sociology, 61: 639-658.Google Scholar
Ingwersen, Peter, 2012. “The pragmatics of a diachronic journal impact factor”, Scientometrics, 92: 319-324.Google Scholar
Kaplan, Norman and Storer, Norman W., 1968. “Scientific Communication” in Sills, David L., ed., International Encyclopedia of the Social Sciences, vol. 14 (New York, Macmillan: 112-117).Google Scholar
Kieser, Alfred, 2010. “Unternehmen Wissenschaft?” Leviathan. Berliner Zeitschrift für Sozialwissenschaft, 38: 347-367.Google Scholar
Kieser, Alfred, 2012. „jourqual. Der Gebrauch, nicht der Missbrauch, ist das Problem. Oder: warum Wirtschaftsinformatik die beste deutschsprachige betriebswirtschaftliche Zeitschrift ist“, Die Betriebswirtschaft, 72: 93-110.Google Scholar
Kirchner, Frank, ed., 2004. World Guide to Scientific Associations and Learned Societies (Munich, Saur).Google Scholar
Lederberg, Joshua, 2000. “How the Science Citation Index Got Started” in Cronin, Blaise and Atkins, Helen B., eds., The Web of Knowledge: A Festschrift in Honor of Eugene Garfield (Mulford, Information Today, Inc.: 25-64).Google Scholar
Macdonald, Stuart and Kam, Jacqueline, 2007. “Ring a Ring o’ Roses: Quality Journals and Gamesmanship in Management Studies”, Journal of Management Studies, 44: 640-655.Google Scholar
Macdonald, Stuart and Kam, Jacqueline, 2011. “The Skewed Few: People and Papers of Quality in Management Studies”, Organization, 18: 467-475.Google Scholar
Martins, Herminio, 1974. “Time and Theory in Sociology” in Rex, John, ed., Approaches to Sociology: An Introduction to Major Trends in British Sociology (London, Routledge & Kegan Paul: 246-294).Google Scholar
Martyn, John and Gilchrist, Alan, 1968. “An Evaluation of British Scientific Journals”, ASLIB Occasional Publication, #1.Google Scholar
Marx, Karl, 1906 [1867]. Capital. A Critique of Political Economy (New York, Modern Library).Google Scholar
Merton, Robert K., 1979. “Foreword” in Garfield, Eugene, Citation Indexing: Its Theory and Application in Science, Technology and Humanities (New York, John Wiley & Sons: v-ix).Google Scholar
Merton, Robert K., 2000. “On the Garfield Input to the Sociology of Science: A Retrospective Collage” in Cronin, Blaise and Atkins, Helen B., eds., The Web of Knowledge: A Festschrift in Honor of Eugene Garfield (Mulford, Information Today, Inc: 435-448).Google Scholar
Merton, Robert K. and Garfield, Eugene, 1986. “Foreword” in Price, Derek J. de Solla, Little Science, Big Science… And Beyond (New York, Columbia University Press: vii-xii).Google Scholar
Meyer, John W. and Rowan, Brian, 1977. “Institutionalized Organizations: Formal Structure as Myth and Ceremony” American Journal of Sociology, 83: 340-363.Google Scholar
Mirowski, Philip, 2011. Science-Mart. Privatizing American Science (Cambridge, Harvard University Press).Google Scholar
Moed, Henk, Colledge, Lisa, Reedijk, Jan, Moya-Anegon, Felix, Guerrero-Bote, Vicente, Plume, Andrew and Amin, Mayur, 2012. “Citation-based Metrics are Appropriate Tools in Journal Assessment Provided that they are Accurate and Used in an Informed Way”, Scientometrics, 92: 367-376.Google Scholar
Münch, Richard, 2007. Die akademische Elite. Zur sozialen Konstruktion wissenschaftlicher Exzellenz (Frankfurt a.M, Suhrkamp).Google Scholar
Pendlebury, David A., 2009. “The Use and Misuse of Journal Metrics and Other Citation Indicators”, Archivum Immunologiae et Therapiae Experimentalis, 57: 1-11.Google Scholar
Persson, Olle, 1985. “Scandinavian social science in international journals”, Social Science Information Studies, 5: 185-190.CrossRefGoogle Scholar
Power, Michael, 1997. The Audit Society: Rituals of Verification (New York, Oxford University Press).Google Scholar
Price, Derek S. de, 1978. “Editorial statements”, Scientometrics, 1: 7.Google Scholar
Pudovkin, A. I. and Garfield, Eugene, 2012. “Rank Normalization of Impact Factors Will Resolve Vanclay’s Dilemma with trif. Comments on the paper by Jerome Vanclay”, Scientometrics, 92: 409-412.CrossRefGoogle Scholar
Schofer, Evan, 2004. “Cross-national Differences in the Expansion of Science, 1970-1990” Social Forces, 83: 215-248.Google Scholar
Sica, Alan, 2012. “Editor’s Remarks” Contemporary Sociology, 41: 137-138.Google Scholar
Smith, Anthony D., 1983. “Nationalism and Classical Social Theory”, British Journal of Sociology, 34: 19-38.Google Scholar
Smith, Derek R., 2012. “Impact Factors, Scientometrics and the History of Citation-based Research”, Scientometrics, 92: 419-427.Google Scholar
Strathern, Marilyn, ed., 2000. Audit Cultures: Anthropological Studies in Accountability, Ethics, and the Academy (London, Routledge).Google Scholar
Streeck, Wolfgang and Feick, Jürgen, 2012. “Impact Factor and sase’s annual prize”, Socio-Economic Review, 10: 625.Google Scholar
Testa, Jim, 2012. The Thomson Reuters Journal Selection Process (http://thomsonreuters.com/products_services/science/free/essays/journal_selection_process/).Google Scholar
Ulrich’s Periodical Directory, 2011 (http://ulrichsweb.serialssolutions.com/).Google Scholar
unesco and issc, eds., 2010. World Social Science Report 2010: Knowledge Divides (Paris, unesco Publishing).Google Scholar
Vanclay, Jerome, 2011. “Impact Factor: Outdated Artefact or Stepping-stone to Journal Certification?”, Scientometrics, 92: 211-228.Google Scholar
Ward, Kevin, 2010. “Research Monographs: An Overview” in unesco and issc, eds., World Social Science Report 2010: Knowledge Divides (Paris, unesco Publishing: 300-302).Google Scholar
Wilhite, Allen W. and Fong, Eric A., 2012. “Coercive Citation in Academic Publishing”, Science, 335: 542-543.Google Scholar
Willmott, Hugh, 2011. “Journal List Fetishism and the Perversion of Scholarship: Reactivity and the ABS List”, Organization, 18: 429-442.Google Scholar
Wimmer, Andreas and Schiller, Nina Glick, 2003. “Methodological Nationalism, the Social Sciences and the Study of Migration: An Essay in Historical Epistemology”, International Migration Review, 37: 576-610.Google Scholar
Wouters, Paul and Leydesdorff, Loet, 1994. “Has Price’s Dream Come True: Is Scientometrics a Hard Science?”, Scientometrics, 31: 193-222.Google Scholar
Zitt, Michel, 2012. “The Journal Impact Factor: Angel, Devil, or Scapegoat? A Comment on J.K. Vanclay’s Article 2011”, Scientometrics, 92: 485-503.Google Scholar
Supplementary material: PDF

Fleck supplementary tables

Fleck supplementary tables

Download Fleck supplementary tables(PDF)
PDF 230.8 KB