Hostname: page-component-6bf8c574d5-l72pf Total loading time: 0 Render date: 2025-03-04T14:51:35.965Z Has data issue: false hasContentIssue false
  • English
  • Français

The wicked problem of measuring real-world research impact: Using sustainable development goals (SDGs) and targets in academia

Published online by Cambridge University Press:  12 October 2020

Geoffrey R. Chapman Open the ORCID record for Geoffrey R. Chapman [Opens in a new window] ,
Ashley Cully Open the ORCID record for Ashley Cully [Opens in a new window] ,
Jennifer Kosiol Open the ORCID record for Jennifer Kosiol [Opens in a new window] ,
Stephanie A. Macht Open the ORCID record for Stephanie A. Macht [Opens in a new window] ,
Ross L. Chapman Open the ORCID record for Ross L. Chapman [Opens in a new window] ,
Janna Anneke Fitzgerald Open the ORCID record for Janna Anneke Fitzgerald [Opens in a new window]  and
Frank Gertsen Open the ORCID record for Frank Gertsen [Opens in a new window]
Geoffrey R. Chapman*
Affiliation:
School of Business and Law, CQUniversity, 400 Kent St, Sydney, NSW, 2000, Australia
Ashley Cully
Affiliation:
Griffith Business School, Griffith University, 1 Parklands Drive, Southport, QLD4222, Australia
Jennifer Kosiol
Affiliation:
Griffith Business School, Griffith University, 1 Parklands Drive, Southport, QLD4222, Australia
Stephanie A. Macht
Affiliation:
School of Business and Law, CQUniversity, 400 Kent St, Sydney, NSW, 2000, Australia
Ross L. Chapman
Affiliation:
School of Business and Law, CQUniversity, 400 Kent St, Sydney, NSW, 2000, Australia
Janna Anneke Fitzgerald
Affiliation:
Griffith Business School, Griffith University, 1 Parklands Drive, Southport, QLD4222, Australia
Frank Gertsen
Affiliation:
Department of Materials and Production, Aalborg University, The Faculty of Engineering and Science, Fibigerstraede 16, 9220Aalborg, Denmark
*
*Corresponding author. Email: g.chapman@cqu.edu.au
Get access Rights & Permissions [Opens in a new window]

Abstract

This paper proposes that the United Nation's sustainable development goals (SDGs) and associated targets form an effective framework for determining real-world research impact. Existing bibliometrics that assess the quality of academic work are usually quantitative and self-referential, reducing the focus on real-world issues. The same measurements are often adopted by funding bodies, pressuring researchers to increase compliance, and further reducing integrity and real-world impact. A series of world cafés were conducted, collecting data on how researchers, their institutions, and network organisations can contribute to, and measure research aligned with the SDGs and targets. The results showed that participants were generally positive towards using the SDGs and targets to measure impact and quality of academic research. Suggestions to assist greater adoption of the SDGs and targets as a measure of impact included: aligning governmental and institutional funding; changing key performance indicators; increasing cross-disciplinary work; aligning mission/vision statements; and legitimising SDG-focused projects at conferences.

Keywords

sustainable development goalsreal-world impactresearch impactbibliometricswicked problems
Type
Research Article
Information
Journal of Management & Organization , Volume 26 , Special Issue 6: Wicked Problems and Sustainable Development Goals , November 2020 , pp. 1030 - 1047
Copyright
Copyright © Cambridge University Press and Australian and New Zealand Academy of Management 2020

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Agarwal, A., Durairajanayagam, D., Tatagari, S., Esteves, S. C., Harlev, A., Henkel, R., & Ramasamy, R. (2016). Bibliometrics: Tracking research impact by selecting the appropriate metrics. Asian Journal of Andrology, 18(2), 296.CrossRefGoogle ScholarPubMed
Aldred, R. (2009). From community participation to organizational therapy? World café and appreciative inquiry as research methods. Community Development Journal, 46(1), 57–71.CrossRefGoogle Scholar
Anderson, L. (2011). Use the world café concept to create an interactive learning environment. Education for Primary Care, 22(5), 337–338.CrossRefGoogle ScholarPubMed
Australian Research Council [ARC] (2015a) Research impact principles and framework. Canberra, Australia.Google Scholar
Australian Research Council [ARC] (2015b) Engagement and impact assessment, available at: https://www.arc.gov.au/engagement-and-impact-assessment. Accessed 02 January 2020.Google Scholar
Bebbington, J., & Unerman, J. (2018). Achieving the United Nations sustainable development goals: An enabling role for accounting research. Accounting, Auditing & Accountability Journal, 31(1), 2–24.CrossRefGoogle Scholar
Brew, A., Boud, D., Namgung, S. U., Lucas, L., & Crawford, K. (2016). Research productivity and academics’ conceptions of research. Higher Education, 71(5), 681–697.CrossRefGoogle Scholar
Brown, J. B., & Isaacs, D. (2005). The world café: Shaping our futures through conversations that matter. San Francisco: Berrett-Koehler.Google Scholar
Bryman, A. (2016). Social research methods. Oxford, New York: Oxford University Press.Google Scholar
Chapman, R. L. (2012) The downside of relying on research outputs to assess business faculty performance: Comments from down under regarding ‘facilitating and creating synergies between teaching and research: The role of the academic administrator’. Journal of Management Education, 36 (4), 495–502, Sage, Thousand Oaks, Calif.CrossRefGoogle Scholar
Disley, Y. P. (2013). Sustainable development goals for people and planet. Nature, 495, 21.Google Scholar
Doyle, J. (2018). Reconceptualising research impact: Reflections on the real-world impact of research in an Australian context. Higher Education Research & Development, 37(7), 1366–1379.CrossRefGoogle Scholar
Drew, C. H., Pettibone, K. G., Finch, F. O., Giles, D., & Jordan, P. (2016). Automated research impact assessment: A new bibliometrics approach. Scientometrics, 106(3), 987–1005.CrossRefGoogle ScholarPubMed
Fouché, C., & Light, G. (2011). An invitation to dialogue: ‘The world café’ in social work research. Qualitative Social Work, 10(1), 28–48.CrossRefGoogle Scholar
Fullarton, C., & Palermo, J. (2008). Evaluation of a large group method in an educational institution: The world café versus large group facilitation. Journal of Institutional Research, 14(1), 109–117.Google Scholar
George, G., Howard-Grenville, J., Joshi, A., & Tihanyi, L. (2016). Understanding and tackling societal grand challenges through management research. Academy of Management Journal, 59(6), 1880–1895.CrossRefGoogle Scholar
Glick, W. H., Tsui, A., & Davis, G. F. (2018). The moral dilemma of business research. BizEd, 17(3), 32–37.Google Scholar
Greenhalgh, T., Raftery, J., Hanney, S., & Glover, M. (2016). Research impact: A narrative review. BMC Medicine, 14(1), 78.CrossRefGoogle ScholarPubMed
Griffith University (2019). Sustainable development. Retrieved June 20, 2019, from https://griffith.edu.au/sustainable-development.Google Scholar
Hajdarpasic, A., Brew, A., & Popenici, S. (2015). The contribution of academics' engagement in research to undergraduate education. Studies in Higher Education, 40(4), 644–657.CrossRefGoogle Scholar
Head, B. W. (2019). Forty years of wicked problems literature: Forging closer links to policy studies. Policy and Society, 38(2), 180–197.CrossRefGoogle Scholar
Hughes, A. (2012). Enhancing impact: the value of public sector R&D: summary report: council for industry and higher education.Google Scholar
Jones, M. R. H. (2017). Can research quality be measured quantitatively? On quality of scholarship, numerical research indicators and academic publishing-experiences from Norway. Fennia-International Journal of Geography, 195(2), 164–174.CrossRefGoogle Scholar
Jordan, P. J., Sloan, T. R., Bentley, T., & Langerud, D. H. (2018). Australian and New Zealand Academy of Management (ANZAM) Research Productivity Survey Report 2014–2016. Retrieved from: https://www.anzam.org/wp-content/uploads/2018/12/ANZAM-RPS-2014-16.pdf.Google Scholar
Jorgenson, J., & Steier, F. (2013). Frames, framing, and designed conversational processes: Lessons from the world cafe. The Journal of Applied Behavioral Science, 49(3), 388–405.CrossRefGoogle Scholar
Körfgen, A., Förster, K., Glatz, I., Maier, S., Becsi, B., Meyer, A., Kromp-Kolb, H., & Stötter, J. (2018). It's a hit! mapping Austrian research contributions to the sustainable development goals. Sustainability, 10(9), 3295.CrossRefGoogle Scholar
Latour, B., & Woolgar, S. (2013). Laboratory life: The construction of scientific facts. Princeton, NJ: Princeton University Press.CrossRefGoogle Scholar
Leal Filho, W., Azeiteiro, U., Alves, F., Pace, P., Mifsud, M., Brandli, L., Caeiro, S. S., & Disterheft, A. (2018) Reinvigorating the sustainable development research agenda: The role of the sustainable development goals (SDG). International Journal of Sustainable Development & World Ecology, 25(2), 131–142.CrossRefGoogle Scholar
Leal Filho, W., Shiel, C., Paço, A., Mifsud, M., Ávila, L. V., Brandli, L. L., Molthan-Hill, P., Pace, P., Azeiteiro, U. M., Ruiz Vargas, V., & Caeiro, S. (2019). Sustainable development goals and sustainability teaching at universities: Falling behind or getting ahead of the pack?. Journal of Cleaner Production, 232, 285–294.CrossRefGoogle Scholar
Lucas, L. (2006). Research game in academic life. Maidenhead, UK: McGraw-Hill Education.Google Scholar
MacFarlane, A., Galvin, R., O'Sullivan, M., McInerney, C., Meagher, E., Burke, D., & LeMaster, J. W. (2017). Participatory methods for research prioritization in primary care: An analysis of the world café approach in Ireland and the USA. Family Practice, 34(3), 278–284.Google ScholarPubMed
Maskrey, N., & Underhill, J. (2014). The European statements of hospital pharmacy: Achieving consensus using Delphi and world café methodologies. European Journal of Hospital Pharmacy, 21(5), 264–266.CrossRefGoogle Scholar
McKiernan, P., & Glick, W. H. (2017). Why care about impact? EFMD Global Focus, 11(1), 18–21.Google Scholar
Moosa, I. A. (2016). A critique of the bucket classification of journals: The ABDC list as an example. Economic Record, 92(298), 448–463.CrossRefGoogle Scholar
National Health and Medical Research Council (2018) Research policy: Research translation and impact. Canberra, Australia.Google Scholar
Prewitt, V. (2011). Working in the café: Lessons in group dialogue. The Learning Organization, 18(3), 189–202.CrossRefGoogle Scholar
Quinlan, C., Babin, B., Carr, J., & Griffin, M. (2019) Business research methods (2nd ed.). Andover, UK: South Western Cengage.Google Scholar
RRBM (2017). A vision of responsible research in business and management: striving for useful and credible knowledge (An RRBM Position Paper (November 22, 2017)). Retrieved from https://rrbm.network/position-paper/.Google Scholar
Sachs, J. D. (2012). From millennium development goals to sustainable development goals. The Lancet, 379(9832), 2206–2211.CrossRefGoogle ScholarPubMed
Salvia, A. L., Leal Filho, W., Brandli, L. L., & Griebeler, J. S. (2019). Assessing research trends related to sustainable development goals: Local and global issues. Journal of Cleaner Production, 208, 841–849.CrossRefGoogle Scholar
Sangster, A. (2015). You cannot judge a book by its cover: The problems with journal rankings. Accounting Education, 24(3), 175–186.CrossRefGoogle Scholar
Santos, J. M., & Horta, H. (2018). The research agenda setting of higher education researchers. Higher Education, 76(4), 649–668.CrossRefGoogle Scholar
Saric, J., Blaettler, D., Bonfoh, B., Hostettler, S., Jimenez, E., Kiteme, B., Koné, I., Lys, J., Masanja, H., Steinger, E., Upreti, B. R., Utzinger, J., Winkler, M. S., & Breu, T. (2019). Leveraging research partnerships to achieve the 2030 agenda: Experiences from North-South cooperation. GAIA-Ecological Perspectives for Science and Society, 28(2), 143–150.CrossRefGoogle Scholar
Saunders, M. N. K., & Townsend, K. (2016). Reporting and justifying the number of interview participants in organization and workplace research. British Journal of Management, 00, 1–17.Google Scholar
SDSN Australia/Pacific (2017). Getting started with the SDGs in universities: A guide for universities, higher education institutions, and the academic sector (Australia, New Zealand and Pacific Edition). Melbourne: Sustainable Development Solutions Network – Australia/Pacific.Google Scholar
Seglen, P. O. (1997). Why the impact factor of journals should not be used for evaluating research. BMJ: British Medical Journal, 314(7079), 498.CrossRefGoogle Scholar
Shattock, M. (2014). Can we still speak of there being an academic profession? History of Education, 43(6), 727–739.CrossRefGoogle Scholar
Simsek, Z., Bansal, P., Shaw, J. D., Heugens, P., & Smith, W. K. (2018). From the editors – seeing practice impact in new ways. Academy of Management Journal, 61(6), 2021–2025.CrossRefGoogle Scholar
Smith, M. S., Cook, C., Sokona, Y., Elmqvist, T., Fukushi, K., Broadgate, W., & Jarzebski, M. P. (2018). Advancing sustainability science for the SDGs. Sustainability Science, 13(6), 1483–1487.CrossRefGoogle ScholarPubMed
Steier, F., Brown, J., & Mesquita da Silva, F. (2015). The world cafe in action research settings. Chapter 20: The world cafe in action research settings. In Bradbury, H. (Ed.), The Sage handbook of action research (third) (pp. 211–219). London, Thousand Oaks: Sage Publication.CrossRefGoogle Scholar
United Nations [UN] (2016). Sustainable development goals. Department of economic and social affairs. Available at https://sustainabledevelopment.un.org/sdgs (accessed December 19, 2019)Google Scholar
United Nations [UN] (2018). The sustainable development goals report 2018. New York, NY: United Nations Publications. Available at https://unstats.un.org/sdgs/files/report/2018/TheSustainableDevelopmentGoalsReport2018-EN.pdf (accessed April 24, 2020).Google Scholar
United Nations [UN] (2019). The sustainable development goals report 2019. New York, NY: United Nations Publications. Available at https://unstats.un.org/sdgs/report/2019/The-Sustainable-Development-Goals-Report-2019.pdf (accessed April 24, 2020).Google Scholar
Wang, H., Tong, L., Takeuchi, R., & George, G. (2016). Corporate social responsibility: An overview and new research directions. Academy of Management Journal, 59(2): 534–544.CrossRefGoogle Scholar
Williams, K., & Grant, J. (2018). A comparative review of how the policy and procedures to assess research impact evolved in Australia and the UK. Research Evaluation, 27(2), 93–105.CrossRefGoogle Scholar