1. Introduction
Research ethics are a key component of research quality, meaning that quality research cannot exist without an awareness of the ethical implications that each decision carries in the research process (Plonsky, Reference Plonsky2023). When referring to research ethics, we consider Guillemin and Gillam's (Reference Guillemin and Gillam2004) characterization of ethics as a multidimensional construct comprising both procedural ethics and ethics in practice. Kubanyiova (Reference Kubanyiova2008) terms these macro- and micro-ethics, respectively. Procedural, or macro-, ethics refers to human subjects researchers’ navigation of the requirements of an ethics review board (i.e., the Institutional Review Board (IRB) in the US), whereas “ethics in practice”, or micro-ethics, refers to the day-to-day researcher decisions that carry ethical implications. Prior to Kubanyiova's (Reference Kubanyiova2008) discussion of the distinction between the two perspectives, many of the conversations around research ethics in applied linguistics (AL) focused heavily on navigating the IRB (see Yaw et al., Reference Yaw, Plonsky, Larsson, Sterling and Kytö2023).
In the spirit of extending these conversations to a global audience as part of a larger funded project on research ethics in quantitative humanities, two hybrid symposia were held in 2022, one at Northern Arizona University, USA (NAU) in September and the other at Uppsala University, Sweden (UU) in December. Bringing together quantitative humanities scholars who have focused on research ethics in their professional work, these symposia offered an opportunity to explore ethical considerations in various aspects of quantitative humanities research.
For these symposia, our focus on quantitative research ethics was driven primarily by the relative dearth of work in this area when compared with qualitative AL research (see De Costa, Reference De Costa2014, Reference De Costa2016). As scholars who conduct research using primarily quantitative methods ourselves, we viewed this as an opportunity to examine with a critical lens many of the assumptions that often accompany quantitative research such as objectivity and standardization of analytical protocols. As can be seen from the summaries of the talks below, ethics are woven into decisions made at every stage of the research process, from conceptualization of a project to disseminating research findings.
2. Selected talks and panels from the symposia
2.1 Ethics in language assessment (Isbell and Kim)
Conflicts of interest are widely acknowledged as a concern in research ethics (e.g., Steneck, Reference Steneck2007) and, in testing, researcher independence is highly valued when evaluating validity (Kane, Reference Kane2013). This presentation reported on the initial findings from a systematic review of high-stakes English proficiency test validation studies, with a focus on studies reported in five leading language testing journals over the last five years (2016–2021). Areas of focus in this presentation included conflict of interest (COI) disclosure and broad trends in research methodology, amongst others. The systematic review revealed that 65% of the validation studies included an identifiable COI statement, and that of the 67 studies with a potential conflict, only three disclosed these COIs. Regarding trends in research methodology, we found that research conducted or otherwise supported by a test developer (in contrast to independently conducted research) featured larger sample sizes and more frequently focused on aspects of validity related to the evaluation of responses, generalization of test scores, and extrapolation of test scores to performance in real-world domains. This presentation concluded with a set of recommendations for journals, authors, test developers, professional organizations, and government organizations for improving accuracy and transparency in COI reporting.
2.2 Editorial ethics in applied linguistics (Gass and Plonsky)
Journal editing is a task that is often laden with ethical dimensions. These tensions stem in part from the different stakeholders involved which include authors and reviewers, of course, but also the discipline as a whole as well as the journal publisher. Given that journal editing is often a high-stakes activity and journal editors are in a position to shape and give direction to a discipline, it is surprising that there is little in the way of training to help editors navigate some of the potential conundrums they inevitably face. Further complicating this role and the ethically-charged decisions editors make is the lack of transparency typically involved in the editorial/review process.
In the presentations we gave at NAU and UU, we sought to view editing through the lens of ethics in many of the decisions we have made as editors. We problematized some of the personal experiences we have encountered as editors of Studies in Second Language Acquisition related to, among other issues, (disputed) authorship, (non-)anonymity, excessive self-citation, non-collegiality by reviewers, and handling submissions by friends and colleagues, to name a few. We concluded by laying out what we see as several paths forward which we feel may lead toward more equitable and less ethically fraught editorial practice including a call for editor training and more research on editorial ethics.
2.3 Moral dilemmas in open access publication (Andringa)
The field of AL seems to be witnessing a gradual transition towards open access (OA) publication and instrument sharing, which stand to increase scientific and societal impact of research, transparency, research quality, and equal access. However, there are also factors that impede our efforts to move towards fully open and transparent scientific research practices. This presentation demonstrated that the choice of journals for publishing articles also involves moral dilemmas. One such moral dilemma pertains to OA journals that are not freely available to authors, which directly impedes equity/equal access. Journals following Gold and Hybrid models, for example, which implement author processing fees to access content, necessarily exclude authors on economic grounds owing to high author processing charges. These journals also tend to be journals of high prestige, which is determined by factors such as perceived quality, name recognition, editorial board, and journal impact factor. Publishing in prestigious journals is generally important for career advancement, making it far more attractive to researchers than OA journals. This presentation concluded with a call on academic societies and advanced career scholars to support and launch Diamond OA publication venues.
2.4 Questionable research practices (Larsson, Plonsky, Sterling, Kytö, Yaw, and Wood)
Questionable research practices (QRPs) are often viewed as the “murky waters” of research ethics and cover practices that fall between ideal behavior and absolute misconduct (e.g., falsification). Whether intentional or not, researchers make choices that may lead them to engage in QRPs (Isbell et al., Reference Isbell, Brown, Chan, Derrick, Ghanem, Gutiérrez Arvizu, Schnur, Zhang and Plonsky2022); for example, with limited space in a manuscript, a researcher might opt to leave out some detailed information in the methods section. Other examples include employing instruments without a strong validity argument and HARKing (hypothesizing after results are known). We developed a field-specific taxonomy (k = 58) for the quantitative humanities using the Delphi method (see Plonsky et al., Reference Plonsky, Larsson, Sterling, Kytö, Yaw, Wood, DeCosta, Rabie-Ahmed and Cinagliasubmitted; Sterling et al., Reference Sterling, Plonsky, Larsson, Kytö and Yaw2023). We then surveyed researchers (n = 230) active in the U.S. and Sweden to investigate (1) what types of QRPs and related ethical considerations they have done and (2) the perceived severity of these QRPs (see Larsson et al., Reference Larsson, Plonsky, Sterling, Kytö, Yaw and Woodunder review). The results showed that 95.7% of the respondents reported having engaged in one or more of the practices listed, with giving the same presentation at multiple conferences being the most common one (in 1–20% of their projects in the past five years). There was a strong negative correlation between frequency and severity.
2.5 Past and present of research ethics training (Wood, Yaw, and Sterling)
This session examined research ethics training literature from the 1980s to 2022, using historical trends and materials to jumpstart discussions about current training practices. By tracing the development of research ethics training literature, an expansion can be noted in focus from ethical guidelines for informed consent in the 1980s to research methods textbooks with dedicated ethics chapters in the 2000s to exploration of micro-ethical decision points and QRPs in the 2010s and beyond (see Yaw et al., Reference Yaw, Plonsky, Larsson, Sterling and Kytö2023).
A survey of the current state of research ethics training in AL revealed three key themes: (1) a focus on research ethics pertaining to informed consent, (2) recurring reference to select decision points in the research process (e.g., selection of the appropriate research design), and (3) ethics presented in single sections or chapters of textbooks. Based on these themes, we provide recommendations for research ethics training materials development: expand ‘research ethics’ to include ethical gray-zone issues, be thorough and explicit in discussing these issues in our materials, and incorporate research ethics as a recurring theme throughout courses and textbooks (see Wood et al., Reference Wood, Sterling, Larsson, Plonsky, Kytö and Yawunder review).
To conclude the session, attendees reflected on personal experiences teaching research methods courses, including the research ethics training materials they use and those they would like to have available.
2.6 Creating training materials (Sterling)
Training materials for research ethics often include case scenarios that contain short summaries of an event along with an ethical complication. These scenarios allow scholars to consider ethical dilemmas within a safe space without having to personally experience them, provide interesting talking points for classroom discussions, and prompt critical thinking and future planning on ways to manage ethical challenges. However, scenarios rarely include solutions, and if they do, from a limited number of voices. Instead, they rely on the reader to evaluate the evidence in the case and learn the intended lesson. For this interactive session, we created a set of five scenarios (link below) that included one or more QRPs from our Delphi study (see Plonsky et al., Reference Plonsky, Larsson, Sterling, Kytö, Yaw, Wood, DeCosta, Rabie-Ahmed and Cinagliasubmitted; Sterling et al., Reference Sterling, Plonsky, Larsson, Kytö and Yaw2023). Audience members read each prompt and then worked in groups to discuss issues related to the case and possible solutions they would offer. We additionally asked participants to submit short write-ups on how they would advise a person facing the issues included in each scenario. These responses were shared as part of our team's online training toolkit for researchers in the quantitative humanities to provide improved self-study and in-class training on QRPs.
2.7 Future directions in ethics training (Hancock, Gass, Isbell, Larsson, Plonsky, and Sterling)
In the NAU symposium, a lively panel discussion was held on navigating the many ethical decision points throughout the quantitative research process. Three key takeaways emerged.
First, quantitative research ethics is not binary: research, researchers, and research-specific decisions are not either ethical or unethical, nor do we benefit from such thinking. The research process requires myriad decisions and a willingness to recognize and wade into their grey areas, thoughtfully, transparently, collaboratively, and nonjudgmentally.
Second, much ethics training happens through faculty deliberately mentoring their graduate students, fostering a culture of researcher reflectivity and reflexivity. This includes explicit instruction as well as actively modeling the process of continuously interrogating the critical decisions and assumptions we make throughout the quantitative research process.
Third, research ethics goes beyond the key decision points in each individual research project, for which faculty mentors, those teaching research methodology, and indeed our professional societies are active in developing and offering guidance. The research process also includes dissemination of that research – manuscript reviewing, journal editing, and publishing more broadly – where little formal guidance currently exists for navigating ethical decision points and where training developed and provided by our professional societies could be quite beneficial.
3. Conclusion
The symposia brought together participants from all over the world and enabled a forum for highly fruitful discussion. Overall, although the topic of research ethics has been given far more limited focus than many other topics of relevance to research methods and training, we see many promising developments in the field that suggests that research ethics is gaining momentum. For example, there have been colloquia devoted specifically to research ethics at recent conferences (e.g., Isbell, Reference Isbell2023; Rabie-Ahmed et al., Reference Rabie-Ahmed, Cinaglia, Manchon and De Costa2022) and there is a special issue underway in Research Methods in Applied Linguistics (Isbell & De Costa, Eds.). We believe and hope that this is just the start.
Learn more about our symposia
For information, slides, and sample QRP scenarios from the symposium at Northern Arizona University, please see: https://sites.google.com/view/qrp-humanities/home/events/nau-research-ethics-symposium-2022
For information and slides from the symposium at Uppsala University, please see: https://sites.google.com/view/qrp-humanities/home/events/uu-research-ethics-symposium-2022
For more information about this funded project, please see: https://sites.google.com/view/qrp-humanities
Acknowledgements
In addition to the grant funding agencies that made these symposia possible, we are grateful to all the people who took time out of their busy schedules to attend and contribute to the discussion.
Financial support
This project was co-funded by the Swedish Research Council, the Bank of Sweden Tercentenary Foundation, and the Royal Swedish Academy of Letters, History and Antiquities (Project ID: FOE20-0017) as part of a larger grant project: Questionable research practices: The (un)ethical handling of data in quantitative humanities research (Larsson, Plonsky, Sterling, Kytö, Yaw, & Wood).
Open access
