We write with news of an initiative that the Qualitative Data Repository (QDR, www.qdr.org) is undertaking in partnership with Cambridge University Press (www.cambridge.org) and the technology firm Hypothes.is (https://hypothes.is/), with the generous support of the Robert Wood Johnson Foundation (www.rwjf.org) and the National Science Foundation. The Annotation for Transparent Inquiry (ATI) project seeks to enhance the transparency of qualitative and multi-method research in the health and social sciences. In this short letter we offer some background on the initiative, and let PS readers know about an associated working group.
PS readers are likely to be familiar with supplemental materials—study datasets, relevant information about their creation, and the code used for analysis—being provided to enhance the transparency of quantitative and computational social science. These materials are shared in ways that are consonant with the numeric nature of the data and their analysis as an aggregate body of information. That analysis is typically summarized in tabular form in the text or appendix of published work (see figure 1).
Figure 1 Quantitative Research – Matrix Data
In qualitative research, by contrast, data—text, audio, video, and images—are used and represented somewhat differently. Rather than being analyzed holistically, the data are analyzed individually or in small groups, and serve as distinct inputs to an analysis, supporting discrete inferences and interpretations in the relevant publication. Moreover, the data and their analysis are interwoven through the text of a publication (see figure 2).
Figure 2 Qualitative Research – Individual Pieces of Data
Making qualitative research more transparent and accessible—and therefore more easily disseminated and more quickly incorporated into academic and public dialogue—thus presents a variety of substantive, technical, and ethical challenges. Addressing those challenges requires coordinated engagement by publishers, annotation platforms, and digital repositories.
QDR, Cambridge University Press, and Hypothes.is, with the generous support of the Robert Wood Johnson Foundation, have partnered to develop a solution: ATI. QDR is a domain repository dedicated to curating, preserving and publishing the data underlying qualitative and multi-method research in the health and social sciences. Its operations are supported by the National Science Foundation (Award Nos. 1424191 and 1628636). Cambridge University Press is a leading publisher of social science scholarship. Hypothes.is is a non-profit technology firm that develops open source software enabling the creation, storage, and sharing of web-standard annotations. The Robert Wood Johnson Foundation is the largest philanthropy focused exclusively on health in the United States, and supports research related to improving population health, well-being, and equity. ATI is a new approach to achieving transparency in qualitative research in the health and social sciences.
ATI draws on Andrew Moravcsik’s (Reference Moravcsik2010; Reference Moravcsik2014a; Reference Moravcsik2014b) pioneering work on active citation. ATI is based on the concept of “open annotation,” which allows for the generation, sharing, and discovery of digital annotations across the web. Digital annotations to published scholarship based on qualitative data include analytic notes (that discuss the generation and analysis of the data and how they support textual claims/conclusions), excerpts from data sources (and translations as needed), and potentially the sources themselves (i.e., the work’s evidentiary base, when these can be shared ethically and legally). Annotations can be mapped across the span of a book or article—linked to discrete passages in the text where data sources are deployed. The annotations serve as a digital overlay that can be displayed in tandem with any manuscript on any webpage, appearing right next to annotated text. In addition, the annotations and sources are compiled into a “data supplement” that is viewable in a repository and assigned a persistent identifier (e.g., a Digital Object Identifier [DOI]).
ATI thus augments the main text and footnotes of a research publication, allowing authors to include more substantive content in the text by reducing the need for block quotations or discursive footnotes. By accelerating access to research-enhancing information and easing readability, ATI dramatically strengthens and amplifies the impact of qualitative research. The “ATI at a Glance” page on the QDR website offers more information, including links to a few early prototypes (https://qdr.syr.edu/guidance/ati-at-a-glance), and another page offers some basic instructions for using ATI (https://qdr.syr.edu/deposit/ati-instructions).
QDR and its partners are currently working to assemble a group of approximately 35 outstanding scholars from various health and social science disciplines who will serve as pioneers for this new approach to transparency. Members of this “ATI Pilot Working Group” will produce and evaluate a diverse set of pilot projects that will illustrate the power and promise of open annotation and demonstrate the utility of ATI. Our goal is to include scholars who employ different analytic approaches and diverse types of qualitative data, and study different geographical regions and time periods. This diversity will allow us to identify different ways in which annotation contributes to academic research and publishing. Each working group member will be paired with a graduate student interested in the relevant research topic who will review their pilot project using a pre-determined set of criteria. Each author and graduate student will receive a modest honorarium.
Half of the working group members will “retroactively” annotate an article based on qualitative evidence that they recently published, or have forthcoming, in an academic journal. The other half will employ ATI as they write a piece that they will later submit to an academic journal for review. Once working group members submit their annotation projects to QDR, the repository will curate them and use Hypothes.is technology to generate open annotations. Graduate students will then evaluate the use of ATI. Ultimately, the annotations will be linked to the digital version of the article published on the journal web site. (The ATI Data Supplement will also be published on the QDR site.)
We will invite ATI Pilot Working Group members and graduate student assessors to attend two workshops funded by the Robert Wood Johnson Foundation, in February and September of 2018. In those workshops, we will discuss the experience of, challenges presented by, and promise of using ATI to enhance the openness of qualitative scholarship, and consider ways in which the approach can be further developed.
Please get in touch with Colin Elman (celman@syr.edu) or Diana Kapiszewski (dk784@georgetown.edu) if you have questions or are interested in joining QDR, Cambridge University Press, and Hypothesis in this initiative.
