Command Lines for the Humanities

I begin my classes about poetry by asking my students what they already know. How do they know a poem when they see one? Would they be able to tell the difference between a sonnet and a ballad from across the room or in a column of newsprint? If they are not seeing white space on a page, what do they think a poem should sound like? Do any of these things fit precisely with that thing we might refer to as Poetry with a capital P? The metaphor of pretraining a model has been useful in approaching the question of how our formation as students and critics of literature prepares us to interpret different kinds of poetic genres—or of how we might prepare ourselves to be better readers of genres as such. The more we know about historical genres and media formats, the better we can understand historical contexts and therefore interpret historical texts.¹ And the more we know about the circulation of these genres and their formats, the more we understand about the power structures that underlie our access to the cultural materials we study, now largely mediated not only by the books in school rooms and libraries but by the companies that sell the subscriptions those libraries purchase (or don't purchase) so that we might access (or not have access to) collections of digitized materials from the past.² Asking what poetry means in the era of large language model (LLM) outputs allows us to approach historical and contemporary reading practices alongside questions about our own training: What is it we think we know about poems now and in the past, and where did that knowledge come from?

Prompted by that question, I've spent my career studying (collecting, counting, classifying, interpreting) the history of scholars of literature and linguistics (before these were separate disciplines) who argued about how to define poetry, from its smallest segments to its forms and genres and societal impacts. I'm therefore not too surprised that we now find ourselves in a moment in which, as Matthew Kirschenbaum and Rita Raley write, “linguistic protocols have been eclipsed by arithmetic means.” In some ways that was the dream of one branch of linguistics’ study of prosody—the branch of linguistics from which English departments split in the early twentieth century. This split, I might argue, still haunts our interpretations (at the most granular level) of poems and our conceptions (at the macro level) of the discipline of English poetic criticism.³ Agreeing with Katherine Elkins in our convention seminar, I would argue that our current models for academic research, writing, publication, service, and promotion disallow and disincentivize us from any response other than what aligns with expected, traditional, individual modes of humanities research and undermine collaborative modes at work in linguistics. Could we retrofit some of our disciplinary habits or develop new habits in order to think beyond our disciplinary formations? To the institutional models that Kirschenbaum and Raley identify, I add a few more that I would like to see become normalized within the profession.

Collaboration → Forums, Open Conversations, Working Groups

The seminar format of the conversation at the MLA convention meant that we were encouraged to share papers in advance of the discussion. Working papers and works in progress are important in any new field, and there is plenty of work to go around. When I saw Rita Raley and Seth Perlow again in the seminar AI and/as Form the annual meeting of the American Comparative Literature Association (ACLA) a few months later, it felt like a continued conversation. I commend PMLA for publishing these essays so soon after the MLA convention in January. We all have to be careful about how we manage our extrainstitutional commitments, and those of us working in this space are often too overtaxed to provide what Kirschenbaum and Raley call “sacrificial labor” for our various institutions (I am on the AI committees for the dean of the college, the dean for research, and the provost, and on five other campus executive committees; I served on the interdisciplinary data science search committee, and I direct the Center for Digital Humanities). And yet, what if we imagined, instead of containment as an institutional response, proliferations of new formats for collaborative, interinstitutional work that would lead to collaborative outputs? This is already happening, but faculty members in modern languages are not rewarded or recognized for this work. Institutional administrators and faculty members alike should cite and acknowledge these ongoing, iterative policy papers; they rely on hard-won expertise and years of research and experience working with and thinking about language.⁴ Yes, we are helped by products from the tech industry here. Let's leverage Slack, Zoom, Google Drive, and sync our Outlook calendars across multiple time zones. Fostering and sustaining interinstitutional collaboration—especially with our global colleagues who have different regulations and funding structures—is beneficial not only to us as individual scholars but to the entire profession.⁵ I don't mean to reenact an era of manifestos within the profession as it currently stands (though the genre of the manifesto document might be helpful);⁶ I mean to encourage avenues for collaborative, proactive response and cooperation across institutions and associations so that we might continue to reimagine the profession ourselves. Perhaps with enough participation and shared knowledge we could (in some cases) intervene to prevent the inevitability of foolhardy reliance on all manner of questionable AI-related products and services.⁷ At the very least, we might take advantage of our interest in cocreated textual outputs to think about collaboratively writing a research paper.⁸

Communication → Rapid Response Publications, Communicating to Broader Audiences

We also need to ensure that these collaborations become, in Kirschenbaum and Raley's words, “capturable by the metrics of performance review.” Now more than ever, we must move away from valuing the single-author monograph or peer-reviewed article as the sole marker of achievement. Publishing, circulating, citing, and valuing as scholarly labor the outputs of these collaborative working groups and forums is vital, and our funding structures, strapped though they may be, need to support this scholarly infrastructure broadly and effectively.⁹ Scholars need to cite scholar-built resources and workflows (software, web applications, databases) as research outputs. We need to, once and for all, eliminate the sense that public writing or public scholarship is not real scholarship. We need to take seriously the idea that in some cases we cannot wait to respond until we have an article-length argument that will come out in a peer-reviewed publication in twelve or fourteen months. We should regularize the shorter forms that allow us to distill complicated arguments to a broader public even—maybe especially—when they correspond to longer-form writing.¹⁰ Shifting our values toward collaboration and shorter, process-oriented publications will help us intervene in areas of machine learning (style, character) where we are experts.¹¹ It is not easy to move outside our regular models of publication and promotion, and I am not advocating that we abandon them but that we meaningfully expand them. I would click the bait that advertised a review of Character.ai by my colleagues who study character or agency, for example, and I would like for important evaluative work to count at the institutional level. Can we work with our university communications units to get these and other interventions, engagements, and critiques into op-eds and on social media platforms? Is the endgame to improve the closed LLMs with our input, to provide drag on the corporations that control them, or to build our own alternatives and explain them to different publics? I think that will depend on the scholar, but the more we own our expertise and turn it into actionable and, yes, marketable, interventions, the more likely our expertise will be taken seriously by researchers who are publishing papers at a lightning pace. Or, to put it another way, we already know how to find relevant data and to succeed in tasks that aren't easily measured by simple metrics, but we need to fine-tune our ability to communicate our expertise in simpler ways.

These systems are not one monolithic entity. What parts of our own training might be applicable for critical interventions, and how can we collectively respond in public (in addition to publishing our white papers and special issues and organizing seminars)? I believe strongly in the need for projects like AI for Humanists, but I believe even more in the need for humanities for AI.¹² We know more about character, style, image, creativity, poetry, and so forth, than computer scientists, and we likely always will, but we are not great at translating this knowledge into the kinds of critique or intervention that broader publics will read, nor are we rewarded for doing so. Of course, many of us are already collaborating meaningfully with researchers in machine learning, but here I mean also the kind of critique that comes from those who are not interested in collaboration per se, but whose critique exposes how much better an output could have been had the experts (the humanists) been consulted. All the interventions Kirschenbaum and Raley propose could use better circulation beyond the pages of this publication, but particularly the impetus to “bring the entire tradition of critical judgment—as it has been honed, debated, and theorized over the centuries—to bear on the problem of how to qualitatively evaluate model output.”

Method → Making Data Work for Us

How can we put our values as humanists into practice in a way that is applied rather than theoretical? We must focus on and clearly define our method, our praxis, as something that translates into the language of workflow. Some of you may be rolling your eyes, but others may be looking at the list of grant deadlines or going over Gantt charts and time lines and feeling grateful that you had to translate your research project into a work plan. When we do this, we break down how we work into its important component parts. We aren't forced to slow down and think about our process as researchers and writers as often when we have an argument or an idea for an article or a book. How do we do research? How does our writing process—and our planning for our writing—inform the arguments we make or the histories we write? I am asking whether we might translate our epistemic practices into communicable information, into a kind of workflow that might be broadly applicable (I would argue that these practices are already broadly applied, but not by experts). Humanities approaches are essential for understanding data and for understanding the world; they are not extra- or cocurricular. If, and let's just speculate here, everything becomes a branch of data science, then let's make it a slower, humanistic data science in which contextualization, close reading, critical thinking, collaboration, and communication are the starting points. We need more work, not less, that explains how we do what we do and why it matters. We need a command line for the humanities we can argue for beyond (and alongside) the claim that we are uniquely equipped to intervene.¹³

Collectives → Datasets, Data Reviewing, Open Repositories, Data in the Humanities

Kirschenbaum and Raley ask us to move past copyright battles and “instead work to cultivate and support alternative data resources. Let there be legibility all the way down.” If we are going to work to cultivate and support alternative data resources and build community data hubs, then we must value and acknowledge the work of curating and creating datasets fairly within the profession. The Post45 Data Collective and the Nineteenth-Century Data Collective are working toward an umbrella Cultural Data Collective, and the Journal of Cultural Analytics and the Journal of Open Humanities Data both review datasets among other nontraditional outputs.¹⁴ If we are going to value data work as scholarly labor then all of us who are in positions of power must meaningfully intervene at the moment of hiring or of tenure and promotion to recognize datasets and data essays as meaningful scholarly interpretation. Scholars need to be trained to understand their cultural materials as data and rethink their relationships with their already highly mediated forms of digitally assisted research and with the people who have been doing the labor of creating and maintaining these digitally mediated sources. Publishing datasets as research outputs is a good start, but building a scholarly community that knows how to read and evaluate a dataset is the dream. How many faculty members who read review files know how to review a dataset? Graduate students nowadays, one hopes, learned the difference between a text file and a page image as undergraduates in any course where their materials were uploaded electronically into a content management system. We all take a moment to explain this important mediation, don't we?¹⁵

I would add to Kirschenbaum and Raley's remarks on data work that open-source tools and datasets require the often invisible labor of creating, curating, reviewing, and updating or versioning in addition to storing and providing a platform for retrieval of those tools and datasets. For the latter, the default community data hubs are a Wild West. Alan Liu (in, e.g., “Data Moves”) and others in digital humanities have known for years that curated humanities datasets for research are important for all kinds of exploratory data analysis and discovery, and that how we access and think through our resources as data is a crucial first step. If we are going to truly do the work of collaborating on datasets for language models, then we need to reconsider what we mean by “open” and why open access matters (HathiTrust sends me a daily email notifying me that certain texts are no longer available because of Google's copyright restrictions). Legibility all the way down, the cultivating and supporting of alternative data resources (deciding parameters, training annotators) relies on a huge amount of work that our profession has yet to learn how to measure, and it often relies on the invisible and undercompensated labor of information specialists and librarians who are tasked with doing coordination work on our behalf in addition to their other full-time jobs. We need to build relationships and collaborations with the people on whose labor our scholarly infrastructure already relies, if we haven't already done so.

I would not know anything about poetry without my decades-long conversation with the Historical Poetics reading group. I would not be able to participate in any conversations about AI or data science in the humanities had I not wanted to build a database so that I could search for the weird scansion marks that prosodists made on poems. When I started on this path, I didn't know the difference between a dataset and a website, and I certainly couldn't have told you what natural language processing is. The researchers at the Center for Digital Humanities at Princeton University have been interlocutors and collaborators and advocates for the humanities in many ways that I reflect on here only tangentially, from holding institutes on data science and the humanities to organizing sessions supported by the National Endowment for the Humanities on linguistic diversity in natural language processing to getting humanities datasets into Introduction to Data Science classrooms (and training graduate students in R along the way). Some days I would like to be able to think about a poem all day. Some days I still do that. That's what I naively thought my job as a professor was going to be. But historical poetry, though seemingly made more accessible by large-scale digitization than ever before, was also rendered unsearchable by Google Books’ OCR, and my own pretrained assumptions about literary value and archives and canons and literary history and even the affordances of particular kinds of close reading began to stretch so thin I could see right through them. We are already in a moment where our scholarly research and writing exceed the knowledge infrastructures and systems of evaluation of our profession. How quickly we buy into and build new infrastructures and systems is up to us.