1 - On method : text-mining, corpora and the historical study of language

Summary

This chapter discusses text-mining, and details how it is used throughout this book. The first section charts the shifting relationship between historians, computing and quantification since 1945, and investigates why corpus-based approaches have found little favour to date. The second makes the theoretical case for text-mining for the analysis of historical language, particularly political language. The third moves to practice, and asks how and how far historians’ current methodology for text analysis, based primarily on close reading, might be improved by these techniques. The fourth and final section introduces the three main corpora used in this book, outlining how they were constructed, interrogated and interpreted. This section also offers a defence against the principal objections which might be made against this methodology.

Quantification and computing in history since 1945

To chart the development of computing in history, it is first necessary to understand the awkward relationship between historians and quantitative methods more generally. While statistical modelling is self-evidently intrinsic to economic history, this field has developed quantitative research traditions (such as cliometrics) largely in isolation, and is now usually categorised as a social science. In the 1960s and 1970s however, huge computer-driven quantitative projects were popular (especially in America) and millions of dollars were invested in the promise of a new social history which might delineate dominant patterns in human behaviour through the mass processing of demographic statistics. As Hans Kellner has observed, ‘historians were confidently processing larger and larger quantities of information, producing broader comparisons and wider conclusions, and, in general, extending the historical domain toward a goal of “total History”’. An example was the Philadelphia Project, which promised to measure social mobility to prove the truth or falsity of the ‘American Dream’. A second – on alliance aggregation – taxonomised and tabulated past diplomatic pacts, with the aim of building a scientific model which could specify and quantify the preconditions for successful peace settlements. Both took many years, and involved hundreds of researchers. However, neither produced more than enormous yet incomplete piles of punched cards recording thousands of readings from databases into which biographies and historical events – hastily forced into binary categories – had been put.