1 - Issues in corpus design

Summary

A corpus is not simply a collection of texts. Rather, a corpus seeks to represent a language or some part of a language. The appropriate design for a corpus therefore depends upon what it is meant to represent. The representativeness of the corpus, in turn, determines the kinds of research questions that can be addressed and the generalizability of the results of the research. For example, a corpus composed primarily of news reportage would not allow a general investigation of variation in English. Similarly, research based on a corpus containing a single type of conversation – such as conversations between teenagers – could not be generalized to conversation overall. Thus, whether you are designing a corpus of your own, choosing a corpus to use in a study, or reading others' corpus-based work, issues of representativeness in corpus design are crucial.

It is important to realize up front that representing a language – or even part of a language – is a problematic task. We do not know the full extent of variation in languages or all the contextual variables that need to be covered in order to capture all variation in texts. However, attention to certain issues will ensure that a corpus is as representative as possible, given our current knowledge of language. This methodology box introduces these issues, as well as some means for improving corpus design in the future.