A Robust Transformation Procedure for Interpreting Political Text

Abstract

In a recent article in the American Political Science Review, Laver, Benoit, and Garry (2003, “Extracting policy positions from political texts using words as data,” 97:311—331) propose a new method for conducting content analysis. Their Wordscores approach, by automating text-coding procedures, represents an advance in content analysis that will potentially have a large long-term impact on research across the discipline. To allow substantive interpretation, the scores produced by the Wordscores procedure require transformation. In this note, we address several shortcomings in the transformation procedure introduced in the original program. We demonstrate that the original transformation distorts the metric on which content scores are placed—hindering the ability of scholars to make meaningful comparisons across texts—and that it is very sensitive to the texts that are scored—opening up the possibility that researchers may generate, inadvertently or not, results that depend on the texts they choose to include in their analyses. We propose a transformation procedure that solves these problems.