This paper examines the construction and use of data sets in political science. We focus on three interrelated questions: How might we assess data quality? What factors shape data quality? and How can these factors be addressed to improve data quality? We first outline some problems with existing data set quality, including issues of validity, coverage, and accuracy, and we discuss some ways of identifying problems as well as some consequences of data quality problems. The core of the paper addresses the second question by analyzing the incentives and capabilities facing four key actors in a data supply chain: respondents, data collection agencies (including state bureaucracies and private organizations), international organizations, and finally, academic scholars. We conclude by making some suggestions for improving the use and construction of data sets.

It is a capital mistake, Watson, to theorise before you have all the evidence. It biases the judgment.

—Sherlock Holmes in “A Study in Scarlet”

Statistics make officials, and officials make statistics.”

—Chinese proverb

Limited dependent variable (LDV) data are common in political science, and political methodologists have given much good advice on dealing with them. We review some methods for LDV “change point problems” and demonstrate the use of Bayesian approaches for count, binary, and duration-type data. Our applications are drawn from American politics, Comparative politics, and International Political Economy. We discuss the tradeoffs both philosophically and computationally. We conclude with possibilities for multiple change point work.

How should one analyze data when the underlying models being tested are statistically intractable? In this article, we offer a simulation approach that involves creating sets of artificial data with fully known generating models that can be meaningfully compared to real data. The strategy depends on constructing simulations that are well matched to the data against which they will be compared. Our particular concern is to consider concurrently how voters place parties on issue scales and how they evaluate parties based on issues. We reconsider the Lewis and King (2000) analysis of issue voting in Norway. The simulation findings resolve the ambiguity that Lewis and King report, as voters appear to assimilate and contrast party placements and to evaluate parties directionally. The simulations also provide a strong caveat against the use of individually perceived party placements in analyses of issue voting.

Would holding elections by mail increase voter turnout? Many electoral reform advocates predict that mail ballot elections will boost participation, basing their prediction on the high turnout rate among absentee voters and on the rise in voter turnout after Oregon switched to voting by mail. However, selection problems inherent to studies of absentee voters and Oregon give us important reasons to doubt whether their results would extend to more general applications of voting by mail. In this paper, we isolate the effects of voting in mail ballot elections by taking advantage of a natural experiment in which voters are assigned in a nearly random process to cast their ballots by mail. We use matching methods to ensure that, in our analysis, the demographic characteristics of these voters mirror those of polling-place voters who take part in the same elections. Drawing on data from a large sample of California counties in two general elections, we find that voting by mail does not deliver on the promise of greater participation in general elections. In fact, voters who are assigned to vote by mail turn out at lower rates than those who are sent to a polling place. Analysis of a sample of local special elections, by contrast, indicates that voting by mail can increase turnout in these otherwise low-participation contests.

Numerous studies demonstrate that the highly educated are more likely to vote. The literature has not determined, however, why this is the case. The panel structure of the Current Population Survey allows us to exploit exogenous life cycle differences across otherwise similar cohorts, so that the marginal effect of education can be isolated from unobserved correlated factors. We find that an additional year of schooling has very little impact on voter turnout.

Beginning in 1999, Curtis Signorino challenged the use of traditional logits and probits analysis for testing discrete-choice, strategic models. Signorino argues that the complex parametric relationships generated by even the simplest strategic models can lead to wildly inaccurate inferences if one applies these traditional approaches. In their stead, Signorino proposes generating stochastic formal models, from which one can directly derive a maximum likelihood estimator. We propose a simpler, alternative methodology for theoretically and empirically accounting for strategic behavior. In particular, we propose carefully and correctly deriving one's comparative statics from one's formal model, whether it is stochastic or deterministic does not particularly matter, and using standard logit or probit estimation techniques to test the predictions. We demonstrate that this approach performs almost identically to Signorino's more complex suggestion.

The recent article by Carrubba, Yuen, and Zorn (2007) (CYZ) attempts to relate the strategic random utility models in Signorino (1999, 2002, 2003) and in Signorino and Yilmaz (2003) to existing game theory practice and to existing statistical techniques. It contributes to this literature by reminding us that comparative statics analysis can be applied to the equilibria of these models. There are a number of claims in CYZ, however, that require clarification. In particular, the article's primary claim is that comparative statics analysis, in combination with one of three proposed statistical estimators, provides a simpler alternative to methods previously advocated. This claim (or combination of claims) is incorrect. When one examines the procedure CYZ recommends, it is no simpler for substantive researchers than anything previously recommended. Moreover, none of the proposed estimators are new: they are exactly the same methods introduced in Signorino (1999, 2003), in Signorino and Yilmaz (2003), in Signorino, Walker, and Bas (2002), and in Bas, Signorino, and Walker (2007).