4 - Case Selection after Regression

Summary

Before a scholar can begin case-study research to test, refine, or interpret a regression analysis, a surprisingly complex prior challenge needs to be addressed: selecting cases for in-depth analysis from the comparatively large data set necessary for regression. Seawright and Gerring (2008) have proposed a set of systematic case-selection rules as a solution to this challenge. Yet the question remains, which of the several available methods should be used in a given project? Several scholars arguing in favor of deliberate case selection have focused on three of these alternatives: typical cases, deviant cases, and extreme cases on the dependent variable. Others have argued for random sampling (Fearon and Laitin 2008: 764–66) or deliberate sampling intended to represent the full range of variation in the data (King et al. 1994: 139–46).

I argue that the existing advice is incomplete or misleading when the goal of case-study research is discovery. I develop this argument by showing that, across a wide range of goals, the alternatives with the best chances of facilitating discovery are either deviant-case selection or the rarely discussed alternative of selecting extreme cases on the main independent variable. This argument is developed with reference to a variety of discovery-related goals: searching for sources of measurement error in the dependent or key independent variables; testing for or trying to discover an omitted variable, which may or may not be correlated with the independent variable of interest; exploring hypotheses about causal pathways; finding a case with a causal effect close to the population mean; and discovering substantive sources of causal heterogeneity.

Unlike much existing research on qualitative and case-study methods, this discussion is not driven by reference to examples of excellent research. In many other kinds of qualitative research practices, the analyst's judgment is a central ingredient in the application of the research tool; consequently, there is a great deal to learn from studying how judgment was employed in notably successful applications of the research tools in question. By contrast, systematic case selection is an algorithmic process: it takes a certain kind of information as an input, and then follows logical or mathematical rules to convert that input into a case. Because systematic case-selection rules reduce scholars’ reliance on judgment, the statistical properties of case-selection algorithms are more important than notably successful examples of their application.