6 - Combining Case Studies and Natural Experiments

Summary

Continuing the movement away from observational regression studies, this chapter considers designs that combine qualitative research with natural experiments. Natural experiments have become an important research design in the social sciences because they can – when all goes well – extend the powerful leverage for causal inference generated by randomization into domains of the social sciences where scholars lack the knowledge, the resources, or the capacity to intervene in a true experimental manner (Dunning 2012: 1–15). In such non-experimental situations, it is sometimes the case that some actor intervenes to assign a treatment of interest randomly: using a lottery to distribute vouchers for reduced-cost private schooling (Angrist et al. 2002), for example, or assigning new voting procedures to districts in what is argued to be an effectively random sequence over time (Hidalgo 2010). If such randomized interventions are strictly implemented, and if some collateral assumptions – to be discussed later – are met, then causal inference works the same as if the research design were a true experiment. After all, the law of large numbers (which justifies treating the control group in an experiment as revealing the average potential outcome of the treatment group under the control condition) does not care whether the person implementing a randomization has a PhD in the social sciences. Whether a randomization is part of an academic research project, is part of a policy for distributive justice, or arises for other reasons has no real bearing on causal inference: randomized intervention, with a few additional assumptions, justifies causal inference even if that was not the original intent of the intervention.

Recall the potential-outcomes framework: for each case i, the potential outcomes are Y_i,t under treatment and Y_i,c under control. The treatment variable, D_i, determines which of these is observed. If that treatment is randomly assigned – by the researcher or anyone else – then the law of large numbers applies, and the mean of Y_i,t among treatment cases becomes equal to the mean of Y_i,t among control cases as the sample size increases. This means that, as in a laboratory experiment, the treatment group and the control group can serve as each others’ counterfactuals, and causal inference at the group level is justified.