Hostname: page-component-cd9895bd7-hc48f Total loading time: 0 Render date: 2024-12-26T08:09:19.541Z Has data issue: false hasContentIssue false
  • English
  • Français

A Dialogue with the Data: The Bayesian Foundations of Iterative Research in Qualitative Social Science

Published online by Cambridge University Press:  13 February 2019

Tasha Fairfield  and
Andrew Charman
Get access Rights & Permissions [Opens in a new window]

Abstract

We advance efforts to explicate and improve inference in qualitative research that iterates between theory development, data collection, and data analysis, rather than proceeding linearly from hypothesizing to testing. We draw on the school of Bayesian “probability as extended logic,” where probabilities represent rational degrees of belief in propositions given limited information, to provide a solid foundation for iterative research that has been lacking in the qualitative methods literature. We argue that mechanisms for distinguishing exploratory from confirmatory stages of analysis that have been suggested in the context of APSA’s DA-RT transparency initiative are unnecessary for qualitative research that is guided by logical Bayesianism, because new evidence has no special status relative to old evidence for testing hypotheses within this inferential framework. Bayesian probability not only fits naturally with how we intuitively move back and forth between theory and data, but also provides a framework for rational reasoning that mitigates confirmation bias and ad-hoc hypothesizing—two common problems associated with iterative research. Moreover, logical Bayesianism facilitates scrutiny of findings by the academic community for signs of sloppy or motivated reasoning. We illustrate these points with an application to recent research on state building.

Type
Reflection
Information
Perspectives on Politics , Volume 17 , Issue 1 , March 2019 , pp. 154 - 167
Copyright
Copyright © American Political Science Association 2019 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

They thank Andy Bennett, Ruth B. Collier, David Collier, Justin Grimmer, Macartan Humphreys, Alan Jacobs, Jack Levy, James Mahoney, Jason Sharman, Hillel Soifer, and Elisabeth Wood for detailed comments and intellectual engagement. They are also grateful to journal editor Michael Bernhard, Devin Caughey, Christopher Darnton, Steven Goodman, Jacob Hacker, Antoine Maillet, James Mahon, Richard Nielsen, Craig Parsons, Jessica Rich, and Ken Shadlen, as well as seminar participants at the Center for Advanced Study in the Behavioral Sciences, the Syracuse Institute for Qualitative and Multi-Method Research, Rutgers, Princeton, Yale, University of Texas–Austin, University of California–Berkeley, and the University of Oregon.

A list of permanent links to Supplemental Materials provided by the authors precedes the References section.

References

Ansell, Ben and Samuels, David. 2016. “Journal Editors and ‘Results-Free’ Research: A Cautionary Note.” Comparative Political Studies 49(13): 1809–15.Google Scholar
Beach, Derek and Pedersen, Rasmus. 2013. Process-Tracing Methods: Foundations and Guidelines . Ann Arbor: University of Michigan Press.CrossRefGoogle Scholar
Bennett, Andrew. 2015. “Appendix: Disciplining Our Conjectures: Systematizing Process Tracing with Bayesian Analysis.” In Process Tracing in the Social Sciences: From Metaphor to Analytic Tool, ed. Bennett, Andrew and Checkel, Jeffrey. New York: Cambridge University Press.Google Scholar
Bennett, Andrew and Checkel, Jeffrey, eds. 2015. Process Tracing in the Social Sciences: From Metaphor to Analytic Tool. New York: Cambridge University Press.Google Scholar
Bowers, Jake, Nagler, Jonathan, Gerring, John, Jacobs, Alan, Green, Don, and Humphreys, Macartan. 2015. “A Proposal for a Political Science Registry.” Available at http://blogs.bu.edu/jgerring/files/2015/09/AproposalforaPoliticalScienceRegistry.pdf .Google Scholar
Brady, Henry and Collier, David. 2010. Rethinking Social Inquiry. Lanham: Rowman and Littlefield.Google Scholar
Broockman, David, Kalla, Joshua, and Aronow, Peter. 2015 “Irregularities in LaCour (2014).” Available at https://stanford.edu/∼dbroock/broockman_kalla_aronow_lg_irregularities.pdf.Google Scholar
Büthe, Tim and Jacobs, Alan. 2015. “Conclusion: Research Transparency for a Diverse Discipline.” Qualitative and Multimethod Research: Newsletter of the American Political Science Association’s QMMR Section 13(1): 50–63.Google Scholar
Cox, Richard. 1961. The Algebra of Probable Inference . Baltimore, MD: Johns Hopkins University Press.Google Scholar
Fairfield, Tasha and Charman, Andrew. 2017. “Explicit Bayesian Analysis for Process Tracing,” Political Analysis 25(3): 363–80.CrossRefGoogle Scholar
Glaser, Barney and Strauss, Anselm. 1967. The Discovery of Grounded Theory: Strategies for Qualitative Research. Chicago: Aldine Publishers.Google Scholar
Gregory, Phil. 2005. Bayesian Logical Data Analysis for the Physical Science. New York: Cambridge University Press.CrossRefGoogle Scholar
Humphreys, Macartan and Jacobs, Alan. 2015. “Mixing Methods: A Bayesian Approach.” American Political Science Review 109(4): 653–73.CrossRefGoogle Scholar
Humphreys, Macartan, Sanchez de la Sierra, Raul, and van der Windt, Peter. 2013. “Fishing, Commitment, and Communication: A Proposal for Comprehensive Nonbinding Research Registration.” Political Analysis 21: 1–20.Google Scholar
Hunter, Douglas. 1984. Political/Military Applications of Bayesian Analysis. Boulder, CO: Westview Press.Google Scholar
Jackman, Simon and Western, Bruce. 1994. “Bayesian Inference for Comparative Research.” American Political Science Review 88(2): 412–23.Google Scholar
Jacobs, Alan. 2019. “Pre-registration and Results-Free Review in Observational and Qualitative Research.” In The Production of Knowledge , ed. Elman, Colin, Gerring, John, and Mahoney, James. New York: Cambridge University Press.Google Scholar
Jaynes, E. T. 2003. Probability Theory: The Logic of Science. New York: Cambridge University Press.10.1017/CBO9780511790423CrossRefGoogle Scholar
Jeffrey, Richard. 1983. The Logic of Decision. Chicago: University of Chicago Press.Google Scholar
Jefferys, William. 2003. “Bayes’ Theorem,” Journal of Scientific Exploration 17(3): 537–42.Google Scholar
Jefferys, William. 2007. “Bayesians Can Learn from Old Data.” Available at https://repositories.lib.utexas.edu/bitstream/handle/2152/29425/BayesiansOldData.pdf?sequence=1.Google Scholar
Kapiszewski, Diana, MacLean, Lauren, and Read, Benjamin. 2015a. Field Research in Political Science. New York: Cambridge University Press.10.1017/CBO9780511794551CrossRefGoogle Scholar
Kapiszewski, Diana, MacLean, Lauren and Read, Benjamin. 2015b. “Reconceptualizing Field Research.” Unpublished manuscript.Google Scholar
King, Gary, Keohane, Robert, and Verba (KKV), Sidney. 1994. Designing Social Inquiry . Princeton, NJ: Princeton University Press.Google Scholar
Kurtz, Marcus. 2009. “The Social Foundations of Institutional Order: Reconsidering War and the ‘Resource Curse’ in Third World State Building.” Politics & Society 37(4): 479–520.10.1177/0032329209349223CrossRefGoogle Scholar
Laitin, David. 2013. “Fisheries Management.” Political Analysis 21: 42–27.10.1093/pan/mps033CrossRefGoogle Scholar
Lieberman, Evan. 2016. “Can the Biomedical Research Cycle be a Model for Political Science.” Perspectives on Politics 14(4): 1055–66.CrossRefGoogle Scholar
Loredo, T. J. 1990. “From Laplace to Supernova SN1987A: Bayesian Inference in Astrophysics.” In Maximum Entropy and Bayesian Methods, ed. Fougere, P. F.. The Netherlands: Kluwer Academic Publishers.Google Scholar
MacKay, David. 2003. Information Theory, Inference, and Linear Algorithms. New York: Cambridge University Press.Google Scholar
Mahoney, James. 2015. “Process Tracing and Historical Explanation.” Security Studies 24: 200–18.CrossRefGoogle Scholar
McKeown, Timothy. 1999. “Case Studies and the Statistical Worldview.” International Organization 53(1): 161–90.10.1162/002081899550841CrossRefGoogle Scholar
Monogan, James. 2015. “Research Preregistration in Political Science.” PS: Political Science and Politics 48(3): 425–29.Google Scholar
Ragin, Charles. 1997. “Turning the Tables: How Case-Oriented Research Challenges Variable-Oriented Research.” Comparative Social Research 16: 27–42.Google Scholar
Sivia, D. S., 2006, “Data Analysis—A Dialogue with the Data.” In Metrology VII: Advanced Mathematical and Computational Tools, ed. Ciarlini, P., Filipe, E., Forbes, A. B., Pavese, F., Perruchet, C., and Siebert, B.: Hackensack, NJ: World Scientific Publishing Co.Google Scholar
Van Evera, Stephen. 1997. Guide to Methods for Students of Political Science. Ithaca, NY: Cornell University Press.Google Scholar
Western, Bruce. 2001. “Bayesian Thinking about Macrosociology.” American Journal of Sociology 107(2): 353–78.10.1086/323639CrossRefGoogle Scholar
Yom, Sean. 2015. “From Methodology to Practice: Inductive Iteration in Comparative Research.” Comparative Political Studies 48(5): 616–44.10.1177/0010414014554685CrossRefGoogle Scholar
Yom, Sean. 2018. “Analytic Transparency, Radical Honesty, and Strategic Incentives.” PS: Political Science & Politics 51(2): 416–21.Google Scholar
Supplementary material: PDF

Fairfield and Charman supplementary material

Appendices A-C

Download Fairfield and Charman supplementary material(PDF)
PDF 4.6 MB