32 - The Conflict Entailed in Using a Post Hoc Theory to Organize a Research Report

Summary

Kerr (1998) discusses in a very thorough and thoughtful manner the ethical dilemma that I faced many years earlier in seeking to bring a research report (Wallsten, 1976) to publication. Kerr called it HARKing, hypothesizing after the results are known. HARKing can take many forms, some of which are more egregious than others, but the essence of it is that a hypothesis, explanation, or theory developed to explain a data set or a pattern of results is introduced as motivating the experiment in the first place. Kerr (1998) documents the fact that in recent years at least some authors and research mentors have suggested, if not actually encouraged, putting ex post facto hypotheses at the front of the paper for a host of possible reasons, including that the paper becomes more readable.

We are (or at least, were) trained from graduate days on – actually for many of us, from undergraduate days on – not to present empirical research as support for a theory that grew from that very research. In broad outline form, the scientific method entails deducing predictions from theory and then testing them in an experiment. The theory accrues support if its predictions are sustained. Other possible outcomes are that the data provide partial support for the theory and the basis for revision or that they disconfirm it altogether, ideally simultaneously suggesting a new theory or perspective. The revisions or new theory are tested in a new experiment and the cycle continues. This is a very gross characterization of the ....-deduction-inference-deduction-inference-.... chain that links theory development and testing, but it captures the very important point that theories gain support on the basis of the outcomes of experiments designed to test them. It is misleading and a gross violation of scientific principles to develop an explanation for a set of data and then to present that data as supporting the explanation.