Is generalization decay a fundamental law of psychology?

Abstract

Generalizations strengthen in traditional sciences, but in psychology (and social and behavioral sciences, more generally) they decay. This is usually viewed as a problem requiring solution. It could be viewed instead as a law-like phenomenon. Generalization decay cannot be squelched because human behavior is metastable and all behavioral data collected thus far have resulted from a thin sliver of human time.

Generalizations decay.

Lee J. Cronbach (1975, p. 122)

In traditional scientific disciplines, to use Scriven's (1956) terminology, generalizations generally strengthen and can even trigger productive theoretical upheavals. The partial overthrow of Newton's mechanics by Einstein's special theory of relativity would not have happened had Einstein not generalized the Galilean principle of relativity in mechanics (also apparent in Newton's theory) to electrodynamics. Generalization strengthens understanding, sometimes at great cost to status-quo theories. In contrast, in psychology – and more broadly, the social and behavioral sciences – generalizations decay.

The “problem” of generalization decay has long been the subject of scholarly attention, yet no satisfactory solution has been found (Cronbach, 1975; Scriven, 1956). A widely discussed reason for such decay is that behavioral phenomena are interactively determined but psychological theories invariably underspecify the full range of interactions and often misspecify the nature of effects (Campbell, 1957; Yarkoni, 2022). An optimistic view is that psychology can overcome generalization decay by adopting an interactionist approach to theory generation and testing (e.g., Cronbach, 1957; Eysenck, 1997). Calls for methodological reform such as Almaatouq et al.'s proposed “integrative experiment design” also fall into the optimist's camp. However, none of the optimists’ proposals confront the fact that for most topics, psychology does not offer a theoretical basis for knowing how high the order of interactions must be for generalization decay to be squelched.

Take Almaatouq et al.'s example where the design space (i.e., the space defined by all of the measured independent variables and putative moderators) has upward of 50 factors. Even if each factor was binary, the design space would have over a quadrillion cells. Almaatouq et al. cryptically refer to statistical methods that could start with a highly circumscribed sample of cells in this overwhelming space, but these statements do little to inspire confidence in the overall project.

Permitting less ambition, assume a design space of a mere dozen variables split equally between binary and ternary factors. This space has 46,656 cells. Imagine that the researchers learn that, of the 5,667 possible interaction effects, several hundred are significant including over a dozen with n > 7. Would anyone have reason for confidence that such higher-order interactions would replicate if such a costly experiment could ever be repeated? Even if they were all replicable (an amazingly improbable occurrence), would they productively advance fundamental theory in psychology? After all, Newell's (1973) concern (the entry point for Almaatouq et al.) was not mainly about the lack of generalizability in psychology but, rather, about the slim prospect of theoretical unification even among fine examples of work by the best and brightest minds of his time.

Generalization decay cannot be eliminated through design mandates or interactionist projects because human behavior is metastable over time (Gergen, 1973). Psychologists often make claims about human behavior and cognition as if it applied to all humans across time, but this is unknowable since virtually all research participant data have been collected in a sliver of human history, and even historical records only go back thousands, not hundreds of thousands, of years, as would be required. However, even if psychologists fully exploited historical records of “dead minds,” as Atari and Henrich (2023) call for, or even if they miraculously recovered the full record of humanity's past, we have no trace of humanity's actual future. We do not know what proportion of human existence lies ahead or what metastable states it will occupy. Our theories do not project us clearly into the future as in physics, which provides a basis for estimation of physical transformations of the universe over unimaginable timescales (Dyson, 1979; Krauss & Starkman, 2000). They do not even project us as well into the past. We have no equivalent of the cosmic microwave background.

Psychologists cannot study the moderating role of social factors that do not yet exist. When life expectancy is universally less than 100 years, psychologists cannot formulate a generalizable theory of lifespan development that applies to humans who might live in an epoch following actuarial escape velocity – the point at which remaining life expectancy increases with time due to mortality rates that plummet due to disruptive scientific and technological breakthroughs (de Grey, 2004). We do not know what human experience will be like in the future. Imagine that in 50 years, the interpretation of quantum mechanics (based on evidence or insights that currently do not exist) indisputably favors the Everettian many worlds hypothesis and it becomes common knowledge that each of us exists in possibly infinite branches of decohered worlds – duh! – what then will social psychologists have to say about the self concept?

Psychology's uncertainty about humanity's past and future may be inevitable, but its comfort with a focus on the moving present reveals a parochial disposition that the traditional sciences outgrew long ago. If young Einstein did not stretch his mind to imagine whether light waves would appear to him to be at rest if he were able to run alongside at the speed of light, and if he did not have Maxwell's equations to show that the wave-at-rest counterfactual could not resolve the equations, he may not have discovered one of the most important theories in the history of science. Psychology banished introspection as a reliable method long ago; and it does not have the equivalent of Maxwell's equations, but where are its creative Machian Gedankenexperiments that may lead us out of musty local minima? The absence of thought experiments that revolutionize theoretical understanding in psychology is itself a mystery that deserves scholarly attention.

Returning to Scriven, “science has not advanced by solving all problems but often by abandoning them…” (1956, p. 339). If psychology pines for generalizability, resisting the apparent law of generalization decay, psychologists will need to seek new problems and ways of understanding. A productive path forward may be to seek greater consilience with traditional sciences (Wilson, 1998). Biocosmology offers a promising recent example (Cortês, Kauffman, Liddle, & Smolin, 2022). Alternatively, psychology could accept historicism as a metatheoretical foundation (Gergen, 1973), and its future might split along such lines.