Internal and External Validity

In brief

Donald Campbell and Julian Stanley gave psychology its working vocabulary for study quality in a 1963 monograph, Experimental and Quasi-Experimental Designs for Research. They split the question "is this a good study?" into two. Internal validity asks whether the study, on its own terms, has actually shown that the treatment caused the outcome, rather than something else sneaking in. External validity asks a different question: to whom, to what settings, and to what times does the result apply? A study can be strong on one and weak on the other. The uncomfortable fact that runs through research design is that the moves which buy internal validity, tight control, random assignment, a stripped-down laboratory, tend to cost external validity, because a controlled setting is not the messy world the finding is meant to describe.

The full treatment

The problem it answers

Suppose an experiment reports that a new teaching method raised test scores. Two suspicions arise. The first is local: maybe the method did nothing, and the improvement came from the students being older by the end of term, or from the keenest students being assigned to the new method. If those rival explanations are open, the causal claim collapses even before we ask about anyone else. The second is global: grant that the method worked here, in this school, this term. Does it work in a different school, with weaker students, a different teacher, next year? Campbell's insight was that these worries demand separate defenses. Internal validity rules out alternative causes within the study; external validity concerns the reach of the conclusion beyond it.

Threats to internal validity

Campbell and Stanley catalogued the ways a within-study causal claim can fail. History: some event other than the treatment intervenes during the study. Maturation: participants change on their own (they grow tired, older, more practiced). Testing: taking a pre-test alters performance on the post-test. Instrumentation: the measuring tool drifts. Regression to the mean: participants selected for extreme scores drift back toward average regardless of treatment. Selection: the groups differed before the treatment began. Attrition: the participants who drop out are not random. The randomized controlled experiment is powerful precisely because random assignment neutralizes most of these at once: if people are assigned to conditions by chance, then history, maturation, and pre-existing differences are, on average, balanced across groups, so a difference at the end is attributable to the one thing that differed by design, the treatment.

Threats to external validity, and ecological validity

External validity fails when something about the study's people, setting, or procedure limits how far the finding travels. The sample may be unrepresentative. The laboratory task may be unlike anything the participant does in life. The mere fact of being observed may change behavior. A closely related and often confused idea is ecological validity: whether the study's materials, tasks, and setting resemble the real-world situation the researcher wants to speak about. A memory experiment using random word lists has weak ecological validity as a model of how you remember a conversation. Ecological validity is best understood as one component of external validity, concerning the realism of the setting rather than the breadth of the population.

The lab-versus-field tension

The two validities trade against each other because the instrument for one is a threat to the other. To secure internal validity you strip the setting down: hold everything constant, remove distractions, standardize the task, so that only the manipulated variable moves. But a stripped-down setting is, by that very fact, unlike the cluttered world. Stanley Milgram's obedience studies (1963) are the standard illustration. In the laboratory an experimenter in a coat could induce a majority of ordinary people to deliver what they believed were dangerous shocks, a result of enormous internal clarity: the situation, not the person, drove the behavior. But critics asked immediately whether the finding was ecologically valid. Did participants really believe the shocks were real? Would the same obedience appear without the trappings of a prestigious university's laboratory? The debate over Milgram's realism is the internal-versus-external tension made concrete: a design vivid enough to isolate the cause was artificial enough to raise doubts about its reach.

Lineage

The word "validity" in psychology first belonged to measurement, not experiments. Lee Cronbach and Paul Meehl, in a 1955 paper, laid out a layered account of construct validity: whether a test measures the abstract thing it claims to. Campbell and Stanley narrowed and repurposed the term in 1963 for experimental design, coining the internal-external split that became standard. Campbell later refined it with Thomas Cook in a 1979 book that added statistical conclusion validity and construct validity of the treatment, giving four kinds. Running beneath all of this is the older logic of causal inference from John Stuart Mill (1806 to 1873), whose methods of agreement and difference are the ancestor of the controlled comparison. The modern sampling critique that revived external validity as a live worry, the WEIRD-sample argument, is a direct descendant: it takes Campbell's abstract "generalization to populations" and names the specific population psychology had quietly overgeneralized from.

The strongest case for it

The distinction is one of the most useful pieces of conceptual hygiene in the social sciences, because it stops two different arguments from being mistaken for each other. Before it, a critic could say "your study is bad" and leave the author guessing whether the charge was "your result is an artifact" or "your result will not travel." Campbell forced the critic to say which. This matters because the two are fixed by different means: internal validity mainly through design (randomization, control groups, blinding), external validity mainly through the choice of samples, settings, and replication. The framework also imposes an honest ordering. A finding with no internal validity has nothing to generalize; the question of reach does not even arise until the local causal claim is secure. That ordering is why the randomized experiment, despite its artificiality, earned its central place: it is the cleanest known way to license a causal claim at all, and a shaky cause generalized widely is worse than a solid cause generalized narrowly.

The strongest case against it

The sharpest challenge is that the tradeoff has been overstated, and that external validity is frequently the wrong thing to demand. Douglas Mook, in a 1983 paper pointedly titled "In Defense of External Invalidity," argued that many experiments never intended to generalize to the real world in the first place. Their aim is to test whether a theory can produce an effect at all, or to show that some behavior is possible under specified conditions. Harry Harlow's monkeys clinging to a cloth surrogate mother do not tell us how monkeys behave in the wild, and were never meant to; they refute the claim that attachment is only about feeding. For such work, Mook argued, artificiality is a feature, and complaining that the lab is unlike life misses the point. The "tension" is real only for research whose goal is prediction in a target setting, not for research whose goal is understanding a mechanism.

A second line of attack says the framework flattered psychology's actual practice. Joseph Henrich, Steven Heine, and Ara Norenzayan, in 2010, showed that a large share of published findings rested on WEIRD samples (Western, Educated, Industrialized, Rich, Democratic), often undergraduates, and that this narrow base differs from the rest of humanity on exactly the dimensions being studied, from visual perception to moral reasoning to notions of the self. Researchers had long paid lip service to external validity while treating a convenience sample as if it were humanity. The distinction did not fail, but its external-validity half had been ignored in practice for decades. The replication crisis sharpened the worry: many textbook effects, once thought internally solid, did not reproduce even in similar samples, which suggests the two problems compound rather than trade cleanly.

There is also a conceptual objection. Lee Cronbach, in his later work, argued that Campbell's system leaned too hard on internal validity and that "external validity" bundles together several different questions (units, treatments, outcomes, settings) that behave differently and should not travel under one label. Generalization, on this view, is not a property a study has or lacks but a matter of substantive theory about which features of a situation matter, something no design feature can settle on its own.

Where it stands now

The vocabulary is universal: every methods textbook and every grant reviewer uses it. What has shifted is the balance of anxiety. For a long time the prestige of the randomized experiment made internal validity the senior partner, and external validity a courtesy paid in the discussion section. The WEIRD critique and the replication crisis together moved external validity back to the center, alongside a renewed appetite for field experiments, multi-site replications, and pre-registration. The modern consensus is that neither validity is prior in principle: a finding earns its keep only when it is both a real effect and an effect that means what the author says it means for the world beyond the lab.

Test yourself

Think of a study you have cited or believed, perhaps one about willpower, persuasion, or how people behave under authority. Ask the two questions separately. First: within that study, was the causal claim clean, or could age, self-selection, or some coincident event explain the result just as well? Second: who were the participants, where were they, and is there any reason the finding should hold for people unlike them in a setting unlike that one? If you have been treating a tidy result from a room full of undergraduates as a fact about human beings, you have felt the gap the distinction was built to name.