Problem-Solving Heuristics

In brief

Many problems have no procedure guaranteed to find the answer in reasonable time. Chess has simple rules, but the number of possible games defeats exhaustive search. People, and later computer programs, respond with heuristics: rules of thumb that trade a guarantee of correctness for a workable path through an unmanageable space of possibilities. George Polya catalogued heuristics for mathematics in How to Solve It (1945); Allen Newell and Herbert Simon formalized heuristic search as the mechanism of thought itself in Human Problem Solving (1972). But treating a new situation as an instance of a familiar pattern, the trait that makes a heuristic useful, has a cost. Karl Duncker's candle problem showed that fixating on an object's normal use can hide a solution in plain sight. Abraham Luchins's water-jar experiments showed that a procedure that worked well can keep firing after it stops being the best option. A separate debate asks whether solutions arrive gradually or all at once, as a sudden restructuring of the problem.

The full treatment

The problem it answers

An algorithm is a procedure guaranteed to reach a correct answer given enough time: long division, a sorting routine, brute-force search of a chessboard. For small, well-defined problems this works. For most problems people face, it does not, because the space of possible moves grows combinatorially. Herbert Simon's bounded rationality, laid out in "A Behavioral Model of Rational Choice" (1955), starts from this fact: real agents cannot compute the optimal solution to most nontrivial problems, so they satisfice, settling for a solution good enough, found by a method that need not check everything. Heuristic problem solving is the machinery of satisficing.

How heuristic search works

Newell and Simon modeled problem solving as movement through a "problem space": a start state, a goal state, and legal operators that transform one into the other. An algorithm tries every path; a heuristic narrows the search using information about which paths look promising. Their means-ends analysis compares the current state to the goal, isolates the largest difference, and applies an operator known to reduce differences of that kind, repeating until the gap closes, a method they implemented in the General Problem Solver (1957). Polya's heuristics for mathematics work the same way at human scale: draw a diagram, work backward from the goal, or find a related problem already solved.

The key demonstration: Duncker's candle problem

Karl Duncker (1903 to 1940), a Gestalt psychologist trained under Wolfgang Kohler, gave subjects a candle, a box of tacks, and matches, and asked them to attach the candle to a wall so it would not drip wax on the table below. The solution is to empty the box, tack it to the wall, and stand the candle inside it. Most subjects struggled, because the box arrived already performing a function, holding tacks, and that role was hard to see past. When Duncker instead handed subjects the tacks loose beside an empty box, far more solved the problem, since the box was not already cast as a container. He called this fixation on an object's customary use functional fixedness. The mechanism that speeds recognition of an object's normal role is the same one that blocks recognition of an unfamiliar one.

Mental set: the Luchins water-jar experiments

Abraham Luchins (1914 to 2005) ran a parallel demonstration with arithmetic. Subjects measured exact quantities of water using three jars of stated capacities. A run of early problems could all be solved with one formula: fill the largest jar, then subtract the smaller ones a set number of times. Luchins then gave a problem solvable by that same formula and also, more simply, by adding or subtracting just two jars. Subjects drilled on the formula overwhelmingly missed the shortcut; some failed outright on a later problem where the old formula no longer worked. A control group given only the later problems, without drilling, found the simple solution readily. Luchins named the rigidifying effect Einstellung, German for "set." Warning subjects to watch for a simpler method reduced the effect but did not remove it for everyone.

Insight versus incremental problem solving

A related question is whether solutions arrive by degrees or all at once. Kohler's studies of chimpanzees on Tenerife, published as The Mentality of Apes (1925), described an ape named Sultan who, after apparently fruitless activity, suddenly joined two sticks to reach a banana outside his cage, a result Kohler read as a restructuring of the whole problem rather than the trial-and-error learning Edward Thorndike had documented in cats. Janet Metcalfe and David Wiebe (1987) tested the distinction by asking subjects to rate, while working, how close they felt to a solution. On incremental problems, this "warmth" rating climbed steadily; on classic insight problems, it stayed flat and then jumped just before the answer, with no felt warning. Work led by Mark Jung-Beeman and John Kounios in the early 2000s found a matching neural signature, a burst of gamma-band activity in the right anterior temporal lobe moments before a reported sudden solution.

The word "heuristic" also names something else: Amos Tversky and Daniel Kahneman's "heuristics and biases" program describes shortcuts for estimating a probability, not for finding a solution, and produces systematic errors of judgment rather than functional fixedness. Both traditions share a starting intuition, that limited minds must economize, but answer different questions.

Lineage

Functional fixedness and the Gestalt account of insight descend from the school founded by Max Wertheimer, Wolfgang Kohler, and Kurt Koffka in early twentieth-century Germany, which studied thinking as organized wholes rather than chains of association; Duncker worked within that circle, and Luchins, in the United States, drew directly on its idea of set. This line stood against the behaviorist account of Edward Thorndike and later B. F. Skinner, on which problem solving is trial and error shaped by reinforcement, with no room for sudden restructuring. Newell and Simon's problem-space theory, arriving decades later from early artificial intelligence, recast the question in computational terms, tested by building programs, including the Logic Theorist (1956), that used heuristic search directly.

The strongest case for it

Heuristic search is not a compromise forced on limited minds; it is the only method that scales. Simon's bounded rationality treats the absence of exhaustive search as a fact about real problems, not a failure of the solver. The General Problem Solver showed that a small set of general heuristics, applied to an explicit problem space, could solve puzzles across domains without domain-specific rules, a result that shaped cognitive science and artificial intelligence for decades. On the failure side, functional fixedness and mental set are durable, repeatedly replicated findings with real use: knowing a habitual category or procedure can blind a solver to a better path underwrites brainstorming rules that defer judgment and design methods that force alternate uses of familiar objects.

The strongest case against it

The sharpest challenge targets the Gestalt half of the picture. Robert Weisberg, with Joseph Alba, published studies beginning in 1981 arguing that classic "insight" and "fixation" problems, including variants of Duncker's materials and Norman Maier's two-string problem, are not well explained by a special restructuring process or by fixation in the strong sense Duncker proposed. Giving subjects hints or relevant prior experience improved performance in ways a pure fixation account would not predict, since prior experience is exactly what that account says should hurt; they argued ordinary memory retrieval does most of the work credited to insight. Thomas Ormerod, James MacGregor, and Edward Chronicle offered a related alternative, a progress-monitoring theory on which solvers track how close they are getting and switch strategy when progress stalls, needing no unobservable restructuring. On the heuristic-search side, critics within artificial intelligence noted that Newell and Simon's general, weak methods proved less powerful than approaches loaded with domain knowledge, a lesson the field learned through the 1970s and 1980s as expert systems outperformed general problem-space search.

Where it stands now

Functional fixedness and mental set remain standard findings in cognitive psychology, taught with the caveat that later work finds the effects vary with training, expertise, and how a task is described. The insight-versus-incremental debate has no single winner; most researchers treat it as a matter of degree, with the warmth-rating and neural evidence showing insight solutions at least feel and look different, even as the mechanism stays contested between Gestalt-style restructuring and Weisberg-style continuity with ordinary memory. Newell and Simon's problem-space framework remains foundational to cognitive architectures such as Soar and ACT-R, even as machine learning increasingly solves hard problems through statistical pattern-fitting rather than explicit heuristic search.

Test yourself

Think of a tool, template, or method you reach for automatically because it has worked before. Ask whether you are still choosing it, or whether it has become the shape your thinking takes before you have even looked at the problem in front of you. Then ask what you would see if you set it down.