Multivariable change

Intuitive problem

A room is warmed by two heaters with separate dials. Turn the first dial up a notch and the temperature at your desk rises by some amount. How large is that amount, and does it depend on where the second dial happens to sit? For one input, the derivative already answers questions of this shape. With two inputs the question splits, and it gains a twist: there is one sensitivity per dial, and each sensitivity can itself shift when the other dial moves. A pair of numbers, plus a rule for combining them, has to do the work one number used to do.

Definition

Let f(x, y) be a function of two inputs x and y. The partial derivative of f with respect to x, written f_x, is the ordinary single-variable derivative of f taken while y is held fixed at its current value; f_y is the same with the roles swapped. Nothing new is being differentiated: freeze one input and the single-variable machinery applies unchanged to the other.

The gradient of f at a point, written grad f, is the vector whose components are the partials: grad f = (f_x, f_y). It lives in the input plane, the space of dial settings, and collects the separate sensitivities into one directed object.

One consequence deserves emphasis at the start. Each partial is itself a function of both inputs. The number f_x is read off at a specific operating point (x, y), and its value generally changes when y moves, even though y sat frozen during the differentiation. So "how much does heater one matter" has no fixed answer: it is read at the current settings of both dials, and turning heater two can change it.

Derivation

The honest starting point is the linear approximation, because with several inputs that approximation is what differentiability means. Near a point (a, b), where a and b are the current input values, a differentiable f is well approximated by a flat sheet:

f(a + h, b + k) is approximately f(a, b) + f_x(a, b) h + f_y(a, b) k

where h and k are small steps in x and y, and the approximation error shrinks faster than the step size as the step shrinks. That error condition is the definition: f is differentiable at (a, b) precisely when such a linear approximation exists there. The partials supply the coefficients, but the linear sheet is the primary object, a point the counterexample below makes sharp.

Now ask how f changes along a chosen direction. Let u = (u1, u2) be a unit vector, a direction of length one, and take a step of small length s along it, so h = s u1 and k = s u2. Substituting into the linear approximation, the change in f is approximately s times the quantity f_x u1 + f_y u2. The rate of change per unit distance in direction u is therefore

f_x u1 + f_y u2, which is grad f dot u,

the dot product of the gradient with the chosen direction. Only after this substitution does the quantity earn its name: the directional derivative of f in direction u.

The steepest direction now costs nothing extra. The dot product of two vectors is the product of their lengths times the cosine of the angle between them, and u has length one, so the directional derivative equals the length of grad f times cosine of theta, where theta is the angle between the gradient and the chosen direction. Cosine peaks at one when theta is zero. So the direction of fastest increase is the gradient's own, the rate there is the gradient's length, and the rate falls off by the cosine as the direction swings away, reaching zero at right angles to the gradient and turning maximally negative directly against it.

Visual representation

Picture a contour map of f: curves in the input plane along which f holds a constant value, the way an elevation map draws curves of constant height. Along a contour f does not change, so the directional derivative there is zero, and by the cosine formula that happens at right angles to the gradient. The gradient at each point therefore sticks out perpendicular to the contour through that point, crossing the level curves toward higher values, with tightly packed contours signalling a long gradient.

The map picture also corrects a persistent misreading. The hill analogy behind "steepest ascent" invites drawing the gradient on the hillside itself, an arrow lying on the surface of the graph. It does not live there. The gradient is drawn on the map, the flat input plane, pointing across the contours; it has one component per input and none for the output. The hill analogy breaks in a second place: with three or more inputs there is no hill to picture, while the map idea survives as level surfaces and the algebra of partials, gradient, and dot product carries over without change.

Worked example

Take f(x, y) = x^2 y + y^3. Differentiating with y held fixed gives f_x = 2xy. Differentiating with x held fixed gives f_y = x^2 + 3y^2.

At the operating point (1, 2): f_x = 4 and f_y = 1 + 12 = 13, so grad f = (4, 13). In the unit direction u = (3/5, 4/5), the directional derivative is 4 times 3/5 plus 13 times 4/5, which is 64/5 = 12.8. The steepest rate available at this point is the gradient's length, sqrt(16 + 169) = sqrt(185), roughly 13.6, achieved along (4, 13) scaled to unit length; the chosen u lies near that direction and captures much of it.

Now repeat at a second operating point, (1, 1): f_x = 2 and f_y = 1 + 3 = 4, so grad f = (2, 4). Both sensitivities dropped when y moved from 2 to 1, by different factors. This is the interaction promised in the definition: the partials are functions, read anew at each operating point, and a control ranking made at one setting can flip at another.

Counterexample

Existence of both partials is weaker than differentiability, and one standard function exhibits the gap. Define f(x, y) = xy / (x^2 + y^2) away from the origin, with f(0, 0) = 0. Along the x axis, y is zero, so f is identically zero there and f_x(0, 0) = 0; the same argument along the y axis gives f_y(0, 0) = 0. Both partials exist at the origin. Yet at every point of the diagonal line y = x off the origin, f takes the value x^2 / (2 x^2) = 1/2, so along the diagonal the function approaches 1/2 while its value at the origin is 0. The function fails even to be continuous there, and no flat sheet can approximate it. The partials watch the two axis directions alone and can miss what happens between them; the linear approximation is the stronger notion, and the right one.

Limits and boundary conditions

The machinery assumes smoothness. The story holds where f is differentiable, and the counterexample shows that the bare existence of partials at a point does not secure this; partials continuous in a neighborhood of the point do. Everything here is also local. The gradient reports rates for small steps near the operating point, and following it uphill says nothing by itself about where a global peak lies or whether one exists. And each conclusion is tied to its operating point: the gradient changes from point to point, which is exactly why sensitivities must be re-evaluated when the inputs move.

Common mistakes

The first mistake is letting the frozen variable thaw: computing f_x while allowing y to drift with x. That yields a different quantity, a total sensitivity rather than a partial one, and in general a different number. The second is placing the gradient on the graph. It is a vector in the input space, perpendicular to the level curves, with no output component. The third is aiming at the largest partial: steepest ascent points along the full gradient, and a direction that maxes out one partial ignores what the other components contribute, which the dot product weighs correctly.

Build with it

Return to the two-heater room, now with numbers. The temperature at the desk, in degrees, is modelled by T(p, q) = 15 + 2p + q + 0.5 p q, where p and q are the settings of heaters one and two. The operating point is (p, q) = (4, 6).

Compute both partials T_p and T_q as functions of p and q, then evaluate them and the gradient at the operating point. State which heater matters more at that setting and defend the ranking from the two evaluated numbers; as part of the defense, say whether the ranking is a fixed fact about the room or a fact about this operating point, backed by a setting where it flips if one exists. Then predict the temperature change for a step of length 0.1 in the unit direction (0.6, 0.8) using the dot product, and verify by evaluating T directly at the stepped settings and subtracting. Repeat the verification with the step halved.

Success means three things: the gradient at (4, 6) is correct; the control ranking is defended from the evaluated partials, with its dependence on the operating point stated correctly; and the predicted change lands within one percent of the directly evaluated change, with the gap shrinking when the step is halved. If the gap fails to shrink, either the partials were read away from the operating point or the direction was off unit length; rule out both by hand.