Extrema & Lagrange Multipliers
Finding a maximum is two separate problems: locating the candidates, and deciding which is which. The first is a first-derivative condition, the second a second-derivative one, and both change shape when the domain carries a constraint. The multiplier method turns a constrained problem into an unconstrained one, and the implicit function theorem is what makes it legal.
- Prove that an interior extremum forces $\nabla f=0$.
- Classify a critical point from the definiteness of the Hessian, and know when the test is silent.
- Derive the Lagrange condition $\nabla f=\sum\lambda_i\nabla g_i$ from the tangent space picture.
- Run a full constrained optimisation on a compact set, boundary included.
Optimisation under constraints is most of applied mathematics. Solving the constraint and substituting works only when you can solve it; the multiplier method never needs to, which is why it survives into settings where the constraint is a differential equation.
Support vector machines, portfolio optimisation, maximum entropy distributions, and the equilibrium conditions of thermodynamics are all multiplier computations. The multiplier itself usually has a meaning: a price, a temperature, a marginal cost.
Lagrange introduced the method in his 1788 Mécanique analytique as a way to handle mechanical constraints. The inequality version — the Karush–Kuhn–Tucker conditions of the 1940s–50s — is the foundation of modern nonlinear programming and is where the subject went next.
Unconstrained extrema
If $f$ is differentiable at an interior point $a$ of its domain and $a$ is a local maximum or minimum, then $Df(a)=0$.
For each direction $u$, the function $t\mapsto f(a+tu)$ has an interior extremum at $t=0$, so its one-variable derivative $Df(a)u$ vanishes. True for every $u$, so $Df(a)=0$. Such an $a$ is called a critical point.
A symmetric matrix $H$ is positive definite if $h^{T}Hh>0$ for all $h\ne0$, negative definite if the inequality reverses, and indefinite if $h^{T}Hh$ takes both signs. By the spectral theorem from MAT247H1, the sign pattern of the eigenvalues decides which.
Let $f$ be $C^2$ with $\nabla f(a)=0$.
The proof is Taylor. With $\nabla f(a)=0$, $f(a+h)-f(a)=\tfrac12h^{T}Hf(a)h+o(\|h\|^2)$. If $H$ is positive definite its smallest eigenvalue $\mu>0$ gives $h^{T}Hh\ge\mu\|h\|^2$, and for small $h$ that beats the remainder. Indefinite: pick eigenvectors of opposite sign and the two restrictions of $f$ go opposite ways.
All of $x^4+y^4$, $-x^4-y^4$ and $x^4-y^4$ have zero gradient and zero Hessian at the origin, and the origin is a minimum, a maximum and a saddle respectively. When the Hessian is singular, the second-order test cannot possibly decide, and you must look at $f$ itself.
Constrained extrema
Let $f$ and $g_1,\dots,g_k$ be $C^1$ near $a$, let $M=\{x:g_1(x)=\dots=g_k(x)=0\}$, and suppose the gradients $\nabla g_1(a),\dots,\nabla g_k(a)$ are linearly independent. If $f$ restricted to $M$ has a local extremum at $a$, then there are scalars $\lambda_1,\dots,\lambda_k$ with
The geometry in one line: at a constrained extremum the level set of $f$ is tangent to $M$, so their normals are parallel.
Maximise $f(x,y,z)=xyz$ on $x^2+y^2+z^2=3$. The condition $\nabla f=\lambda\nabla g$ reads $yz=2\lambda x$, $xz=2\lambda y$, $xy=2\lambda z$. Multiplying the three by $x,y,z$ and comparing gives $x^2=y^2=z^2=1$, so the candidates are the eight points $(\pm1,\pm1,\pm1)$. The maximum value is $1$, attained where an even number of coordinates are negative.
Maximise $x_1\cdots x_n$ subject to $x_1+\dots+x_n=s$ with all $x_i>0$. The multiplier equations force all $x_i$ equal, so the maximum of the product is $(s/n)^n$. Rearranged, that is exactly $\sqrt[n]{x_1\cdots x_n}\le(x_1+\dots+x_n)/n$.
Global extrema on a compact set
A continuous $f$ on a compact $K$ attains its maximum, by Unit 3. To find it:
Then compare finitely many values. No second-order test is needed: the extreme value theorem already promised the maximum exists, so the largest candidate is it.
Maximise $f(x,y)=x^2-y^2+2x$ on $[-1,1]^2$. Interior: $\nabla f=(2x+2,-2y)$ vanishes at $(-1,0)$, which is on the boundary, so there is no interior critical point. Edges: on $x=1$, $f=3-y^2$ with maximum $3$ at $y=0$; on $x=-1$, $f=-1-y^2$ with maximum $-1$; on $y=\pm1$, $f=x^2+2x-1$ with maximum $2$ at $x=1$. Largest value $3$ at $(1,0)$.
• Applying the first-order condition at a boundary point. It holds at
interior points only; the maximum of $x$ on $[0,1]$ has nonzero
derivative.
• Concluding "saddle" from a singular Hessian. Singular means the test is
silent, not that the point is a saddle.
• Forgetting to check that the constraint gradients are independent.
Where they are dependent the multiplier equation can fail outright.
• Solving the multiplier equations and forgetting the constraint itself.
It is one of the equations of the system.
• Assuming a critical point exists. On a non-compact set $f$ may have no
extremum at all, and $e^x$ on $\mathbb{R}$ has neither.
Practice
Original problems written for this guide in the style of Spivak and Munkres exercises. Write a full solution before opening the answer.
Find and classify the critical points of $f(x,y)=x^3-3xy+y^3$.
Solution
Find the point of the plane $x+2y+3z=6$ nearest the origin.
Solution
Maximise $x+y+z$ on the intersection of $x^2+y^2=1$ and $z=x+y$.
Solution
Classify the origin for $f(x,y)=x^2y^2$.
Solution
Find the largest-area axis-aligned rectangle inscribed in $x^2/a^2+y^2/b^2=1$.
Solution
Minimise $x$ on $\{y^2=x^3\}$ and explain why the multiplier equation breaks.
Solution
If the constraint is $g(x)=c$, show $\lambda=dM/dc$ where $M(c)$ is the constrained maximum value.
Solution
Show the maximum of $x^{T}Ax$ on the unit sphere, for symmetric $A$, is the largest eigenvalue.
Solution
Expanded topic checklist
Course coverage. This unit is part of the complete MAT257Y1 topic map. It should cover the following items before you move on:
- First-order condition. Interior extrema are critical points; the boundary is excluded for a reason.
- Second-order test. Definiteness of the Hessian, the $2\times2$ discriminant, and the degenerate case.
- Lagrange multipliers. Statement, the tangent space proof, and the independence hypothesis.
- Global optimisation. Interior, boundary, corners, then compare.
Spivak treats maxima briefly and Lagrange multipliers as an exercise; Munkres gives the tangent space argument in full. For the second-order constrained test, look up the bordered Hessian in an optimisation text. Confirm what the syllabus expects.
Implicit function theorem gives the tangent space, which gives Lagrange. Taylor gives the second-order test. Extreme value theorem is what guarantees the maximum you are hunting for exists at all.
Deep practice extensions
Use these after the main practice set. They are designed to force definition-level reasoning, theorem selection, and mixed application rather than routine calculation only.
Why does the first-order condition need an interior point?
Guide
State the Lagrange condition geometrically.
Guide
When does the second-order test say nothing?
Guide
Why check compactness before hunting for a maximum?