Chain Rule & Higher Derivatives
Once the derivative is a linear map, the chain rule stops being a formula to memorise and becomes the only thing it could be: the derivative of a composition is the composition of the derivatives. Higher derivatives then arrive as multilinear maps, and the surprise of the unit is that mixed partials commute — but only under a hypothesis, and there is a function that shows why.
- Prove the chain rule from the $o(\|h\|)$ definition, not from coordinates.
- Translate $D(g\circ f)(a)=Dg(f(a))\,Df(a)$ into a matrix product and back.
- Prove Clairaut's theorem on equality of mixed partials, and state where it fails.
- Write the second-order Taylor expansion with the Hessian and control the remainder.
Almost nothing you differentiate is given directly; it is built by composing simpler maps. A chain rule that is a clean algebraic statement about linear maps is what lets you differentiate anything assembled from pieces, in any number of variables, without a new argument each time.
Backpropagation is the chain rule applied right-to-left through a composition of hundreds of maps; the cost saving over left-to-right is entirely a statement about matrix shapes. Change of variables in physics is the same identity read the other way.
Clairaut asserted the equality of mixed partials in 1740; a hypothesis-free version is false, and Schwarz gave the continuity condition that repairs it in 1873. Peano's counterexample from the 1890s is the one still used, and it is in this unit.
The chain rule
Let $f:A\to\mathbb{R}^m$ be differentiable at $a$ with $A$ open in $\mathbb{R}^n$, and let $g:B\to\mathbb{R}^p$ be differentiable at $b=f(a)$ with $f(A)\subseteq B$ open. Then $g\circ f$ is differentiable at $a$ and
In matrices, $J(g\circ f)(a)=Jg(b)\,Jf(a)$: a $p\times m$ times an $m\times n$, giving $p\times n$. The shapes only fit one way, which is a useful memory aid.
Note where the proof would break if we only knew directional derivatives: $k(h)$ does not travel along a line.
Let $u(r,\theta)=f(r\cos\theta,r\sin\theta)$. Then $Ju=Jf\cdot Jg$ with $g$ the polar map of Unit 4, giving
Every "chain rule in several variables" formula from a first calculus course is one row of one matrix product.
Higher derivatives
$f$ is $C^k$ on an open set if all partial derivatives up to order $k$ exist and are continuous there; $C^\infty$ (or smooth) if this holds for every $k$. Since $Df$ is itself a map into the space of matrices, $D^2f(a)$ is a linear map into linear maps — equivalently a bilinear map $\mathbb{R}^n\times\mathbb{R}^n\to\mathbb{R}^m$.
For scalar $f$, the Hessian at $a$ is the $n\times n$ matrix $Hf(a)$ with entries $\partial^2f/\partial x_i\partial x_j$. It represents the bilinear form $(h,k)\mapsto D^2f(a)(h,k)$.
If $\partial^2f/\partial x\,\partial y$ and $\partial^2f/\partial y\,\partial x$ exist near $a$ and are continuous at $a$, they agree at $a$. Hence the Hessian of a $C^2$ function is symmetric.
$f(x,y)=xy\dfrac{x^2-y^2}{x^2+y^2}$ with $f(0,0)=0$.
Both mixed partials exist at the origin and differ. Neither is continuous there, so Clairaut is not contradicted — it is illustrated.
Taylor's theorem in $\mathbb{R}^n$
If $f$ is $C^2$ on a ball around $a$, then for $h$ in that ball
and consequently $f(a+h)=f(a)+\nabla f(a)\cdot h+\tfrac12h^{T}Hf(a)h+o(\|h\|^2)$.
The proof is the one-variable Taylor theorem applied to $\varphi(t)=f(a+th)$ on $[0,1]$; the chain rule supplies $\varphi'(t)=\nabla f(a+th)\cdot h$ and $\varphi''(t)=h^{T}Hf(a+th)h$.
$f(x,y)=x^2+4xy+y^2$ has $\nabla f(0,0)=(0,0)$ and constant Hessian $\begin{smallmatrix}2&4\\4&2\end{smallmatrix}$. The expansion at the origin reproduces $f$ exactly with zero remainder, as it must: every derivative of order three and higher vanishes.
• Writing the chain rule in the wrong order. $Df(a)\,Dg(b)$ usually
does not even have compatible shapes.
• Applying Clairaut without continuity of the second partials. Peano's
function is the standing counterexample.
• Assuming $C^1$ implies $C^2$. Each level of smoothness is a genuinely
stronger hypothesis.
• Treating the Hessian as a linear map. It is a bilinear form; the matrix
looks the same but the change-of-basis rule does not.
• Expanding to second order and calling the remainder $o(\|h\|)$. It is
$o(\|h\|^2)$, and the sharper statement is the one you need.
Practice
Original problems written for this guide in the style of Spivak and Munkres exercises. Write a full solution before opening the answer.
With $f(t)=(t,t^2)$ and $g(x,y)=x^2y$, compute $(g\circ f)'(1)$ two ways.
Solution
If $f$ is invertible near $a$ with differentiable inverse, what is $D(f^{-1})(f(a))$?
Solution
For $u(x)=\varphi(\|x\|)$ on $\mathbb{R}^n$ minus the origin, compute $\nabla u$.
Solution
Verify Clairaut for $f(x,y)=x^3y^2+e^{xy}$.
Solution
Suppose $f$ is differentiable and homogeneous of degree $d$, meaning $f(tx)=t^df(x)$ for $t>0$. Show $x\cdot\nabla f(x)=df(x)$.
Solution
Find the Hessian of $f(x,y)=x^3-3xy^2$ at the origin and describe the critical point.
Solution
If $\gamma$ is a differentiable curve inside the level set $\{f=c\}$, what is $\nabla f(\gamma(t))\cdot\gamma'(t)$?
Solution
Show $\Delta u=u_{rr}+\tfrac1ru_r+\tfrac1{r^2}u_{\theta\theta}$.
Solution
Expanded topic checklist
Course coverage. This unit is part of the complete MAT257Y1 topic map. It should cover the following items before you move on:
- Chain rule. Statement as composition of linear maps, the $o(\|h\|)$ proof, matrix form.
- Higher derivatives. $C^k$, the second derivative as a bilinear map, the Hessian.
- Clairaut. Proof by second differences, and Peano's counterexample when continuity fails.
- Taylor. Second-order expansion with an explicit remainder and the $o(\|h\|^2)$ form.
Spivak proves the chain rule in about a page; Munkres separates the partial-derivative bookkeeping from the linear-map argument and is easier to follow the first time. Confirm the assigned text and problem numbers against the current syllabus.
Chain rule builds every later computation. Clairaut makes the Hessian symmetric, which is what lets Unit 8 diagonalise it. Taylor is the bridge between the derivative and the classification of critical points.
Deep practice extensions
Use these after the main practice set. They are designed to force definition-level reasoning, theorem selection, and mixed application rather than routine calculation only.
Explain by shapes why $Dg(b)\,Df(a)$ and not the reverse.
Guide
Which hypothesis fails in Peano's example?
Guide
To what order is the second-order Taylor remainder small?
Guide
Is a composition of $C^k$ maps $C^k$?