13. Power series and Taylor¶
Your calculator gives \(\sin 0.3\), \(e^{1.7}\) and \(\ln 5\) to ten digits, yet all a chip can really do is add and multiply. How? The answer is that these functions are polynomials of infinite degree: sums like \(x - x^3/6 + x^5/120 - \dots\) that you can stop whenever you have enough digits. In this chapter you learn when such an infinite polynomial (a power series) makes sense, how to find the right one for a given function, how to bound the error when you stop early, and how the series for \(e^x\), \(\sin x\) and \(\cos x\) turn out to be one series in disguise. At the end you meet a function so flat at one point that its series says nothing about it at all.
In this chapter
You will learn:
- what a power series is, and why each one converges on an interval centred at its centre (the radius of convergence)
- how to find the radius with the ratio and root tests, and what can happen at the two ends
- that a power series can be differentiated and integrated term by term inside its interval
- Taylor polynomials, and Taylor's theorem with the Lagrange remainder, with a full proof
- the series for \(e^x\), \(\sin x\), \(\cos x\), \(\ln(1+x)\) and \(\arctan x\), and how to compute \(\pi\) with them
- Euler's formula \(e^{i\theta} = \cos\theta + i\sin\theta\), straight from the series
- why "smooth" does not mean "equal to its Taylor series": the function \(e^{-1/x^2}\)
You need: Chapter 12 (convergence of series, the comparison, ratio, root and alternating series tests), Chapter 6 (Rolle's theorem) and Chapter 5 (derivatives of \(e^x\), \(\sin\), \(\ln\), \(\arctan\)). For 13.5, complex numbers as in Algebra, Chapter 9.
Levels: 13.1 and 13.3 to 13.4 are intermediate (first-year university); 13.2 and 13.5 are intermediate to advanced; 13.6 is advanced.
13.1 Power series and the radius of convergence¶
Start with a series you already know. For \(|x| < 1\) the geometric series gives
Read it the other way round: the function \(1/(1-x)\), which is not a polynomial, is equal to an endless polynomial, as long as \(|x| < 1\). Outside that interval the left side makes no sense (at \(x = 2\) it is \(1 + 2 + 4 + \dots\)), although the right side is a perfectly good number, \(-1\). So the equality is true on an interval and false off it. That is the whole story of this section in miniature.
Definition. A power series centred at \(a\) is a series
where the coefficients \(c_0, c_1, c_2, \dots\) are fixed numbers and \(x\) is a variable. For each value of \(x\) it is an ordinary series of numbers, which may converge or diverge. Where it converges, its sum is a function of \(x\).
At \(x = a\) every term after the first is zero, so a power series always converges at its centre. The question is how far from \(a\) it keeps converging. Four examples, all centred at 0, show the range of answers.
| Series | Converges for | Why |
|---|---|---|
| \(\sum x^n\) | \(-1 < x < 1\) | geometric, ratio \(x\) |
| \(\sum x^n / n\) (\(n \ge 1\)) | \(-1 \le x < 1\) | at \(x = 1\) harmonic, at \(x = -1\) alternating |
| \(\sum x^n / n!\) | every \(x\) | ratio test, below |
| \(\sum n!\, x^n\) | only \(x = 0\) | terms grow without bound |
In each case the set where the series converges is an interval centred at 0, perhaps of radius 0 or infinite radius, perhaps with or without its ends. That is no accident.
Lemma (Abel). If \(\sum c_n x_0^n\) converges for some \(x_0 \ne 0\), then \(\sum c_n x^n\) converges absolutely for every \(x\) with \(|x| < |x_0|\).
Proof. The terms of a convergent series tend to 0, so \(c_n x_0^n \to 0\), and in particular they are bounded: \(|c_n x_0^n| \le M\) for all \(n\). Put \(r = |x|/|x_0|\), which is less than 1. Then
The geometric series \(\sum M r^n\) converges, so by comparison \(\sum |c_n x^n|\) converges. \(\square\)
The same argument, turned around, says that if the series diverges at \(x_1\), it diverges at every \(x\) farther from the centre (if it converged at such an \(x\), the lemma would make it converge at \(x_1\)).
Theorem (radius of convergence). For every power series \(\sum c_n (x-a)^n\) there is a number \(R\) with \(0 \le R \le \infty\), called the radius of convergence, such that the series converges absolutely when \(|x - a| < R\) and diverges when \(|x - a| > R\).
Proof. Shift so that \(a = 0\). Let \(S\) be the set of numbers \(|x|\) for which the series converges. If \(S\) is unbounded, Abel's lemma gives convergence everywhere; put \(R = \infty\). Otherwise let \(R\) be the least upper bound of \(S\) (the smallest number that is at least every element of \(S\); Chapter 18 shows such a number always exists). If \(|x| < R\), there is a point \(x_0\) with \(|x| < |x_0|\) at which the series converges (otherwise \(|x|\) itself would be an upper bound smaller than \(R\)), and Abel's lemma gives absolute convergence at \(x\). If \(|x| > R\), the series cannot converge at \(x\), because then \(|x|\) would be in \(S\) and bigger than its least upper bound. \(\square\)
The interval \((a - R, a + R)\) is the open interval of convergence. The theorem says nothing about the two endpoints \(x = a \pm R\); they must be checked one by one with the tests of Chapter 12.
Finding \(R\)¶
The ratio and root tests turn into formulas for \(R\).
Ratio formula. If \(\left| c_{n+1}/c_n \right| \to L\), then \(R = 1/L\) (with \(1/0 = \infty\) and \(1/\infty = 0\)).
Proof. Apply the ratio test to the series of numbers \(\sum c_n x^n\) for a fixed \(x \ne 0\):
The series converges absolutely if \(L|x| < 1\), that is \(|x| < 1/L\), and diverges if \(L|x| > 1\) (the terms do not tend to 0). So the radius is exactly \(1/L\). \(\square\)
Root formula (Cauchy–Hadamard). If \(|c_n|^{1/n} \to L\), then again \(R = 1/L\). More generally, with no assumption that a limit exists,
The limit superior \(\limsup\) of a bounded sequence is the largest number that some subsequence converges to (for an unbounded sequence it is \(\infty\)). The proof is the root test applied to \(|c_n x^n|^{1/n} = |c_n|^{1/n}|x|\), exactly as above. The root formula always works; the ratio formula needs the ratios to have a limit, which fails for series with gaps such as \(\sum x^{2n}\), where every other coefficient is 0.
Example 1. \(\sum_{n \ge 0} \dfrac{x^n}{n!}\). Here \(c_{n+1}/c_n = n!/(n+1)! = 1/(n+1) \to 0\), so \(R = \infty\): the series converges for every real \(x\).
Example 2. \(\sum_{n \ge 1} \dfrac{(x-3)^n}{n\, 2^n}\). The ratio is
so \(R = 2\) and the series converges absolutely for \(1 < x < 5\). Ends: at \(x = 5\) the terms are \(2^n/(n 2^n) = 1/n\), the harmonic series, which diverges. At \(x = 1\) they are \((-2)^n/(n 2^n) = (-1)^n/n\), which converges by the alternating series test. The set of convergence is \([1, 5)\).
Example 3. \(\sum_{n \ge 1} \dfrac{x^n}{n^2}\). The ratio \(n^2/(n+1)^2 \to 1\) gives \(R = 1\). At both ends \(|x^n/n^2| = 1/n^2\), and \(\sum 1/n^2\) converges, so this series converges on the closed interval \([-1, 1]\).
So the four possibilities at the ends, (open, half-open both ways, closed), all happen: \(\sum x^n\) converges on \((-1,1)\), \(\sum x^n/n\) on \([-1,1)\), \(\sum (-x)^n/n\) on \((-1,1]\), and \(\sum x^n/n^2\) on \([-1,1]\).
Example 4. \(\sum_{n \ge 0} 2^n x^{2n}\). The odd coefficients are 0, so use the series as a geometric one in \(y = 2x^2\): it converges exactly when \(2x^2 < 1\), so \(R = 1/\sqrt 2\). Check with the root formula: the nonzero coefficients are \(c_{2n} = 2^n\), and \(|c_{2n}|^{1/(2n)} = 2^{1/2}\), so \(\limsup |c_n|^{1/n} = \sqrt 2\) and \(R = 1/\sqrt 2\).
13.2 Differentiating and integrating term by term¶
Level: intermediate to advanced.
A polynomial is differentiated one term at a time. Can you do the same with a power series, an infinite sum? Not every infinite sum of functions behaves this well (Chapter 19 has sums of smooth functions whose sum has corners), but power series do.
Theorem (term-by-term calculus). Suppose \(f(x) = \sum_{n \ge 0} c_n (x-a)^n\) has radius of convergence \(R > 0\). Then on the open interval \(|x - a| < R\):
- \(f\) is differentiable, and \(f'(x) = \sum_{n \ge 1} n c_n (x-a)^{n-1}\);
- \(\int_a^x f(t)\,dt = \sum_{n \ge 0} \dfrac{c_n}{n+1}(x-a)^{n+1}\);
- both new series have the same radius \(R\).
Part 3 is easy and worth proving now. The differentiated series \(\sum n c_n x^{n-1}\) converges at a point \(x \ne 0\) exactly when \(\sum n c_n x^n\) does (multiply by the fixed number \(x\)), so it is enough to compare \(\limsup |n c_n|^{1/n}\) with \(\limsup |c_n|^{1/n}\). Since \(n^{1/n} \to 1\) (take logarithms: \(\ln n / n \to 0\)), multiplying the \(n\)th root by \(n^{1/n}\) does not change any subsequential limit, so the two \(\limsup\)s are equal. The integrated series is the same argument with \(1/(n+1)\).
Parts 1 and 2 need more than a single series of numbers: they are about a limit of functions, and the right tool is uniform convergence. The full proof is in Chapter 19, where the key fact is that a power series converges uniformly on every closed interval \([a - r, a + r]\) with \(r < R\). Here we use the theorem.
A consequence: \(f'\) is again a power series with radius \(R\), so it can be differentiated again, and again. A power series is infinitely differentiable inside its interval of convergence. Differentiating \(k\) times and putting \(x = a\) kills every term except the constant one:
So the coefficients of a power series are fixed by the derivatives of its sum at the centre. In particular, a function has at most one power series at a given centre: two series with the same sum near \(a\) have the same coefficients. This uniqueness is what lets you find series by any trick you like and be sure you have the right one.
Example 5 (new series from old). Differentiate the geometric series for \(|x| < 1\):
Multiply by \(x\): \(\sum_{n \ge 1} n x^n = \dfrac{x}{(1-x)^2}\). At \(x = 1/2\) this gives a sum you might meet in probability (the expected number of tosses of a fair coin up to the first head):
Check: \(\tfrac12 + \tfrac24 + \tfrac38 + \tfrac4{16} + \tfrac5{32} = 1.78\), and the partial sums keep climbing towards 2.
Example 6 (logarithm). Replace \(x\) by \(-t\) in the geometric series: \(\dfrac{1}{1+t} = 1 - t + t^2 - t^3 + \dots\) for \(|t| < 1\). Integrate from 0 to \(x\) term by term:
Example 7 (arctangent). Replace \(x\) by \(-t^2\): \(\dfrac{1}{1+t^2} = 1 - t^2 + t^4 - \dots\) for \(|t| < 1\). Since \(\arctan' t = 1/(1+t^2)\) and \(\arctan 0 = 0\),
Both of these also hold at \(x = 1\), which gives two famous sums; that needs an extra argument, given in 13.4.
13.3 Taylor polynomials and Taylor's theorem¶
The previous section went from series to functions. Now go the other way: given a function, such as \(\sin x\), that you can differentiate as often as you like, which polynomial imitates it best near a point \(a\)?
The tangent line \(f(a) + f'(a)(x-a)\) is the degree-1 answer from Chapter 7: it has the same value and the same slope as \(f\) at \(a\). Ask for the same value, slope, second derivative, and so on up to the \(n\)th, and the formula for \(c_k\) in 13.2 tells you the answer at once.
Definition. If \(f\) has \(n\) derivatives at \(a\), its Taylor polynomial of degree \(n\) at \(a\) is
It is the only polynomial of degree at most \(n\) whose value and first \(n\) derivatives at \(a\) match those of \(f\). When \(a = 0\) it is also called a Maclaurin polynomial. If \(f\) has derivatives of every order, letting \(n \to \infty\) gives the Taylor series of \(f\) at \(a\).
Example 8. \(f(x) = \sqrt{1+x}\) at \(a = 0\). The derivatives are
so at 0 they are \(1, \tfrac12, -\tfrac14, \tfrac38\), and dividing by \(0!, 1!, 2!, 3!\):
At \(x = 0.1\): \(P_1 = 1.05\), \(P_2 = 1.04875\), \(P_3 = 1.0488125\), while \(\sqrt{1.1} = 1.0488088\ldots\) Each extra term gains roughly one more correct digit, because each term carries another factor of \(x = 0.1\).
How good is \(P_n\)? The difference \(R_n(x) = f(x) - P_n(x)\) is the remainder. Without a bound on it, a Taylor polynomial is a guess. The theorem that bounds it is the most useful single result in this chapter.
Taylor's theorem (Lagrange form of the remainder). Suppose \(f\) has \(n+1\) derivatives on an open interval containing \(a\) and \(x\). Then there is a number \(c\) strictly between \(a\) and \(x\) such that
For \(n = 0\) this says \(f(x) = f(a) + f'(c)(x - a)\): it is the mean value theorem. Taylor's theorem is the mean value theorem with \(n\) more terms, and the remainder looks exactly like "the next term", except that the derivative is taken at an unknown point \(c\) instead of at \(a\).
Proof. Fix \(x \ne a\). The trick is to treat the centre as the variable. For \(t\) between \(a\) and \(x\) define
where the constant \(M\) is chosen so that \(g(a) = 0\), that is, \(M = R_n(x)/(x-a)^{n+1}\). Also \(g(x) = 0\): at \(t = x\) every power of \((x - t)\) vanishes except in the \(k = 0\) term, which is \(f(x)\), and \(f(x) - f(x) = 0\).
Since \(g(a) = g(x) = 0\), Rolle's theorem (Chapter 6) gives a \(c\) strictly between them with \(g'(c) = 0\). Now differentiate \(g\) with respect to \(t\), using the product rule on each term of the sum:
(for \(k = 0\) the second part is absent). Summed from \(k = 0\) to \(n\), these telescope: the second part of term \(k\) cancels the first part of term \(k-1\), and only the first part of the last term survives. So
Setting \(g'(c) = 0\) and dividing by \((x-c)^n \neq 0\) gives \(M = f^{(n+1)}(c)/(n+1)!\). Since \(M\) was \(R_n(x)/(x-a)^{n+1}\), this is exactly the claimed formula. \(\square\)
In practice you rarely know \(c\). You use the theorem as a bound: if \(|f^{(n+1)}| \le K\) between \(a\) and \(x\), then
Example 9 (how many terms for \(e\)?). Take \(f = e^x\), \(a = 0\), \(x = 1\). Every derivative is \(e^t\), which is at most \(e < 3\) on \([0, 1]\). So
For six correct decimals you need the error below \(5 \times 10^{-7}\). With \(n = 9\) the bound is \(3/10! \approx 8.3 \times 10^{-7}\), not enough; with \(n = 10\) it is \(3/11! \approx 7.5 \times 10^{-8}\), enough. So eleven terms (up to \(1/10!\)) give \(e = 2.718282\) to six decimals.
Example 10 (sine of a small angle). Approximate \(\sin 0.1\) by \(0.1 - 0.1^3/6 = 0.0998333\ldots\) Because the \(x^4\) coefficient of sine is 0, \(P_3 = P_4\), so you may use the remainder with \(n = 4\): it is at most \(|\sin^{(5)}| \cdot 0.1^5/5! \le 0.1^5/120 \approx 8.3 \times 10^{-8}\). Check: \(\sin 0.1 = 0.0998334166\ldots\), and the error is \(8.3 \times 10^{-8}\), right at the bound.
There is a second form of the remainder that needs no unknown point.
Taylor's theorem (integral form). If \(f^{(n+1)}\) is continuous between \(a\) and \(x\), then
Proof. For \(n = 0\) this is the fundamental theorem of calculus: \(f(x) - f(a) = \int_a^x f'(t)\,dt\). Suppose it holds for \(n - 1\). Integrate by parts, with \(u = f^{(n)}(t)\) and \(dv = (x-t)^{n-1}dt/(n-1)!\), so \(v = -(x-t)^n/n!\):
The left side is \(R_{n-1}(x)\) by the induction hypothesis, and \(R_{n-1}(x) - \frac{f^{(n)}(a)}{n!}(x-a)^n = R_n(x)\). \(\square\)
A short history. James Gregory knew series of this kind in the 1670s, and Isaac Newton used them freely. Brook Taylor published the general formula in Methodus Incrementorum (1715), deriving it from finite differences without any discussion of convergence. Colin Maclaurin used the special case at 0 heavily in his Treatise of Fluxions (1742), which is why that case carries his name. The remainder in the form above is due to Joseph-Louis Lagrange, in his Théorie des fonctions analytiques (1797). Lagrange hoped to build all of calculus on power series; 13.6 shows why that hope could not work.
13.4 The great series¶
A function equals its Taylor series at \(x\) exactly when \(R_n(x) \to 0\) as \(n \to \infty\). For the functions below we check this directly from the remainder.
The exponential¶
All derivatives of \(e^x\) equal \(e^x\), which is 1 at 0, so the Taylor series is \(\sum x^n/n!\). On the interval between 0 and \(x\), \(e^t \le e^{|x|}\), so
For a fixed \(x\), the factorial wins: once \(n + 1 > 2|x|\), each further step multiplies \(|x|^{n+1}/(n+1)!\) by less than \(\tfrac12\), so it tends to 0. Therefore, for every real \(x\),
Sine and cosine¶
The derivatives of \(\sin\) cycle through \(\sin, \cos, -\sin, -\cos\), which at 0 are \(0, 1, 0, -1\). Every derivative is bounded by 1, so \(|R_n(x)| \le |x|^{n+1}/(n+1)! \to 0\) for every \(x\), and
for all real \(x\). Sine has only odd powers and cosine only even ones, which matches \(\sin(-x) = -\sin x\) and \(\cos(-x) = \cos x\). Differentiating the sine series term by term gives the cosine series, as it must.
Example 11. \(\cos 0.5\). The degree-4 polynomial gives \(1 - 0.125 + 0.0026042 = 0.8776042\). Since the \(x^5\) coefficient is 0, the remainder can be taken with \(n = 5\): at most \(0.5^6/720 \approx 2.2 \times 10^{-5}\). Check: \(\cos 0.5 = 0.8775826\), error \(2.2 \times 10^{-5}\).
The logarithm, including at \(x = 1\)¶
Example 6 gave \(\ln(1+x) = x - x^2/2 + x^3/3 - \dots\) for \(|x| < 1\). At \(x = 1\) the series is the alternating harmonic series, which converges; but does it converge to \(\ln 2\)? Term-by-term integration was only promised inside the interval. Here is a direct argument. The finite geometric sum gives, for \(t \ne -1\),
(Check: multiply by \(1+t\); the sum telescopes to \(1 - (-t)^n\).) Integrate from 0 to \(x\) with \(0 \le x \le 1\):
The last integral is between 0 and \(\int_0^x t^n\,dt = x^{n+1}/(n+1) \le 1/(n+1)\), which tends to 0. So the series converges to \(\ln(1+x)\) on \([0,1]\), and in particular
It converges painfully slowly: the error after \(n\) terms is about \(1/(2n)\). For computing, \(x\) should be small. Example 12: \(\ln 1.2 \approx 0.2 - 0.02 + 0.0026667 - 0.0004 = 0.1822667\); the error is at most \(0.2^5/5 = 6.4 \times 10^{-5}\). Check: \(\ln 1.2 = 0.1823216\), error \(5.5 \times 10^{-5}\).
The arctangent and \(\pi\)¶
The same argument with \(1/(1+t^2)\) shows that the arctangent series of Example 7 converges to \(\arctan x\) on \([-1, 1]\); the remainder is \(\pm\int_0^x t^{2n}/(1+t^2)\,dt\), at most \(1/(2n+1)\) in size. At \(x = 1\), \(\arctan 1 = \pi/4\), so
This is the Leibniz series (also called the Madhava–Leibniz or Gregory–Leibniz series). Gottfried Leibniz found it in the 1670s; James Gregory had the arctangent series a few years earlier; and Kerala texts of the sixteenth century attribute the series to Madhava of Sangamagrama, around 1400, together with corrections that speed it up.
It is beautiful and useless for computing: 1000 terms give \(3.1405926\ldots\), still wrong in the third decimal, because the error after \(n\) terms is about \(1/n\). The fix is to use \(\arctan\) at small \(x\), where the terms shrink like powers of \(x^2\). In 1706 John Machin found
Proof. Let \(\theta = \arctan\frac15\). The double-angle formula \(\tan 2\theta = 2\tan\theta/(1-\tan^2\theta)\) gives \(\tan 2\theta = \frac{2/5}{24/25} = \frac{5}{12}\), and again \(\tan 4\theta = \frac{10/12}{119/144} = \frac{120}{119}\). Then by the subtraction formula
Since \(0 < 4\theta - \pi/4 < \pi/2\) (because \(\tan 4\theta\) is just above 1), this says \(4\theta - \pi/4 = \arctan\frac{1}{239}\). \(\square\)
Each term of the \(\arctan\frac15\) series is about 25 times smaller than the last, so every term gains about 1.4 digits. Machin computed 100 digits of \(\pi\) this way by hand, and formulas of this kind held the record until the age of computers.
Try it in Python. Exact fractions avoid any rounding:
from fractions import Fraction as F
def arctan_inv(m, terms): # arctan(1/m) by its series
return sum(F((-1)**k, (2*k + 1) * m**(2*k + 1)) for k in range(terms))
for n in (2, 5, 10):
approx = 4 * (4 * arctan_inv(5, n) - arctan_inv(239, n))
print(n, float(approx))
With \(n = 5\) terms the result is \(3.14159268\), already within \(3 \times 10^{-8}\) of \(\pi\); with \(n = 10\) it is correct to all the digits a float can show.
The binomial series¶
One of Newton's early discoveries (around 1665, described in his letters of 1676 to Leibniz, sent through Henry Oldenburg) was that \((1+x)^\alpha\) has a series for every real exponent \(\alpha\), not just whole numbers:
The coefficients are the derivatives at 0 divided by \(k!\), as always: the \(k\)th derivative of \((1+x)^\alpha\) is \(\alpha(\alpha-1)\cdots(\alpha-k+1)(1+x)^{\alpha-k}\). When \(\alpha\) is a whole number the coefficients become 0 after \(k = \alpha\) and you get the ordinary binomial theorem. Example 8 is the case \(\alpha = \tfrac12\). (Proving that the remainder tends to 0 for every \(|x| < 1\) is fiddly with the Lagrange form; a clean route is to show that the series \(g(x)\) satisfies \((1+x)g' = \alpha g\), as \((1+x)^\alpha\) does, and that this forces \(g(x)/(1+x)^\alpha\) to be constant.)
Using series to compute¶
Series are not only for evaluating functions. Because they can be multiplied, substituted and integrated, they solve problems that have no closed form.
Example 13 (an integral with no formula). \(\int_0^1 e^{-x^2}dx\) has no elementary antiderivative (Chapter 10). Substitute \(-x^2\) into the series for \(e^x\) and integrate term by term:
The series alternates with decreasing terms, so the error is less than the first term left out. Five terms give \(0.74749\); the next term is \(1/1320 \approx 0.00076\), so the true value lies between \(0.74673\) and \(0.74749\). It is \(0.746824\ldots\)
Example 14 (a limit). \(\displaystyle\lim_{x\to 0}\frac{\sin x - x}{x^3}\). Replace \(\sin x\) by its series: \(\dfrac{-x^3/6 + x^5/120 - \dots}{x^3} = -\dfrac16 + \dfrac{x^2}{120} - \dots \to -\dfrac16\). Three uses of L'Hôpital's rule give the same answer, with more work.
13.5 Euler's formula¶
Level: intermediate to advanced.
Look at the three series side by side:
The exponential has every term, and the trigonometric functions share its terms between them, with signs that alternate in pairs. The signs are exactly what powers of \(i = \sqrt{-1}\) produce: \(i^0 = 1\), \(i^1 = i\), \(i^2 = -1\), \(i^3 = -i\), \(i^4 = 1\), and then the cycle repeats.
To use complex numbers in a series you need convergence for complex terms. Say \(\sum z_n\) converges to \(s\) if the partial sums get close to \(s\), where "close" uses the complex modulus \(|z|\), the distance from 0 in the plane. Every fact used above carries over: in particular, if \(\sum |z_n|\) converges, so does \(\sum z_n\) (apply the real result to the real and imaginary parts). The series \(\sum z^n/n!\) converges absolutely for every complex \(z\), so we may define
Now put \(z = i\theta\) with \(\theta\) real and sort the terms by the cycle of powers of \(i\) (rearranging is allowed because the series converges absolutely, Chapter 12):
Euler's formula. For every real \(\theta\),
Geometrically, \(e^{i\theta}\) is the point at angle \(\theta\) on the unit circle. At \(\theta = \pi\) it gives \(e^{i\pi} = -1\), often written \(e^{i\pi} + 1 = 0\): five basic constants in one line. Leonhard Euler published the formula in his Introductio in analysin infinitorum (1748); Roger Cotes had found an equivalent statement in logarithmic form around 1714.
The formula is useful, not just pretty. One more fact about the series is needed: \(e^{z}e^{w} = e^{z+w}\) for all complex \(z, w\). Proof: multiply the two absolutely convergent series and collect the terms of total degree \(n\) (the Cauchy product, allowed for absolutely convergent series):
by the binomial theorem. \(\square\)
Example 15 (addition formulas for free). \(e^{i(a+b)} = e^{ia}e^{ib}\). Expand both sides with Euler's formula:
Comparing real and imaginary parts gives both addition formulas at once.
Example 16 (de Moivre). \((\cos\theta + i\sin\theta)^n = (e^{i\theta})^n = e^{in\theta} = \cos n\theta + i\sin n\theta\). With \(n = 3\), expanding the left side and taking the real part: \(\cos 3\theta = \cos^3\theta - 3\cos\theta\sin^2\theta = 4\cos^3\theta - 3\cos\theta\).
Example 17 (cosine as an average). Adding \(e^{i\theta} = \cos\theta + i\sin\theta\) and \(e^{-i\theta} = \cos\theta - i\sin\theta\):
This is how the differential equations of Chapter 14 turn oscillations into exponentials, and how Fourier series (Chapter 21) are usually written. Compare the hyperbolic functions \(\cosh x = (e^x + e^{-x})/2\) and \(\sinh x = (e^x - e^{-x})/2\): the same formulas without the \(i\), which is why \(\cosh\) and \(\sinh\) have the series of \(\cos\) and \(\sin\) with every sign \(+\).
13.6 A smooth function that is not its Taylor series¶
Level: advanced.
So far every function we tried equalled its Taylor series. Is that always so, as long as the function has derivatives of every order? Lagrange, building calculus on series, assumed something close to it. Augustin-Louis Cauchy showed in his 1823 lectures that it is false, with this example:
The graph is flat near 0 to an astonishing degree: \(f(0.1) = e^{-100} \approx 3.7 \times 10^{-44}\).
Claim. \(f\) has derivatives of every order at every point, and \(f^{(n)}(0) = 0\) for all \(n\).
Proof. Away from 0, \(f\) is a composition of smooth functions, so it has all derivatives there. By induction on \(n\), for \(x \ne 0\),
for some polynomial \(p_n\). For \(n = 0\), \(p_0 = 1\). If it holds for \(n\), differentiate with the chain and product rules, writing \(u = 1/x\) so that \(du/dx = -u^2\):
since \(\frac{d}{dx}e^{-1/x^2} = 2x^{-3}e^{-1/x^2} = 2u^3 e^{-1/x^2}\). That is a polynomial in \(u\) times \(e^{-1/x^2}\), as claimed. (For example \(f''(x) = (4x^{-6} - 6x^{-4})e^{-1/x^2}\).)
Next, a limit: for every whole number \(m\), \(\dfrac{e^{-1/x^2}}{x^m} \to 0\) as \(x \to 0\). Put \(y = 1/x^2\); for \(x \ne 0\), \(|1/x^m| = y^{m/2}\), so the expression has size \(y^{m/2}e^{-y}\), and as \(x \to 0\), \(y \to \infty\). An exponential beats any power: from the series, \(e^y > y^k/k!\) for \(y > 0\) and any \(k\), so with \(k > m/2\), \(y^{m/2}e^{-y} < k!\, y^{m/2 - k} \to 0\).
Now show \(f^{(n)}(0) = 0\) by induction. For \(n = 0\) it is the definition. If \(f^{(n)}(0) = 0\), then
since \(p_n(1/x)/x\) is a finite sum of terms \(c/x^m\), each of which, times \(e^{-1/x^2}\), tends to 0. \(\square\)
So the Taylor series of \(f\) at 0 is \(0 + 0x + 0x^2 + \dots\), which converges everywhere, to the zero function. But \(f(x) > 0\) for every \(x \ne 0\). The Taylor series converges, and to the wrong function. All the information in the derivatives at 0 does not determine \(f\) even a tiny distance away.
Words. A function with derivatives of every order is smooth, or \(C^\infty\). A function that equals a convergent power series near every point of its domain is real analytic, or \(C^\omega\). Every analytic function is smooth (13.2), and Cauchy's example shows that not every smooth function is analytic.
Why should anyone care? Three reasons.
- Bump functions. Using \(f\), the function \(g(x) = f(x)\) for \(x > 0\) and \(g(x) = 0\) for \(x \le 0\) is smooth, and so is \(g(x)\,g(1-x)\), which is positive on \((0,1)\) and zero elsewhere. Smooth functions that switch on and off like this are impossible for analytic functions: an analytic function that is zero on an interval is zero on its whole interval of convergence (all its derivatives vanish there, so all its coefficients vanish). Bump functions are the basic tool for gluing local constructions together in analysis and geometry (the "partitions of unity" of MAT257Y1 Unit 10).
- No free lunch from derivatives. Émile Borel proved in 1895 that every sequence of numbers is the sequence of derivatives at 0 of some smooth function. So a Taylor series can even diverge everywhere except at its centre, for a perfectly smooth function. We state this without proof.
- The complex contrast. For functions of a complex variable, having one derivative everywhere in a disc already forces the function to equal its power series in that disc. This rigidity is the heart of complex analysis (Chapter 22). Cauchy's example shows why: as a function of complex \(z\), \(e^{-1/z^2}\) at \(z = iy\) is \(e^{1/y^2}\), which blows up as \(y \to 0\), so in the complex plane it is not even continuous at 0.
This also explains a puzzle about radius of convergence. The series of \(1/(1+x^2) = 1 - x^2 + x^4 - \dots\) has radius 1, though the function is perfectly smooth on the whole real line. The reason is in the complex plane: \(1/(1+z^2)\) blows up at \(z = \pm i\), which are at distance 1 from 0. In general the radius of convergence of a Taylor series is the distance from the centre to the nearest complex point where the function misbehaves; that is a theorem of complex analysis.
Check yourself¶
1. Find the radius and the exact set of convergence of \(\sum_{n \ge 1} \frac{(x+1)^n}{3^n \sqrt n}\).
Ratio: \(\frac{3^n\sqrt n}{3^{n+1}\sqrt{n+1}} \to \frac13\), so \(R = 3\), centre \(-1\), and the open interval is \((-4, 2)\). At \(x = 2\) the terms are \(1/\sqrt n\), a \(p\)-series with \(p = \tfrac12\), which diverges. At \(x = -4\) they are \((-1)^n/\sqrt n\), which converges by the alternating series test. The set is \([-4, 2)\).
2. Find the radius of convergence of \(\sum_{n \ge 1} \frac{n^n}{n!}x^n\).
Ratio: \(\frac{(n+1)^{n+1}}{(n+1)!}\cdot\frac{n!}{n^n} = \left(\frac{n+1}{n}\right)^n = \left(1 + \frac1n\right)^n \to e\). So \(R = 1/e\).
3. Use known series to find the Maclaurin series of \(x^2 e^{-x}\) and give its 10th derivative at 0.
\(x^2 e^{-x} = x^2\sum \frac{(-x)^n}{n!} = \sum_{n \ge 0}\frac{(-1)^n}{n!}x^{n+2}\). The coefficient of \(x^{10}\) comes from \(n = 8\): \(\frac{1}{8!}\). Since \(c_{10} = f^{(10)}(0)/10!\), \(f^{(10)}(0) = \frac{10!}{8!} = 90\).
4. Find the sum of \(\sum_{n \ge 1} \frac{n^2}{2^n}\).
From \(\sum n x^n = x/(1-x)^2\), differentiate and multiply by \(x\): \(\sum n^2 x^n = x\frac{d}{dx}\frac{x}{(1-x)^2} = \frac{x(1+x)}{(1-x)^3}\). At \(x = \tfrac12\): \(\frac{(1/2)(3/2)}{1/8} = 6\). Check: the partial sums \(0.5, 1.5, 2.625, 3.625, \dots\) approach 6.
5. Write the Taylor polynomial of degree 3 of \(\ln x\) at \(a = 1\), and bound its error at \(x = 1.1\).
Derivatives: \(1/x, -1/x^2, 2/x^3, -6/x^4\); at 1 they are \(1, -1, 2\). So \(P_3 = (x-1) - \frac{(x-1)^2}{2} + \frac{(x-1)^3}{3}\), and \(P_3(1.1) = 0.1 - 0.005 + 0.000333 = 0.0953333\). Remainder: \(|f^{(4)}(c)|/4! \cdot 0.1^4 = \frac{6}{c^4 \cdot 24}\cdot 10^{-4} \le 2.5 \times 10^{-5}\) for \(c \ge 1\). Check: \(\ln 1.1 = 0.0953102\), error \(2.3 \times 10^{-5}\).
6. How many terms of the series for \(\sin x\) guarantee \(\sin 1\) to within \(10^{-6}\)?
Using \(P_{2m+1}\) and the remainder with \(n = 2m+2\) (the next coefficient is 0), the error is at most \(1/(2m+3)!\). Need \((2m+3)! > 10^6\): \(9! = 362880\) is too small, \(11! = 39916800\) works, so \(2m + 3 = 11\), \(m = 4\): the polynomial up to \(x^9\), five nonzero terms: \(1 - \frac16 + \frac1{120} - \frac1{5040} + \frac1{362880} = 0.8414710\). Check: \(\sin 1 = 0.8414710\).
7. Use series to find \(\lim_{x\to0}\frac{e^x - 1 - x - x^2/2}{x^3}\) and \(\lim_{x \to 0}\frac{1-\cos x}{x^2}\).
\(e^x - 1 - x - x^2/2 = x^3/6 + x^4/24 + \dots\), so the first limit is \(\tfrac16\). \(1 - \cos x = x^2/2 - x^4/24 + \dots\), so the second is \(\tfrac12\).
8. Verify Machin-type formula \(\frac{\pi}{4} = \arctan\frac12 + \arctan\frac13\) (Euler's) and say why it converges more slowly than Machin's.
\(\tan(A+B) = \frac{1/2 + 1/3}{1 - 1/6} = \frac{5/6}{5/6} = 1\), and \(0 < A + B < \pi\), so \(A + B = \pi/4\). The terms of \(\arctan\frac12\) shrink by a factor of about 4 per step, against about 25 for \(\arctan\frac15\), so each term gains about 0.6 digits instead of about 1.4.
9. Use Euler's formula to compute \((1+i)^8\) and to express \(\sin^2\theta\) without squares.
\(1 + i = \sqrt2\,e^{i\pi/4}\), so \((1+i)^8 = 16\,e^{2\pi i} = 16\). \(\sin^2\theta = \left(\frac{e^{i\theta} - e^{-i\theta}}{2i}\right)^2 = \frac{e^{2i\theta} - 2 + e^{-2i\theta}}{-4} = \frac{1 - \cos 2\theta}{2}\).
10. Show that the Taylor series of \(\frac{1}{1-x}\) at \(a = 3\) has radius 2, and explain why with the picture of 13.6.
\(\frac{1}{1-x} = \frac{1}{-2 - (x-3)} = -\frac12\cdot\frac{1}{1 + (x-3)/2} = -\frac12\sum_{n\ge0}\left(-\frac{x-3}{2}\right)^n\), which converges when \(|x-3| < 2\). The function blows up at \(x = 1\), at distance 2 from the centre 3, and the radius is exactly the distance to that bad point.
11. Let \(g(x) = e^{-1/x^2}\) for \(x > 0\) and \(g(x) = 0\) for \(x \le 0\). Is \(g\) smooth? Is it analytic at 0?
Smooth: for \(x > 0\) its derivatives are those of \(e^{-1/x^2}\); for \(x < 0\) they are 0; and at 0 every one-sided derivative is 0 by the argument of 13.6, so all derivatives exist and are continuous. It is not analytic at 0: its Taylor series there is 0, but \(g(x) > 0\) for every \(x > 0\), so no power series equals \(g\) on any interval around 0.
What you learned¶
- A power series converges absolutely inside an interval \(|x-a| < R\) and diverges outside it; the ends must be checked separately, and every combination happens.
- \(R\) comes from the ratio test or, always, from \(1/R = \limsup |c_n|^{1/n}\).
- Inside the interval a power series can be differentiated and integrated term by term (proof in Chapter 19), so its coefficients are \(f^{(k)}(a)/k!\) and the series is unique.
- Taylor's theorem: \(f(x) = P_n(x) + f^{(n+1)}(c)(x-a)^{n+1}/(n+1)!\), the mean value theorem with more terms; the remainder bound tells you how many terms you need.
- \(e^x\), \(\sin x\) and \(\cos x\) equal their series everywhere; \(\ln(1+x)\) and \(\arctan x\) on \((-1, 1]\) and \([-1, 1]\); Machin's formula turns the arctangent series into a fast way to compute \(\pi\).
- Euler's formula \(e^{i\theta} = \cos\theta + i\sin\theta\) follows from the series and gives trigonometric identities for free.
- Smooth is not analytic: \(e^{-1/x^2}\) has all derivatives 0 at 0 but is not the zero function.
Next: Chapter 14, equations whose unknown is a function, and how to solve them exactly, approximately and in principle.
Further reading¶
- M. Spivak, Calculus, 4th edition (2008), chapters 20, 23 and 24: Taylor's theorem and power series, with full proofs.
- T. M. Apostol, Calculus, Volume 1, 2nd edition (1967), chapters 7, 10 and 11.
- E. Hairer and G. Wanner, Analysis by Its History (1996): the series of Newton, Leibniz, Euler and Taylor in their historical order.
- G. H. Hardy, A Course of Pure Mathematics (1908; 10th edition 1952), chapters 8 and 9.
- J. M. Borwein and P. B. Borwein, Pi and the AGM (1987), for how \(\pi\) was computed from Machin to the present.
- On this site: MAT137 Unit 14: Power Series & Taylor Series, MAT159H1 Unit 4: Taylor's Theorem, MAT159H1 Unit 7: Power Series; the gentler convergence tests lesson; Algebra, Chapter 8 and Chapter 9 for series and complex numbers.