Differential and Integral Calculus
5. Taylor polynomial
Taylor polynomial
Example
Compare the graph of \(\sin x\) (red) with the graphs of the polynomials \[ x-\frac{x^3}{3!}+\frac{x^5}{5!}-\dots + \frac{(-1)^nx^{2n+1}}{(2n+1)!} \] (blue) for \(n=1,2,3,\dots,12\).
\(\displaystyle\sum_{k=0}^{n}\frac{(-1)^{k}x^{2k+1}}{(2k+1)!}\)
Definition: Taylor polynomial
Let \(f\) be \(k\) times differentiable at the point \(x_{0}\). Then the Taylor polynomial \begin{align} P_n(x)&=P_n(x;x_0)\\\ &=f(x_0)+f'(x_0)(x-x_0)+\frac{f''(x_0)}{2!}(x-x_0)^2+ \\ & \dots +\frac{f^{(n)}(x_0)}{n!}(x-x_0)^n\\ &=\sum_{k=0}^n\frac{f^{(k)}(x_0)}{k!}(x-x_0)^k\\ \end{align} is the best polynomial approximation of degree \(n\) (with respect to the derivative) for a function \(f\), close to the point \(x_0\).
Note. The special case \(x_0=0\) is often called the Maclaurin polynomial.
If \(f\) is \(n\) times differentiable at \(x_0\), then the Taylor polynomial has the same derivatives at \(x_0\) as the function \(f\), up to the order \(n\) (of the derivative).
The reason (case \(x_0=0\)): Let \[ P_n(x)=c_0+c_1x+c_2x^2+c_3x^3+\dots +c_nx^n, \] so that \begin{align} P_n'(x)&=c_1+2c_2x+3c_3x^2+\dots +nc_nx^{n-1}, \\ P_n''(x)&=2c_2+3\cdot 2 c_3x\dots +n(n-1)c_nx^{n-2} \\ P_n'''(x)&=3\cdot 2 c_3\dots +n(n-1)(n-2)c_nx^{n-3} \\ \dots && \\ P^{(k)}(x)&=k!c_k + x\text{ terms} \\ \dots & \\ P^{(n)}(x)&=n!c_n \\ P^{(n+1)}(x)&=0. \end{align}
From these way we obtain the coefficients one by one: \begin{align} c_0= P_n(0)=f(0) &\Rightarrow c_0=f(0) \\ c_1=P_n'(0)=f'(0) &\Rightarrow c_1=f'(0) \\ 2c_2=P_n''(0)=f''(0) &\Rightarrow c_2=\frac{1}{2}f''(0) \\ \vdots & \\ k!c_k=P_n^{(k)}(0)=f^{(k)}(0) &\Rightarrow c_k=\frac{1}{k!}f^{(k)}(0). \\ \vdots &\\ n!c_n=P_n^{(n)}(0)=f^{(n)}(0) &\Rightarrow c_k=\frac{1}{n!}f^{(n)}(0). \end{align} Starting from index \(k=n+1\) we cannot pose any new conditions, since \(P^{(n+1)}(x)=0\).
Taylor's Formula
If the derivative \(f^{(n+1)}\) exists and is continuous on some interval \(I=\, ]x_0-\delta,x_0+\delta[\), then \(f(x)=P_n(x;x_0)+E_n(x)\) and the error term \(E_n(x)\) satisfies \[ E_n(x)=\frac{f^{(n+1)}(c)}{(n+1)!}(x-x_0)^{n+1} \] at some point \(c\in [x_0,x]\subset I\). If there is a constant \(M\) (independent of \(n\)) such that \(|f^{(n+1)}(x)|\le M\) for all \(x\in I\), then \[ |E_n(x)|\le \frac{M}{(n+1)!}|x-x_0|^{n+1} \to 0 \] as \(n\to\infty\).
\neq omitted here (mathematical induction or integral).
Examples of Maclaurin polynomial approximations: \begin{align} \frac{1}{1-x} &\approx 1+x+x^2+\dots +x^n =\sum_{k=0}^{n}x^k\\ e^x&\approx 1+x+\frac{1}{2!}x^2+\frac{1}{3!}x^3+\dots + \frac{1}{n!}x^n =\sum_{k=0}^{n}\frac{x^k}{k!}\\ \ln (1+x)&\approx x-\frac{1}{2}x^2+\frac{1}{3}x^3-\dots + \frac{(-1)^{n-1}}{n}x^n =\sum_{k=1}^{n}\frac{(-1)^{k-1}}{k}x^k\\ \sin x &\approx x-\frac{1}{3!}x^3+\frac{1}{5!}x^5-\dots +\frac{(-1)^n}{(2n+1)!}x^{2n+1} =\sum_{k=0}^{n}\frac{(-1)^k}{(2k+1)!}x^{2k+1}\\ \cos x &\approx 1-\frac{1}{2!}x^2+\frac{1}{4!}x^4-\dots +\frac{(-1)^n}{(2n)!}x^{2n} =\sum_{k=0}^{n}\frac{(-1)^k}{(2k)!}x^{2k} \end{align}
Example
Which polynomial \(P_n(x)\) approximates the function \(\sin x\) in the interval \([-\pi,\pi]\) so that the absolute value of the error is less than \(10^{-6}\)?
We use Taylor's Formula for \(f(x)=\sin x\) at \(x_0=0\). Then \(|f^{(n+1)}(c)|\le 1\) independently of \(n\) and the point \(c\). Also, in the interval in question, we have \(|x-x_0|=|x|\le \pi\). The requirement will be satisfied (at least) if \[ |E_n(x)|\le \frac{1}{(n+1)!}\pi^{n+1} < 10^{-6}. \] This inequality must be solved by trying different values of \(n\); it is true for \(n\ge 16\).
The required approximation is achieved with \(P_{16}(x)\), which fo sine is the same as \(P_{15}(x)\).
Check from graphs: \(P_{13}(x)\) is not enough, so the theoretical bound is sharp!
Taylor polynomial and extreme values
If \(f'(x_0)=0\), then also some higher derivatives may be zero: \[ f'(x_0)=f''(x_0)= \dots = f^{(n)}(x_0) =0,\ f^{(n+1)}(x_0) \neq 0. \] Then the behaviour of \(f\) near \(x=x_0\) is determined by the leading term (after the constant term \(f(x_0)\)) \[ \frac{f^{(n+1)}(x_0)}{(n+1)!}(x-x_0)^{n+1}. \] of the Taylor polynomial.
This leads to the following result:
Extreme values
- If \(n\) is even, then \(x_0\) is not an extreme point of \(f\).
- If \(n\) is odd and \(f^{(n+1)}(x_0)>0\), then \(f\) has a local minimum at \(x_0\).
- If \(n\) is odd and \(f^{(n+1)}(x_0)<0\), then \(f\) has a local maximum at \(x_0\).
Newton's method
The first Taylor polynomial \(P_1(x)=f(x_0)+f'(x_0)(x-x_0)\) is the same as the linearization of \(f\) at the point \(x_0\). This can be used in some simple approximations and numerical methods.
Newton's method
The equation \(f(x)=0\) can be solved approximately by choosing a starting point \(x_0\) (e.g. by looking at the graph) and defining \[ x_{n+1}=x_n-\frac{f(x_n)}{f'(x_n)} \] for \(n=0,1,2,\dots\) This leads to a sequence \((x_0,x_1,x_2,\dots )\), whose terms usually give better and better approximations for a zero of \(f\).
The recursion formula is based on the geometric idea of finding an approximative zero of \(f\) by using its linearization (i.e. the tangent line).
Example
Find an approximate value of \(\sqrt{2}\) by using Newton's method.
We use Newton's method for the function \(f(x)=x^2-2\) and initial value \(x_0=2\). The recursion formula becomes \[ x_{n+1}= x_n-\frac{x_n^2-2}{2x_n} = \frac{1}{2}\left(x_n+\frac{2}{x_n}\right), \] from which we obtain \(x_1=1{,}5\), \(x_2\approx 1{,}41667\), \(x_3\approx 1{,}4142157\) and so on.
By experimenting with these values, we find that the number of correct decimal places doubles at each step, and \(x_7\) gives already 100 correct decimal places, if intermediate steps are calculated with enough precision.
Taylor series
Taylor series
If the error term \(E_n(x)\) in Taylor's Formula goes to zero as \(n\) increases, then the limit of the Taylor polynomial is the Taylor series of \(f\) (= Maclaurin series for \(x_0=0\)).
The Taylor series of \(f\) is of the form \[ \sum_{k=0}^{\infty}\frac{f^{(k)}(x_0)}{k!}(x-x_0)^k = \lim_{n\to\infty} \sum_{k=0}^{n}\frac{f^{(k)}(x_0)}{k!}(x-x_0)^k . \] This is an example of a power series.
The Taylor series can be formed as soon as \(f\) has derivatives of all orders at \(x_0\) and they are substituted into this formula. There are two problems related to this: Does the Taylor series converge for all values of \(x\)?
Answer: Not always; for example, the function \[ f(x)=\frac{1}{1-x} \] has a Maclaurin series (= geometric series) converging only for \(-1 < x < 1\), although the function is differentiable for all \(x\neq 1\): \[ f(x)=\frac{1}{1-x} = 1+x+x^2+x^3+x^4+\dots \]
If the series converges for some \(x\), then does its sum equal \(f(x)\)? Answer: Not always; for example, the function \[ f(x)=\begin{cases} e^{-1/x^2}, & x\neq 0,\\ 0, & x=0,\\ \end{cases} \] satisfies \(f^{(k)}(0)=0\) for all \(k\in \mathbf{N}\) (elementary but difficult calculation). Thus its Maclaurin series is identically zero and converges to \(f(x)\) only at \(x=0\).
Conclusion: Taylor series should be studied carefully using the error terms. In practice, the series are formed by using some well known basic series.
Examples
\begin{align} \frac{1}{1-x} &= \sum_{k=0}^{\infty} x^k,\ \ |x|< 1 \\ e^x &= \sum_{k=0}^{\infty} \frac{1}{k!}x^k, \ \ x\in \mathbb{R} \\ \sin x &= \sum_{k=0}^{\infty} \frac{(-1)^{k}}{(2k+1)!} x^{2k+1}, \ \ x\in \mathbb{R} \\ \cos x &= \sum_{k=0}^{\infty} \frac{(-1)^{k}}{(2k)!} x^{2k},\ \ x\in \mathbb{R} \\ (1+x)^r &= 1+\sum_{k=1}^{\infty} \frac{r(r-1)(r-2)\dots (r-k+1)}{k!}x^k, |x|<1 \end{align} The last is called the Binomial Series and is valid for all \(r\in \mathbb{R}\). If \(r=n \in \mathbb{N}\), then starting from \(k=n+1\), all the coefficients are zero and in the beginning \[ \binom{n}{k} =\frac{n!}{k!(n-k)!} = \frac{n(n-1)(n-2)\dots (n-k+1)}{k!}. \]
Compare this to the Binomial Theorem: \[ (a+b)^n=\sum_{k=0}^n\binom{n}{k} a^{n-k}b^k =a^n +na^{n-1}b+\dots +b^n \] for \(n\in\mathbb{N}\).
Power series
Definition: Power series
A power series is of the form \[ \sum_{k=0}^{\infty} c_k(x-x_0)^k = \lim_{n\to\infty} \sum_{k=0}^{n}c_k(x-x_0)^k. \] The point \(x_0\) is the centre and the \(c_k\) are the coefficients of the series.
The series converges at \(x\) if the above limit is defined.
There are only three essentially different cases:
Abel's Theorem.
- The power series converges only for \(x=x_0\) (and then it consists of the constant \(c_0\) only)
- The power series converges for all \(x\in \mathbb{R}\)
- The power series converges on some interval \(]x_0-R,x_0+R[\) (and possibly in one or both of the end points), and diverges for other values of \(x\).
The number \(R\) is the radius of convergence of the series. In the first two cases we say that \(R=0\) or \(R=\infty\) respectively.
Example
For which values of the variable \(x\) does the power series \[\sum_{k=1}^{\infty} \frac{k}{2^k}x^k\] converge?
We use the ratio test with \(a_k=kx^k/2^k\). Then \[ \left| \frac{a_{k+1}}{a_k} \right| = \left| \frac{(k+1)x^{k+1}/2^{k+1}}{kx^k/2^k} \right| = \frac{k+1}{2k}|x| \to \frac{|x|}{2} \] as \(k\to\infty\). By the ratio test, the series converges for \(|x|/2<1\), and diverges for \(|x|/2>1\). In the border-line cases \(|x|/2= 1\Leftrightarrow x=\pm 2\) the general term of the series does not tend to zero, so the series diverges.
Result: The series converges for \(-2< x< 2\), and diverges otherwise.
Definition: Sum function
In the interval \(I\) where the series converges, we can define a function \(f\colon I\to \mathbb{R}\) by setting \begin{equation} \label{summafunktio} f(x) = \sum_{k=0}^{\infty} c_k(x-x_0)^k, \tag{1} \end{equation} which is called the sum function of the power series.
The sum function \(f\) is continuous and differentiable on \(]x_0-R,x_0+R[\). Moreover, the derivative \(f'(x)\) can be calculated by differentiating the sum function term by term: \[ f'(x)=\sum_{k=1}^{\infty}kc_k(x-x_0)^{k-1}. \] Note. The constant term \(c_0\) disappears and the series starts with \(k=1\). The differentiated series converges in the same interval \(x\in \, ]x_0-R,x_0+R[\); this may sound a bit surprising because of the extra coefficient \(k\).
Example
Find the sum function of the power series \(1+2x+3x^2+4x^3+\dots\)
This series is obtained by differentiating termwise the geometric series (with \(q=x\)). Therefore, \begin{align} 1+2x+3x^2+4x^3+\dots &= D(1+x+x^2+x^3+x^4+\dots ) \\ &= \frac{d}{dx}\left( \frac{1}{1-x}\right) = \frac{1}{(1-x)^2}. \end{align} Multiplying with \(x\) we obtain \[ \sum_{k=1}^{\infty}kx^{k} = x+2x^2+3x^3+4x^4+\dots = \frac{x}{(1-x)^2}, \] which is valid for \(|x|<1\).
In the case \([a,b]\subset\ ]x_0-R,x_0+R[\) we can also integrate the sum function termwise: \[ \int_a^b f(x)\, dx = \sum_{k=0}^{\infty}c_k\int_a^b (x-x_0)^k\, dx. \] Often the definite integral can be extended up to the end points of the interval of convergence, but this is not always the case.
Example
Calculate the sum of the alternating harmonic series.
Let us first substitute \(q=-x\) to the geometric series. This yields \[ 1-x+x^2-x^3+x^4-\dots =\frac{1}{1-(-x)} = \frac{1}{1+x}. \] By integrating both sides from \(x=0\) to \(x=1\) we obtain \[ 1-\frac{1}{2}+\frac{1}{3}-\frac{1}{4}+\dots =\int_0^1\frac{1}{1+x} =\ln 2. \] Note. Extending the limit of integration all the way up to \(x=1\) should be justified more rigorously here. We shall return to integration later on the course.