# Differential and Integral Calculus

## Differential and Integral Calculus

### 5. Taylor polynomial

### Taylor polynomial

##### Example

Compare the graph of \(\sin x\) (red) with the graphs of the polynomials \[ x-\frac{x^3}{3!}+\frac{x^5}{5!}-\dots + \frac{(-1)^nx^{2n+1}}{(2n+1)!} \] (blue) for \(n=1,2,3,\dots,12\).

##### Definition: Taylor polynomial

Let \(f\) be \(k\) times differentiable at the point \(x_{0}\). Then the Taylor polynomial \begin{align} P_n(x)&=P_n(x;x_0)\\\ &=f(x_0)+f'(x_0)(x-x_0)+\frac{f''(x_0)}{2!}(x-x_0)^2+ \\ & \dots +\frac{f^{(n)}(x_0)}{n!}(x-x_0)^n\\ &=\sum_{k=0}^n\frac{f^{(k)}(x_0)}{k!}(x-x_0)^k\\ \end{align} is the best polynomial approximation of degree \(n\) (with respect to the derivative) for a function \(f\), close to the point \(x_0\).

**Note.** The special case \(x_0=0\) is often called the *Maclaurin polynomial*.

If \(f\) is \(n\) times differentiable at \(x_0\), then the Taylor polynomial has the same derivatives at \(x_0\) as the function \(f\), up to the order \(n\) (of the derivative).

**The reason** (case \(x_0=0\)): Let
\[
P_n(x)=c_0+c_1x+c_2x^2+c_3x^3+\dots +c_nx^n,
\]
so that
\begin{align}
P_n'(x)&=c_1+2c_2x+3c_3x^2+\dots +nc_nx^{n-1}, \\
P_n''(x)&=2c_2+3\cdot 2 c_3x\dots +n(n-1)c_nx^{n-2} \\
P_n'''(x)&=3\cdot 2 c_3\dots +n(n-1)(n-2)c_nx^{n-3} \\
\dots && \\
P^{(k)}(x)&=k!c_k + x\text{ terms} \\
\dots & \\
P^{(n)}(x)&=n!c_n \\
P^{(n+1)}(x)&=0.
\end{align}

From these way we obtain the coefficients one by one: \begin{align} c_0= P_n(0)=f(0) &\Rightarrow c_0=f(0) \\ c_1=P_n'(0)=f'(0) &\Rightarrow c_1=f'(0) \\ 2c_2=P_n''(0)=f''(0) &\Rightarrow c_2=\frac{1}{2}f''(0) \\ \vdots & \\ k!c_k=P_n^{(k)}(0)=f^{(k)}(0) &\Rightarrow c_k=\frac{1}{k!}f^{(k)}(0). \\ \vdots &\\ n!c_n=P_n^{(n)}(0)=f^{(n)}(0) &\Rightarrow c_k=\frac{1}{n!}f^{(n)}(0). \end{align} Starting from index \(k=n+1\) we cannot pose any new conditions, since \(P^{(n+1)}(x)=0\).

##### Taylor's Formula

If the derivative \(f^{(n+1)}\) exists and is continuous on some interval \(I=\, ]x_0-\delta,x_0+\delta[\), then \(f(x)=P_n(x;x_0)+E_n(x)\) and the error term \(E_n(x)\) satisfies \[ E_n(x)=\frac{f^{(n+1)}(c)}{(n+1)!}(x-x_0)^{n+1} \] at some point \(c\in [x_0,x]\subset I\). If there is a constant \(M\) (independent of \(n\)) such that \(|f^{(n+1)}(x)|\le M\) for all \(x\in I\), then \[ |E_n(x)|\le \frac{M}{(n+1)!}|x-x_0|^{n+1} \to 0 \] as \(n\to\infty\).

*\neq omitted here (mathematical induction or integral).*

Examples of Maclaurin polynomial approximations: \begin{align} \frac{1}{1-x} &\approx 1+x+x^2+\dots +x^n =\sum_{k=0}^{n}x^k\\ e^x&\approx 1+x+\frac{1}{2!}x^2+\frac{1}{3!}x^3+\dots + \frac{1}{n!}x^n =\sum_{k=0}^{n}\frac{x^k}{k!}\\ \ln (1+x)&\approx x-\frac{1}{2}x^2+\frac{1}{3}x^3-\dots + \frac{(-1)^{n-1}}{n}x^n =\sum_{k=1}^{n}\frac{(-1)^{k-1}}{k}x^k\\ \sin x &\approx x-\frac{1}{3!}x^3+\frac{1}{5!}x^5-\dots +\frac{(-1)^n}{(2n+1)!}x^{2n+1} =\sum_{k=0}^{n}\frac{(-1)^k}{(2k+1)!}x^{2k+1}\\ \cos x &\approx 1-\frac{1}{2!}x^2+\frac{1}{4!}x^4-\dots +\frac{(-1)^n}{(2n)!}x^{2n} =\sum_{k=0}^{n}\frac{(-1)^k}{(2k)!}x^{2k} \end{align}

##### Example

Which polynomial \(P_n(x)\) approximates the function \(\sin x\) in the interval \([-\pi,\pi]\) so that the absolute value of the error is less than \(10^{-6}\)?

We use Taylor's Formula for \(f(x)=\sin x\) at \(x_0=0\). Then \(|f^{(n+1)}(c)|\le 1\) independently of \(n\) and the point \(c\). Also, in the interval in question, we have \(|x-x_0|=|x|\le \pi\). The requirement will be satisfied (at least) if \[ |E_n(x)|\le \frac{1}{(n+1)!}\pi^{n+1} < 10^{-6}. \] This inequality must be solved by trying different values of \(n\); it is true for \(n\ge 16\).

The required approximation is achieved with \(P_{16}(x)\), which fo sine is the same as \(P_{15}(x)\).

Check from graphs: \(P_{13}(x)\) is not enough, so the theoretical bound is sharp!

### Taylor polynomial and extreme values

If \(f'(x_0)=0\), then also some higher derivatives may be zero: \[ f'(x_0)=f''(x_0)= \dots = f^{(n)}(x_0) =0,\ f^{(n+1)}(x_0) \neq 0. \] Then the behaviour of \(f\) near \(x=x_0\) is determined by the leading term (after the constant term \(f(x_0)\)) \[ \frac{f^{(n+1)}(x_0)}{(n+1)!}(x-x_0)^{n+1}. \] of the Taylor polynomial.

This leads to the following result:

##### Extreme values

- If \(n\) is even, then \(x_0\) is not an extreme point of \(f\).
- If \(n\) is odd and \(f^{(n+1)}(x_0)>0\), then \(f\) has a local minimum at \(x_0\).
- If \(n\) is odd and \(f^{(n+1)}(x_0)<0\), then \(f\) has a local maximum at \(x_0\).

### Newton's method

The first Taylor polynomial \(P_1(x)=f(x_0)+f'(x_0)(x-x_0)\) is the same as the linearization of \(f\) at the point \(x_0\). This can be used in some simple approximations and numerical methods.

##### Newton's method

The equation \(f(x)=0\) can be solved approximately by choosing a starting point \(x_0\) (e.g. by looking at the graph) and defining \[ x_{n+1}=x_n-\frac{f(x_n)}{f'(x_n)} \] for \(n=0,1,2,\dots\) This leads to a sequence \((x_0,x_1,x_2,\dots )\), whose terms usually give better and better approximations for a zero of \(f\).

The recursion formula is based on the geometric idea of finding an approximative zero of \(f\) by using its linearization (i.e. the tangent line).

##### Example

Find an approximate value of \(\sqrt{2}\) by using Newton's method.

We use Newton's method for the function \(f(x)=x^2-2\) and initial value \(x_0=2\). The recursion formula becomes \[ x_{n+1}= x_n-\frac{x_n^2-2}{2x_n} = \frac{1}{2}\left(x_n+\frac{2}{x_n}\right), \] from which we obtain \(x_1=1{,}5\), \(x_2\approx 1{,}41667\), \(x_3\approx 1{,}4142157\) and so on.

By experimenting with these values, we find that the number of correct decimal places doubles at each step, and \(x_7\) gives already 100 correct decimal places, if intermediate steps are calculated with enough precision.

### Taylor series

##### Taylor series

If the error term \(E_n(x)\) in Taylor's Formula goes to zero as \(n\) increases, then the limit of the Taylor polynomial is the Taylor series of \(f\) (= Maclaurin series for \(x_0=0\)).

The Taylor series of \(f\) is of the form
\[
\sum_{k=0}^{\infty}\frac{f^{(k)}(x_0)}{k!}(x-x_0)^k
= \lim_{n\to\infty} \sum_{k=0}^{n}\frac{f^{(k)}(x_0)}{k!}(x-x_0)^k .
\]
This is an example of a **power series**.

The Taylor series can be formed as soon as \(f\) has derivatives of all
orders at \(x_0\) and they are substituted into this formula.
There are two problems related to this: *Does the Taylor series converge for all values of \(x\)?*

Answer: Not always; for example, the function \[ f(x)=\frac{1}{1-x} \] has a Maclaurin series (= geometric series) converging only for \(-1 < x < 1\), although the function is differentiable for all \(x\neq 1\): \[ f(x)=\frac{1}{1-x} = 1+x+x^2+x^3+x^4+\dots \]

**Interaction.**Newton's method. Set the starting point \(x_{0}\) and iterate to find the zeros of the function.

If the series converges for some \(x\), then does its sum equal \(f(x)\)? Answer: Not always; for example, the function \[ f(x)=\begin{cases} e^{-1/x^2}, & x\neq 0,\\ 0, & x=0,\\ \end{cases} \] satisfies \(f^{(k)}(0)=0\) for all \(k\in \mathbf{N}\) (elementary but difficult calculation). Thus its Maclaurin series is identically zero and converges to \(f(x)\) only at \(x=0\).

**Conclusion:** Taylor series should be studied carefully using the
error terms. In practice, the series are formed by using some
well known basic series.

##### Examples

\begin{align} \frac{1}{1-x} &= \sum_{k=0}^{\infty} x^k,\ \ |x|< 1 \\ e^x &= \sum_{k=0}^{\infty} \frac{1}{k!}x^k, \ \ x\in \mathbb{R} \\ \sin x &= \sum_{k=0}^{\infty} \frac{(-1)^{k}}{(2k+1)!} x^{2k+1}, \ \ x\in \mathbb{R} \\ \cos x &= \sum_{k=0}^{\infty} \frac{(-1)^{k}}{(2k)!} x^{2k},\ \ x\in \mathbb{R} \\ (1+x)^r &= 1+\sum_{k=1}^{\infty} \frac{r(r-1)(r-2)\dots (r-k+1)}{k!}x^k, |x|<1 \end{align} The last is called the Binomial Series and is valid for all \(r\in \mathbb{R}\). If \(r=n \in \mathbb{N}\), then starting from \(k=n+1\), all the coefficients are zero and in the beginning \[ \binom{n}{k} =\frac{n!}{k!(n-k)!} = \frac{n(n-1)(n-2)\dots (n-k+1)}{k!}. \]

Compare this to the **Binomial Theorem:**
\[
(a+b)^n=\sum_{k=0}^n\binom{n}{k} a^{n-k}b^k
=a^n +na^{n-1}b+\dots +b^n
\]
for \(n\in\mathbb{N}\).

### Power series

##### Definition: Power series

A power series is of the form \[ \sum_{k=0}^{\infty} c_k(x-x_0)^k = \lim_{n\to\infty} \sum_{k=0}^{n}c_k(x-x_0)^k. \] The point \(x_0\) is the centre and the \(c_k\) are the coefficients of the series.

The series *converges* at \(x\) if the above limit is defined.

There are only three essentially different cases:

##### Abel's Theorem.

- The power series converges only for \(x=x_0\) (and then it consists of the constant \(c_0\) only)
- The power series converges for all \(x\in \mathbb{R}\)
- The power series converges on some interval \(]x_0-R,x_0+R[\) (and possibly in one or both of the end points), and diverges for other values of \(x\).

The number \(R\) is the **radius of convergence** of the series. In the first two cases we say that \(R=0\) or \(R=\infty\) respectively.

##### Example

For which values of the variable \(x\) does the power series \[\sum_{k=1}^{\infty} \frac{k}{2^k}x^k\] converge?

We use the ratio test with \(a_k=kx^k/2^k\). Then \[ \left| \frac{a_{k+1}}{a_k} \right| = \left| \frac{(k+1)x^{k+1}/2^{k+1}}{kx^k/2^k} \right| = \frac{k+1}{2k}|x| \to \frac{|x|}{2} \] as \(k\to\infty\). By the ratio test, the series converges for \(|x|/2<1\), and diverges for \(|x|/2>1\). In the border-line cases \(|x|/2= 1\Leftrightarrow x=\pm 2\) the general term of the series does not tend to zero, so the series diverges.

Result: The series converges for \(-2< x< 2\), and diverges otherwise.

##### Definition: Sum function

In the interval \(I\) where the series converges, we can
define a function \(f\colon I\to \mathbb{R}\) by setting
\begin{equation}
\label{summafunktio}
f(x) = \sum_{k=0}^{\infty} c_k(x-x_0)^k, \tag{1}
\end{equation}
which is called the **sum function** of the power series.

The sum function \(f\) is continuous and differentiable on \(]x_0-R,x_0+R[\).
Moreover, the derivative \(f'(x)\) can be calculated by differentiating the sum function
term by term:
\[
f'(x)=\sum_{k=1}^{\infty}kc_k(x-x_0)^{k-1}.
\]
**Note.** The constant term \(c_0\) disappears and the series starts with \(k=1\).
The differentiated series converges in the same interval
\(x\in \, ]x_0-R,x_0+R[\); this may sound a bit surprising because of the
extra coefficient \(k\).

##### Example

Find the sum function of the power series \(1+2x+3x^2+4x^3+\dots\)

This series is obtained by differentiating termwise the geometric series (with \(q=x\)). Therefore, \begin{align} 1+2x+3x^2+4x^3+\dots &= D(1+x+x^2+x^3+x^4+\dots ) \\ &= \frac{d}{dx}\left( \frac{1}{1-x}\right) = \frac{1}{(1-x)^2}. \end{align} Multiplying with \(x\) we obtain \[ \sum_{k=1}^{\infty}kx^{k} = x+2x^2+3x^3+4x^4+\dots = \frac{x}{(1-x)^2}, \] which is valid for \(|x|<1\).

In the case \([a,b]\subset\ ]x_0-R,x_0+R[\) we can also integrate the sum function termwise: \[ \int_a^b f(x)\, dx = \sum_{k=0}^{\infty}c_k\int_a^b (x-x_0)^k\, dx. \] Often the definite integral can be extended up to the end points of the interval of convergence, but this is not always the case.

##### Example

Calculate the sum of the alternating harmonic series.

Let us first substitute \(q=-x\) to the geometric series. This yields
\[
1-x+x^2-x^3+x^4-\dots =\frac{1}{1-(-x)} = \frac{1}{1+x}.
\]
By integrating both sides from \(x=0\) to \(x=1\) we obtain
\[
1-\frac{1}{2}+\frac{1}{3}-\frac{1}{4}+\dots =\int_0^1\frac{1}{1+x} =\ln 2.
\]
**Note.** Extending the limit of integration all the way up to \(x=1\) should be justified
more rigorously here. We shall return to integration later on the course.