# Explaining The Theory Of Taylor Expansion

## Introduction

The beauty of Taylor expansion is that functions can be expressed in polynomial form.

When I first learned it, I was not surprised why a function $f(x)$ could be expressed as a polynomial. This time, I would like to understand the reason loosely.

Taylor expansion is expressed as follows

Taylor expansion

If the function $f(x)$ is infinitely differentiable on the interval containing $x = a$, then $f(x)$ is

\begin{eqnarray*}
f(x) &=& f(a) + f^{\prime}(a)\, (x\, – a) + \frac{1}{2!}f^{\prime\prime}(a)\, (x-a)^2 + \cdots + \frac{1}{n!}f^{(n)}(a)\, (x-a)^n + \cdots \\
&=& \sum_{k = 0}^{\infty} \frac{1}{k!}f^{(k)}(a)\, (x-a)^k
\end{eqnarray*}

This is called “Taylor expansion around $a$”.

or it may also be expressed as

Taylor expansion
\begin{eqnarray*}
f(x + a) &=& f(x) + f^{\prime}(x)\, a + \frac{1}{2!}f^{\prime\prime}(x)\, a^2 + \cdots + \frac{1}{n!}f^{(n)}(x)\, a^n + \cdots \\
&=& \sum_{k = 0}^{\infty} \frac{1}{k!}f^{(k)}(x)\, a^k
\end{eqnarray*}

Let us now derive the first table expression.

## Why can it be expressed in polynomial form?

Suppose the function $f(x)$ has the form shown in the figure above.

Out of the blue, consider a line passing through the points $(a,\, f(a))$ and $(a + h,\, f(a + h))$ (blue line in Figure 1). The slope of this line is

\begin{eqnarray}
\frac{f(a + h)\, – f(a)}{h}\tag{1}
\end{eqnarray}
Here, there exists a value of $x$ such that the slope of the tangent line to the function $f(x), f^{\prime}(x)$, is exactly equal to (1) somewhere between $a$ and $a + h$ (mean value theorem). So, if the value of its(x) is $a + \theta_1 h\ (0 < \theta_1 < 1)$, then
\begin{eqnarray*}
f^{\prime}(a + \theta_1 h) = \frac{f(a + h)\, – f(a)}{h}
\end{eqnarray*}
therefore,
\begin{eqnarray}
f(a + h) = f(a) + f^{\prime}(a + \theta_1 h)\, h \tag{2}
\end{eqnarray}
However, since our goal is to express the function $f$ in polynomial form, we want to work out the above expression $f^{\prime}(a + \theta_1 h)$.

So next, consider the graph of the function$f^{\prime}(x)$. Suppose it takes the form shown in the figure below.

As before, consider a line passing through the points $(a,\, f^{\prime}(a))$ and $(a + \theta_1 h,\, f^{\prime}(a +\theta_1 h))$ (blue line in Figure 2). The slope of this line is

\begin{eqnarray*}
\frac{f^{\prime}(a + \theta_1 h)\, – f^{\prime}(a)}{\theta_1 h}\tag{3}
\end{eqnarray*}
Here, there exists a value of $x$ such that the slope of the tangent line to the function $f^{\prime}(x)$ is exactly equal to (3), somewhere between $a$ and $a + \theta_1 h$. So, if the value of its $x$ is $a + \theta_1 \theta_2 h\ (0 < \theta_2 < 1)$
\begin{eqnarray*}
f^{\prime\prime}(a + \theta_1 \theta_2 h) = \frac{f^{\prime}(a + h)\, – f^{\prime}(a)}{\theta_1 h}
\end{eqnarray*}
therefore,
\begin{eqnarray*}
f^{\prime}(a + h) = f^{\prime}(a) + \theta_1 f^{\prime\prime}(a + \theta_1 \theta_2 h)\, h
\end{eqnarray*}

Substituting this expression into (2), we obtain

\begin{eqnarray*}
f(a + h) = f(a) + f^{\prime}(a)\, h + \theta_1 f^{\prime\prime}(a + \theta_1 \theta_2 h)\, h^2
\end{eqnarray*}

Repeating these operations many times, it seems that the function $f$ can be expressed as a polynomial in $h$. If the function $f(x)$ is infinitely differentiable around $a$, then

\begin{eqnarray*}
f(a + h) &=& f(a) + f^{\prime}(a)\, h + \theta_1 f^{\prime\prime}(a)\, h^2 + \theta_1 \theta_2 f^{(3)}(a + \theta_1 \theta_2 \theta_3 h)\, h^3 \\
\\
f(a + h) &=& f(a) + f^{\prime}(a)\, h + \theta_1 f^{\prime\prime}(a)\, h^2 + \theta_1 \theta_2 f^{(3)}(a)\, h^3 + \theta_1 \theta_2 \theta_3 f^{(4)}(a + \theta_1 \theta_2 \theta_3 \theta_4 h)\, h^4 \\
\\
&\vdots & \\
\\
f(a + h) &=& f(a) + f^{\prime}(a)\, h + \theta_1 f^{\prime\prime}(a)\, h^2 + \theta_1 \theta_2 f^{(3)}(a)\, h^3 + \cdots + \theta_1 \theta_2 \cdots \theta_{n-1} f^{(n)}(a)\, h^n + \cdots
\end{eqnarray*}

The coefficients $\theta_1 \theta_2 \cdots$ for each term are $c$ and $h = x-a$ and again

\begin{eqnarray*}
f(x) = f(a) + c_1f^{\prime}(a)\, (x – a) + c_2f^{\prime\prime}(a)\, (x-a)^2 &+& c_3f^{(3)}(a)\, (x-a)^3 \\ &+& \cdots + c_n f^{(n)}(a)\, (x-a)^n + \cdots \tag{☆}
\end{eqnarray*}
The result could be transformed as follows. But what is the value of the coefficient $c_i$ ?

## Find the value of the coefficient

To find the value of the coefficient $c_i$, we differentiate equation (☆) by $x$ in the first order. Then, we obtain

\begin{eqnarray*}
f^{\prime}(x) = c_1f^{\prime}(a) + 2\, c_2f^{\prime\prime}(a)\, (x-a) + 3\, c_3f^{(3)}(a)\, (x-a)^2 + \cdots + n\, c_n f^{(n)}(a)\, (x-a)^{n-1} + \cdots
\end{eqnarray*}
Substituting $x = a$, we see that the right-hand side is all zeros except for the first item, so
\begin{eqnarray*}
f^{\prime}(a) &=& c_1 f^{\prime}(a) \\
∴ c_1 &=& 1
\end{eqnarray*}

Similarly, second-order differentiation of equation (☆) with $x$ yields

\begin{eqnarray*}
f^{\prime\prime}(x) = 2\cdot 1\, c_2f^{\prime\prime}(a) + 3\cdot 2\, c_3f^{(3)}(a)\, (x-a) + \cdots + n \cdot (n-1)\, c_n f^{(n)}(a)\, (x-a)^{n-2} + \cdots
\end{eqnarray*}
Substituting $x = a$, we obtain
\begin{eqnarray*}
f^{\prime\prime}(a) &=& 2!\, c_2 f^{\prime\prime}(a) \\
∴ c_1 &=& \frac{1}{2!}
\end{eqnarray*}

Differentiating equation (☆) by $x$, we obtain

\begin{eqnarray*}
f^{(3)}(x) = 3 \cdot 2\cdot 1\, c_3f^{(3)}(a) + \cdots + n \cdot (n-1) \cdot (n-2)\, c_n f^{(n)}(a)\, (x-a)^{n-3} + \cdots
\end{eqnarray*}
Substituting $x = a$, we obtain
\begin{eqnarray*}
f^{\prime\prime}(a) &=& 3!\, c_3 f^{\prime\prime}(a) \\
∴ c_3 &=& \frac{1}{3!}
\end{eqnarray*}

Therefore, the coefficient $c_n$ of the ith $n$ item can be obtained by differentiating equation (☆) by $x$ and substituting $x = a$ for $n$.

\begin{eqnarray*}
c_n = \frac{1}{n!}
\end{eqnarray*}

Therefore, equation (☆) is

\begin{eqnarray*}
f(x) = f(a) + f^{\prime}(a)\, (x – a) + \frac{1}{2!}f^{\prime\prime}(a)\, (x-a)^2 + \cdots + \frac{1}{n!}f^{(n)}(a)\, (x-a)^n + \cdots
\end{eqnarray*}

We were able to derive a tabular expression for the Taylor expansion. 