Explaining The Theory Of Taylor Expansion

This article is available in: 日本語

Introduction
Why can it be expressed in polynomial form?
Find the value of the coefficient

Introduction

The beauty of Taylor expansion is that functions can be expressed in polynomial form.

When I first learned it, I was not surprised why a function \(f(x)\) could be expressed as a polynomial. This time, I would like to understand the reason loosely.

Taylor expansion is expressed as follows

Taylor expansion

If the function \(f(x)\) is infinitely differentiable on the interval containing \(x = a\), then \(f(x)\) is

\begin{eqnarray*}
f(x) &=& f(a) + f^{\prime}(a)\, (x\, – a) + \frac{1}{2!}f^{\prime\prime}(a)\, (x-a)^2 + \cdots + \frac{1}{n!}f^{(n)}(a)\, (x-a)^n + \cdots \\
&=& \sum_{k = 0}^{\infty} \frac{1}{k!}f^{(k)}(a)\, (x-a)^k
\end{eqnarray*}

This is called “Taylor expansion around \(a\)”.

or it may also be expressed as

Taylor expansion

\begin{eqnarray*}
f(x + a) &=& f(x) + f^{\prime}(x)\, a + \frac{1}{2!}f^{\prime\prime}(x)\, a^2 + \cdots + \frac{1}{n!}f^{(n)}(x)\, a^n + \cdots \\
&=& \sum_{k = 0}^{\infty} \frac{1}{k!}f^{(k)}(x)\, a^k
\end{eqnarray*}

Let us now derive the first table expression.

Why can it be expressed in polynomial form?

Suppose the function \(f(x)\) has the form shown in the figure above.

Out of the blue, consider a line passing through the points \( (a,\, f(a)) \) and \( (a + h,\, f(a + h)) \) (blue line in Figure 1). The slope of this line is

\begin{eqnarray}
\frac{f(a + h)\, – f(a)}{h}\tag{1}
\end{eqnarray}

Here, there exists a value of \(x\) such that the slope of the tangent line to the function \(f(x), f^{\prime}(x)\), is exactly equal to (1) somewhere between \(a\) and \(a + h\) (mean value theorem). So, if the value of its(x) is \(a + \theta_1 h\ (0 < \theta_1 < 1)\), then

\begin{eqnarray*}
f^{\prime}(a + \theta_1 h) = \frac{f(a + h)\, – f(a)}{h}
\end{eqnarray*}

therefore,

\begin{eqnarray}
f(a + h) = f(a) + f^{\prime}(a + \theta_1 h)\, h \tag{2}
\end{eqnarray}

However, since our goal is to express the function \(f\) in polynomial form, we want to work out the above expression \(f^{\prime}(a + \theta_1 h)\).

So next, consider the graph of the function\(f^{\prime}(x)\). Suppose it takes the form shown in the figure below.

As before, consider a line passing through the points \((a,\, f^{\prime}(a)) \) and \((a + \theta_1 h,\, f^{\prime}(a +\theta_1 h)) \) (blue line in Figure 2). The slope of this line is

\begin{eqnarray*}
\frac{f^{\prime}(a + \theta_1 h)\, – f^{\prime}(a)}{\theta_1 h}\tag{3}
\end{eqnarray*}

Here, there exists a value of \(x\) such that the slope of the tangent line to the function \(f^{\prime}(x)\) is exactly equal to (3), somewhere between \(a\) and \(a + \theta_1 h\). So, if the value of its \(x\) is \(a + \theta_1 \theta_2 h\ (0 < \theta_2 < 1)\)

\begin{eqnarray*}
f^{\prime\prime}(a + \theta_1 \theta_2 h) = \frac{f^{\prime}(a + h)\, – f^{\prime}(a)}{\theta_1 h}
\end{eqnarray*}

therefore,

\begin{eqnarray*}
f^{\prime}(a + h) = f^{\prime}(a) + \theta_1 f^{\prime\prime}(a + \theta_1 \theta_2 h)\, h
\end{eqnarray*}

Substituting this expression into (2), we obtain

\begin{eqnarray*}
f(a + h) = f(a) + f^{\prime}(a)\, h + \theta_1 f^{\prime\prime}(a + \theta_1 \theta_2 h)\, h^2
\end{eqnarray*}

Repeating these operations many times, it seems that the function \(f\) can be expressed as a polynomial in \(h\). If the function \(f(x)\) is infinitely differentiable around \(a\), then

\begin{eqnarray*}
f(a + h) &=& f(a) + f^{\prime}(a)\, h + \theta_1 f^{\prime\prime}(a)\, h^2 + \theta_1 \theta_2 f^{(3)}(a + \theta_1 \theta_2 \theta_3 h)\, h^3 \\
\\
f(a + h) &=& f(a) + f^{\prime}(a)\, h + \theta_1 f^{\prime\prime}(a)\, h^2 + \theta_1 \theta_2 f^{(3)}(a)\, h^3 + \theta_1 \theta_2 \theta_3 f^{(4)}(a + \theta_1 \theta_2 \theta_3 \theta_4 h)\, h^4 \\
\\
&\vdots & \\
\\
f(a + h) &=& f(a) + f^{\prime}(a)\, h + \theta_1 f^{\prime\prime}(a)\, h^2 + \theta_1 \theta_2 f^{(3)}(a)\, h^3 + \cdots + \theta_1 \theta_2 \cdots \theta_{n-1} f^{(n)}(a)\, h^n + \cdots
\end{eqnarray*}

The coefficients \(\theta_1 \theta_2 \cdots \) for each term are \(c\) and \(h = x-a\) and again

\begin{eqnarray*}
f(x) = f(a) + c_1f^{\prime}(a)\, (x – a) + c_2f^{\prime\prime}(a)\, (x-a)^2 &+& c_3f^{(3)}(a)\, (x-a)^3 \\ &+& \cdots + c_n f^{(n)}(a)\, (x-a)^n + \cdots \tag{☆}
\end{eqnarray*}

The result could be transformed as follows. But what is the value of the coefficient \(c_i\) ?

Find the value of the coefficient

To find the value of the coefficient \(c_i\), we differentiate equation (☆) by \(x\) in the first order. Then, we obtain

\begin{eqnarray*}
f^{\prime}(x) = c_1f^{\prime}(a) + 2\, c_2f^{\prime\prime}(a)\, (x-a) + 3\, c_3f^{(3)}(a)\, (x-a)^2 + \cdots + n\, c_n f^{(n)}(a)\, (x-a)^{n-1} + \cdots
\end{eqnarray*}

Substituting \(x = a\), we see that the right-hand side is all zeros except for the first item, so

\begin{eqnarray*}
f^{\prime}(a) &=& c_1 f^{\prime}(a) \\
∴ c_1 &=& 1
\end{eqnarray*}

Similarly, second-order differentiation of equation (☆) with \(x\) yields

\begin{eqnarray*}
f^{\prime\prime}(x) = 2\cdot 1\, c_2f^{\prime\prime}(a) + 3\cdot 2\, c_3f^{(3)}(a)\, (x-a) + \cdots + n \cdot (n-1)\, c_n f^{(n)}(a)\, (x-a)^{n-2} + \cdots
\end{eqnarray*}

Substituting \(x = a\), we obtain

\begin{eqnarray*}
f^{\prime\prime}(a) &=& 2!\, c_2 f^{\prime\prime}(a) \\
∴ c_1 &=& \frac{1}{2!}
\end{eqnarray*}

Differentiating equation (☆) by \(x\), we obtain

\begin{eqnarray*}
f^{(3)}(x) = 3 \cdot 2\cdot 1\, c_3f^{(3)}(a) + \cdots + n \cdot (n-1) \cdot (n-2)\, c_n f^{(n)}(a)\, (x-a)^{n-3} + \cdots
\end{eqnarray*}

Substituting \(x = a\), we obtain

\begin{eqnarray*}
f^{\prime\prime}(a) &=& 3!\, c_3 f^{\prime\prime}(a) \\
∴ c_3 &=& \frac{1}{3!}
\end{eqnarray*}

Therefore, the coefficient \(c_n\) of the ith \(n\) item can be obtained by differentiating equation (☆) by \(x\) and substituting \(x = a\) for \(n\).

\begin{eqnarray*}
c_n = \frac{1}{n!}
\end{eqnarray*}

Therefore, equation (☆) is

\begin{eqnarray*}
f(x) = f(a) + f^{\prime}(a)\, (x – a) + \frac{1}{2!}f^{\prime\prime}(a)\, (x-a)^2 + \cdots + \frac{1}{n!}f^{(n)}(a)\, (x-a)^n + \cdots
\end{eqnarray*}

We were able to derive a tabular expression for the Taylor expansion.

Click here ▼ for an example of Taylor expansion.