User:IssaRice/Chain rule proofs

Using Newton's approximation

Main idea

The main idea of using Newton's approximation to prove the chain rule is that since f is differentiable at $x_0$ we have the approximation $f(x)\approx f(x_0)+f^{\prime }(x_0)(x-x_0)$ when $x$ is near $x_0$. Similarly since g is differentiable at $f(x_0)$ we have the approximation $g(y)\approx g(f(x_0))+g^{\prime }(f(x_0))(y-f(x_0))$ when $y$ is near $f(x_0)$. Since f is differentiable at $x_0$, it is continuous there also, so we know that $f(x)$ is near $f(x_0)$ whenever $x$ is near $x_0$. This allows us to substitute $f(x)$ into $y$ whenever $x$ is near $x_0$. So we get

$$\begin{array}{rr}g(f(x))& \approx g(f(x_0))+g\text{'}(f(x_0))(f(x)-f(x_0))\\ & \approx g(f(x_0))+g\text{'}(f(x_0))(f\text{'}(x_0)(x-x_0))\end{array}$$

Thus we get $g\circ f(x)\approx g\circ f(x_0)+g^{\prime }(f(x_0))f^{\prime }(x_0)(x-x_0)$, which is what the chain rule says.

Proof

We want to show $g\circ f$ is differentiable at $x_0$ with derivative $L:=g^{\prime }(f(x_0))f^{\prime }(x_0)$. By Newton's approximation, this is equivalent to showing that for every $ϵ>0$ there exists $\delta >0$ such that

$$|g\circ f(x)-(g\circ f(x_0)+L(x-x_0))|\le ϵ|x-x_0|$$

whenever $|x-x_0|\le \delta$. So let $ϵ>0$.

Now we do some algebraic manipulation. Write

$$g(y)=g(y_0)+g^{\prime }(y_0)(y-y_0)+E_g(y,y_0)$$

where $E_g(y,y_0):=g(y)-(g(y_0)+g^{\prime }(y_0)(y-y_0))$. This holds for every $y\in Y$. Since $f(x)\in Y$ we thus have

$$g(f(x))=g(f(x_0))+g^{\prime }(f(x_0))(f(x)-f(x_0))+E_g(f(x),f(x_0))$$

Similarly write

$$f(x)=f(x_0)+f^{\prime }(x_0)(x-x_0)+E_f(x,x_0)$$

where $E_f(x,x_0):=f(x)-(f(x_0)+f^{\prime }(x_0)(x-x_0))$.

Substituting the expression for $f(x)$ in the expression for $g(f(x))$ we get

$$\begin{array}{rr}g(f(x))& =g(f(x_0))+g\text{'}(f(x_0))(f\text{'}(x_0)(x-x_0)+E_f(x,x_0))+E_g(f(x),f(x_0))\\ & =g(f(x_0))+g\text{'}(f(x_0))f\text{'}(x_0)(x-x_0)+g\text{'}(f(x_0))E_f(x,x_0)+E_g(f(x),f(x_0))\end{array}$$

we can rewrite this as $g\circ f(x)-(g\circ f(x_0)+L(x-x_0))=g^{\prime }(f(x_0))E_f(x,x_0)+E_g(f(x),f(x_0))$

Thus our goal now is to show $|g^{\prime }(f(x_0))E_f(x,x_0)+E_g(f(x),f(x_0))|\le ϵ|x-x_0|$.

But by the triangle inequality it suffices to show $|g^{\prime }(f(x_0))E_f(x,x_0)|+|E_g(f(x),f(x_0))|\le ϵ|x-x_0|$.

$|g^{\prime }(f(x_0))E_f(x,x_0)|\le |g^{\prime }(f(x_0))|{ϵ}_1|x-x_0|$ where we are free to choose ${ϵ}_1$.

To get the bound for $|E_g(f(x),f(x_0))|$ (using Newton's approximation), we need to make sure $|f(x)-f(x_0)|$ is small. But by continuity of $f$ at $x_0$ we can do this.

$$\begin{array}{rr}|E_g(f(x),f(x_0))|& \le {ϵ}_2|f(x)-f(x_0)|\\ & ={ϵ}_2|f\text{'}(x_0)(x-x_0)+E_f(x,x_0)|\\ & \le {ϵ}_2|f\text{'}(x_0)||x-x_0|+{ϵ}_2{ϵ}_3|x-x_0|\end{array}$$

old proof

Since $g$ is differentiable at $y_0$, we know $g^{\prime }(y_0)$ is a real number, and we can write

$$g(y)=g(y_0)+g^{\prime }(y_0)(y-y_0)+[g(y)-(g(y_0)+g^{\prime }(y_0)(y-y_0))]$$

(there is no magic: the terms just cancel out)

If we define $E_g(y,y_0):=g(y)-(g(y_0)+g^{\prime }(y_0)(y-y_0))$ we can write

$$g(y)=g(y_0)+g^{\prime }(f(x_0))(y-y_0)+E_g(y,y_0)$$

Newton's approximation says that $|E_g(y,y_0)|\le ϵ|y-y_0|$ as long as $|y-y_0|\le \delta$.

Since $f$ is differentiable at $x_0$, we know that it must be continuous at $x_0$. This means we can keep $|f(x)-y_0|\le \delta$ as long as we keep $|x-x_0|\le {\delta }^{\prime }$.

Since $f(x)\in Y$ and $|f(x)-y_0|\le \delta$, this means we can substitute $y=f(x)$ and get

$$g(f(x))=g(y_0)+g^{\prime }(f(x_0))(f(x)-y_0)+E_g(f(x),y_0)$$

Now we use the differentiability of $f$. We can write

$$f(x)=f(x_0)+f^{\prime }(x_0)(x-x_0)+[f(x)-(f(x_0)+f^{\prime }(x_0)(x-x_0))]$$

Again, we can define $E_f(x,x_0):=f(x)-(f(x_0)+f^{\prime }(x_0)(x-x_0))$ and write this as

$$f(x)=f(x_0)+f^{\prime }(x_0)(x-x_0)+E_f(x,x_0)$$

Now we can substitute this into the expression for $g(f(x))$ to get

$$g(f(x))=g(y_0)+g^{\prime }(f(x_0))(f^{\prime }(x_0)(x-x_0)+E_f(x,x_0))+E_g(f(x),f(x_0))$$

where we have canceled out two terms using $f(x_0)=y_0$.

Thus we have

$$g(f(x))=g(y_0)+g^{\prime }(f(x_0))f^{\prime }(x_0)(x-x_0)+[g^{\prime }(f(x_0))E_f(x,x_0)+E_g(f(x),f(x_0))]$$

We can write this as

$$(g\circ f)(x)-((g\circ f)(x_0)+L(x-x_0))=[g^{\prime }(f(x_0))E_f(x,x_0)+E_g(f(x),f(x_0))]$$

where $L:=g^{\prime }(f(x_0))f^{\prime }(x_0)$. Now the left hand side looks like the expression in Newton's approximation. This means to show $g\circ f$ is differentiable at $x_0$, we just need to show that $|g^{\prime }(f(x_0))E_f(x,x_0)+E_g(f(x),f(x_0))|\le ϵ|x-x_0|$.

The stuff in square brackets is our "error term" for $g\circ f$. Now we just need to make sure it is small, even after dividing by $|x-x_0|$.

But f is differentiable at $x_0$, so by Newton's approximation,

$$|g^{\prime }(f(x_0))E_f(x,x_0)|\le |g^{\prime }(f(x_0))|{ϵ}_1|x-x_0|$$

we also have

$$|E_g(f(x),f(x_0))|\le {ϵ}_2|f(x)-f(x_0)|={ϵ}_2|f^{\prime }(x_0)(x-x_0)+E_f(x,x_0)|$$

We can bound this from above using the triangle inequality:

$$\begin{array}{rr}|E_g(f(x),f(x_0))|& \le {ϵ}_2|f\text{'}(x_0)(x-x_0)|+{ϵ}_2|E_f(x,x_0)|\\ & \le {ϵ}_2|f\text{'}(x_0)||x-x_0|+{ϵ}_2{ϵ}_1|x-x_0|\end{array}$$

Now we can just choose ${ϵ}_1,{ϵ}_2$ small enough.