User:IssaRice/Chain rule proofs

Using Newton's approximation

Main idea

The main idea of using Newton's approximation to prove the chain rule is that since f is differentiable at $x_0$ we have the approximation $f(x)\approx f(x_0)+f^{\prime }(x_0)(x-x_0)$ when $x$ is near $x_0$. Similarly since g is differentiable at $f(x_0)$ we have the approximation $g(y)\approx g(f(x_0))+g^{\prime }(f(x_0))(y-f(x_0))$ when $y$ is near $f(x_0)$. Since f is differentiable at $x_0$, it is continuous there also, so we know that $f(x)$ is near $f(x_0)$ whenever $x$ is near $x_0$. This allows us to substitute $f(x)$ into $y$ whenever $x$ is near $x_0$. So we get

$$\begin{array}{rr}g(f(x))& \approx g(f(x_0))+g\text{'}(f(x_0))(f(x)-f(x_0))\\ & \approx g(f(x_0))+g\text{'}(f(x_0))(f\text{'}(x_0)(x-x_0))\end{array}$$

Thus we get $g\circ f(x)\approx g\circ f(x_0)+g^{\prime }(f(x_0))f^{\prime }(x_0)(x-x_0)$, which is what the chain rule says.

Proof

We want to show $g\circ f$ is differentiable at $x_0$ with derivative $L:=g^{\prime }(f(x_0))f^{\prime }(x_0)$. By Newton's approximation, this is equivalent to showing that for every $ϵ>0$ there exists $\delta >0$ such that

$$|g\circ f(x)-(g\circ f(x_0)+L(x-x_0))|\le ϵ|x-x_0|$$

whenever $|x-x_0|\le \delta$. So let $ϵ>0$.

Now we do some algebraic manipulation. Write

$$g(y)=g(y_0)+g^{\prime }(y_0)(y-y_0)+E_g(y,y_0)$$

where $E_g(y,y_0):=g(y)-(g(y_0)+g^{\prime }(y_0)(y-y_0))$. This holds for every $y\in Y$. Since $f(x)\in Y$ we thus have

$$g(f(x))=g(f(x_0))+g^{\prime }(f(x_0))(f(x)-f(x_0))+E_g(f(x),f(x_0))$$

Similarly write

$$f(x)=f(x_0)+f^{\prime }(x_0)(x-x_0)+E_f(x,x_0)$$

where $E_f(x,x_0):=f(x)-(f(x_0)+f^{\prime }(x_0)(x-x_0))$.

Substituting the expression for $f(x)$ in the expression for $g(f(x))$ we get

$$\begin{array}{rr}g(f(x))& =g(f(x_0))+g\text{'}(f(x_0))(f\text{'}(x_0)(x-x_0)+E_f(x,x_0))+E_g(f(x),f(x_0))\\ & =g(f(x_0))+g\text{'}(f(x_0))f\text{'}(x_0)(x-x_0)+g\text{'}(f(x_0))E_f(x,x_0)+E_g(f(x),f(x_0))\end{array}$$

we can rewrite this as $g\circ f(x)-(g\circ f(x_0)+L(x-x_0))=g^{\prime }(f(x_0))E_f(x,x_0)+E_g(f(x),f(x_0))$

Thus our goal now is to show $|g^{\prime }(f(x_0))E_f(x,x_0)+E_g(f(x),f(x_0))|\le ϵ|x-x_0|$.

But by the triangle inequality it suffices to show $|g^{\prime }(f(x_0))E_f(x,x_0)|+|E_g(f(x),f(x_0))|\le ϵ|x-x_0|$.

$|g^{\prime }(f(x_0))E_f(x,x_0)|\le |g^{\prime }(f(x_0))|{ϵ}_1|x-x_0|$ where we are free to choose ${ϵ}_1$.

To get the bound for $|E_g(f(x),f(x_0))|$ (using Newton's approximation), we need to make sure $|f(x)-f(x_0)|$ is small. But by continuity of $f$ at $x_0$ we can do this.

$$\begin{array}{rr}|E_g(f(x),f(x_0))|& \le {ϵ}_2|f(x)-f(x_0)|\\ & ={ϵ}_2|f\text{'}(x_0)(x-x_0)+E_f(x,x_0)|\\ & \le {ϵ}_2|f\text{'}(x_0)||x-x_0|+{ϵ}_2{ϵ}_3|x-x_0|\\ & =({ϵ}_2|f\text{'}(x_0)|+{ϵ}_2{ϵ}_3)|x-x_0|\end{array}$$

where again we are free to choose ${ϵ}_2,{ϵ}_3$.

TODO: can we do this same proof but without using the error term notation?

TODO: somehow Folland does this without explicitly using continuity of f; i need to understand if he's using it implicitly somehow or he's actually proving it when bounding $\left|\mathbf{h}\right|$ using $\left|u\right|$

old proof

Since $g$ is differentiable at $y_0$, we know $g^{\prime }(y_0)$ is a real number, and we can write

$$g(y)=g(y_0)+g^{\prime }(y_0)(y-y_0)+[g(y)-(g(y_0)+g^{\prime }(y_0)(y-y_0))]$$

(there is no magic: the terms just cancel out)

If we define $E_g(y,y_0):=g(y)-(g(y_0)+g^{\prime }(y_0)(y-y_0))$ we can write

$$g(y)=g(y_0)+g^{\prime }(f(x_0))(y-y_0)+E_g(y,y_0)$$

Newton's approximation says that $|E_g(y,y_0)|\le ϵ|y-y_0|$ as long as $|y-y_0|\le \delta$.

Since $f$ is differentiable at $x_0$, we know that it must be continuous at $x_0$. This means we can keep $|f(x)-y_0|\le \delta$ as long as we keep $|x-x_0|\le {\delta }^{\prime }$.

Since $f(x)\in Y$ and $|f(x)-y_0|\le \delta$, this means we can substitute $y=f(x)$ and get

$$g(f(x))=g(y_0)+g^{\prime }(f(x_0))(f(x)-y_0)+E_g(f(x),y_0)$$

Now we use the differentiability of $f$. We can write

$$f(x)=f(x_0)+f^{\prime }(x_0)(x-x_0)+[f(x)-(f(x_0)+f^{\prime }(x_0)(x-x_0))]$$

Again, we can define $E_f(x,x_0):=f(x)-(f(x_0)+f^{\prime }(x_0)(x-x_0))$ and write this as

$$f(x)=f(x_0)+f^{\prime }(x_0)(x-x_0)+E_f(x,x_0)$$

Now we can substitute this into the expression for $g(f(x))$ to get

$$g(f(x))=g(y_0)+g^{\prime }(f(x_0))(f^{\prime }(x_0)(x-x_0)+E_f(x,x_0))+E_g(f(x),f(x_0))$$

where we have canceled out two terms using $f(x_0)=y_0$.

Thus we have

$$g(f(x))=g(y_0)+g^{\prime }(f(x_0))f^{\prime }(x_0)(x-x_0)+[g^{\prime }(f(x_0))E_f(x,x_0)+E_g(f(x),f(x_0))]$$

We can write this as

$$(g\circ f)(x)-((g\circ f)(x_0)+L(x-x_0))=[g^{\prime }(f(x_0))E_f(x,x_0)+E_g(f(x),f(x_0))]$$

where $L:=g^{\prime }(f(x_0))f^{\prime }(x_0)$. Now the left hand side looks like the expression in Newton's approximation. This means to show $g\circ f$ is differentiable at $x_0$, we just need to show that $|g^{\prime }(f(x_0))E_f(x,x_0)+E_g(f(x),f(x_0))|\le ϵ|x-x_0|$.

The stuff in square brackets is our "error term" for $g\circ f$. Now we just need to make sure it is small, even after dividing by $|x-x_0|$.

But f is differentiable at $x_0$, so by Newton's approximation,

$$|g^{\prime }(f(x_0))E_f(x,x_0)|\le |g^{\prime }(f(x_0))|{ϵ}_1|x-x_0|$$

we also have

$$|E_g(f(x),f(x_0))|\le {ϵ}_2|f(x)-f(x_0)|={ϵ}_2|f^{\prime }(x_0)(x-x_0)+E_f(x,x_0)|$$

We can bound this from above using the triangle inequality:

$$\begin{array}{rr}|E_g(f(x),f(x_0))|& \le {ϵ}_2|f\text{'}(x_0)(x-x_0)|+{ϵ}_2|E_f(x,x_0)|\\ & \le {ϵ}_2|f\text{'}(x_0)||x-x_0|+{ϵ}_2{ϵ}_1|x-x_0|\end{array}$$

Now we can just choose ${ϵ}_1,{ϵ}_2$ small enough.

Limits of sequences

Basic idea:

Let $(x_n{)}_{n=1}^{\mathrm{\infty }}$ be a sequence that converges to $x_0$.

$$\frac{g(f(x_n))-g(f(x_0))}{x_n-x_0}=\frac{g(f(x_n))-g(f(x_0))}{f(x_n)-f(x_0)}\cdot \frac{f(x_n)-f(x_0)}{x_n-x_0}$$

Now use the limit laws to conclude that the limit is $g^{\prime }(f(x_0))\cdot f^{\prime }(x_0)$.

The problem is that $f(x_n)-f(x_0)$ can be zero even when $x_n\ne x_0$.

So we can instead define

$$\varphi (y):=\{\begin{array}{cc}& y\ne f(x_0)\\ g\text{'}(f(x_0))& y=f(x_0)\end{array}$$

The idea is that we want to say $\frac{g(f(x_n))-g(f(x_0))}{f(x_n)-f(x_0)}$ is going to $g^{\prime }(f(x_0))$, so we just define it at the undefined points to already be at that limit.

Now we have

$$\frac{g(f(x_n))-g(f(x_0))}{x_n-x_0}=\varphi (f(x_n))\cdot \frac{f(x_n)-f(x_0)}{x_n-x_0}$$

for all $x_n$.

Differentiability of $g$ at $f(x_0)$ says that if $(y_n{)}_{n=1}^{\mathrm{\infty }}$ is a sequence in $Y\setminus \{y_0\}$, then $\frac{g(y_n)-g(f(x_0))}{y_n-f(x_0)}\to g^{\prime }(f(x_0))$ as $y_n\to f(x_0)$. What if $(y_n{)}_{n=1}^{\mathrm{\infty }}$ is instead a sequence in $Y$? Then we can say $\varphi (y_n)\to g^{\prime }(f(x_0))$ as $n\to \mathrm{\infty }$.

But differentiability of $f$ at $x_0$ implies continuity of $f$ at $x_0$, so this means that $f(x_n)\to f(x_0)$ as $x_n\to x_0$. So we can use $(f(x_n){)}_{n=1}^{\mathrm{\infty }}$ as our sequence to conclude that as $x_n\to x_0$ we have $\varphi (f(x_n))\to g^{\prime }(f(x_0))$.

Now by the limit laws

$$\begin{array}{rr}\underset{n\to \mathrm{\infty }}{lim}\frac{g(f(x_n))-g(f(x_0))}{x_n-x_0}& =(\underset{n\to \mathrm{\infty }}{lim}\varphi (f(x_n)))(\underset{n\to \mathrm{\infty }}{lim}\frac{f(x_n)-f(x_0)}{x_n-x_0})\\ & =g\text{'}(f(x_0))f\text{'}(x_0)\end{array}$$

Since the sequence $(x_n{)}_{n=1}^{\mathrm{\infty }}$ was arbitrary, we can conclude that $\underset{x\to x_0;x\in X\setminus \{x_0\}}{lim}\frac{g(f(x))-g(f(x_0))}{x-x_0}=g^{\prime }(f(x_0))f^{\prime }(x_0)$.

$\frac{g(f(x_n))-g(f(x_0))}{f(x_n)-f(x_0)}\to g^{\prime }(f(x_0))$

TODO: Tao says that division by zero occurs when $f^{\prime }(x_0)=0$, which seems strange to me.