User:IssaRice/Chain rule proofs

Using Newton's approximation

Main idea

The main idea of using Newton's approximation to prove the chain rule is that since f is differentiable at $x_0$ we have the approximation $f(x)\approx f(x_0)+f^{\prime }(x_0)(x-x_0)$ when $x$ is near $x_0$. Similarly since g is differentiable at $f(x_0)$ we have the approximation $g(y)\approx g(f(x_0))+g^{\prime }(f(x_0))(y-f(x_0))$ when $y$ is near $f(x_0)$. Since f is differentiable at $x_0$, it is continuous there also, so we know that $f(x)$ is near $f(x_0)$ whenever $x$ is near $x_0$. This allows us to substitute $f(x)$ into $y$ whenever $x$ is near $x_0$. So we get

$$\begin{array}{rr}g(f(x))& \approx g(f(x_0))+g\text{'}(f(x_0))(f(x)-f(x_0))\\ & \approx g(f(x_0))+g\text{'}(f(x_0))(f\text{'}(x_0)(x-x_0))\end{array}$$

Thus we get $g\circ f(x)\approx g\circ f(x_0)+g^{\prime }(f(x_0))f^{\prime }(x_0)(x-x_0)$, which is what the chain rule says.

Proof

We want to show $g\circ f$ is differentiable at $x_0$ with derivative $L:=g^{\prime }(f(x_0))f^{\prime }(x_0)$. By Newton's approximation, this is equivalent to showing that for every $ϵ>0$ there exists $\delta >0$ such that

$$|g\circ f(x)-(g\circ f(x_0)+L(x-x_0))|\le ϵ|x-x_0|$$

whenever $|x-x_0|\le \delta$. So let $ϵ>0$.

Now we do some algebraic manipulation. Write

$$g(y)=g(y_0)+g^{\prime }(y_0)(y-y_0)+E_g(y,y_0)$$

where $E_g(y,y_0):=g(y)-(g(y_0)+g^{\prime }(y_0)(y-y_0))$. This holds for every $y\in Y$. Since $f(x)\in Y$ we thus have

$$g(f(x))=g(f(x_0))+g^{\prime }(f(x_0))(f(x)-f(x_0))+E_g(f(x),f(x_0))$$

Similarly write

$$f(x)=f(x_0)+f^{\prime }(x_0)(x-x_0)+E_f(x,x_0)$$

where $E_f(x,x_0):=f(x)-(f(x_0)+f^{\prime }(x_0)(x-x_0))$.

Substituting the expression for $f(x)$ in the expression for $g(f(x))$ we get

$$\begin{array}{rr}g(f(x))& =g(f(x_0))+g\text{'}(f(x_0))(f\text{'}(x_0)(x-x_0)+E_f(x,x_0))+E_g(f(x),f(x_0))\\ & =g(f(x_0))+g\text{'}(f(x_0))f\text{'}(x_0)(x-x_0)+g\text{'}(f(x_0))E_f(x,x_0)+E_g(f(x),f(x_0))\end{array}$$

we can rewrite this as $g\circ f(x)-(g\circ f(x_0)+L(x-x_0))=g^{\prime }(f(x_0))E_f(x,x_0)+E_g(f(x),f(x_0))$

Thus our goal now is to show $|g^{\prime }(f(x_0))E_f(x,x_0)+E_g(f(x),f(x_0))|\le ϵ|x-x_0|$.

But by the triangle inequality it suffices to show $|g^{\prime }(f(x_0))E_f(x,x_0)|+|E_g(f(x),f(x_0))|\le ϵ|x-x_0|$.

$|g^{\prime }(f(x_0))E_f(x,x_0)|\le |g^{\prime }(f(x_0))|{ϵ}_1|x-x_0|$ where we are free to choose ${ϵ}_1$.

To get the bound for $|E_g(f(x),f(x_0))|$ (using Newton's approximation), we need to make sure $|f(x)-f(x_0)|$ is small. But by continuity of $f$ at $x_0$ we can do this.

$$\begin{array}{rr}|E_g(f(x),f(x_0))|& \le {ϵ}_2|f(x)-f(x_0)|\\ & ={ϵ}_2|f\text{'}(x_0)(x-x_0)+E_f(x,x_0)|\\ & \le {ϵ}_2|f\text{'}(x_0)||x-x_0|+{ϵ}_2{ϵ}_3|x-x_0|\\ & =({ϵ}_2|f\text{'}(x_0)|+{ϵ}_2{ϵ}_3)|x-x_0|\end{array}$$

where again we are free to choose ${ϵ}_2,{ϵ}_3$.

TODO: can we do this same proof but without using the error term notation?

TODO: somehow Folland does this without explicitly using continuity of f; i need to understand if he's using it implicitly somehow or he's actually proving it when bounding $\left|\mathbf{h}\right|$ using $\left|u\right|$

old proof

Since $g$ is differentiable at $y_0$, we know $g^{\prime }(y_0)$ is a real number, and we can write

$$g(y)=g(y_0)+g^{\prime }(y_0)(y-y_0)+[g(y)-(g(y_0)+g^{\prime }(y_0)(y-y_0))]$$

(there is no magic: the terms just cancel out)

If we define $E_g(y,y_0):=g(y)-(g(y_0)+g^{\prime }(y_0)(y-y_0))$ we can write

$$g(y)=g(y_0)+g^{\prime }(f(x_0))(y-y_0)+E_g(y,y_0)$$

Newton's approximation says that $|E_g(y,y_0)|\le ϵ|y-y_0|$ as long as $|y-y_0|\le \delta$.

Since $f$ is differentiable at $x_0$, we know that it must be continuous at $x_0$. This means we can keep $|f(x)-y_0|\le \delta$ as long as we keep $|x-x_0|\le {\delta }^{\prime }$.

Since $f(x)\in Y$ and $|f(x)-y_0|\le \delta$, this means we can substitute $y=f(x)$ and get

$$g(f(x))=g(y_0)+g^{\prime }(f(x_0))(f(x)-y_0)+E_g(f(x),y_0)$$

Now we use the differentiability of $f$. We can write

$$f(x)=f(x_0)+f^{\prime }(x_0)(x-x_0)+[f(x)-(f(x_0)+f^{\prime }(x_0)(x-x_0))]$$

Again, we can define $E_f(x,x_0):=f(x)-(f(x_0)+f^{\prime }(x_0)(x-x_0))$ and write this as

$$f(x)=f(x_0)+f^{\prime }(x_0)(x-x_0)+E_f(x,x_0)$$

Now we can substitute this into the expression for $g(f(x))$ to get

$$g(f(x))=g(y_0)+g^{\prime }(f(x_0))(f^{\prime }(x_0)(x-x_0)+E_f(x,x_0))+E_g(f(x),f(x_0))$$

where we have canceled out two terms using $f(x_0)=y_0$.

Thus we have

$$g(f(x))=g(y_0)+g^{\prime }(f(x_0))f^{\prime }(x_0)(x-x_0)+[g^{\prime }(f(x_0))E_f(x,x_0)+E_g(f(x),f(x_0))]$$

We can write this as

$$(g\circ f)(x)-((g\circ f)(x_0)+L(x-x_0))=[g^{\prime }(f(x_0))E_f(x,x_0)+E_g(f(x),f(x_0))]$$

where $L:=g^{\prime }(f(x_0))f^{\prime }(x_0)$. Now the left hand side looks like the expression in Newton's approximation. This means to show $g\circ f$ is differentiable at $x_0$, we just need to show that $|g^{\prime }(f(x_0))E_f(x,x_0)+E_g(f(x),f(x_0))|\le ϵ|x-x_0|$.

The stuff in square brackets is our "error term" for $g\circ f$. Now we just need to make sure it is small, even after dividing by $|x-x_0|$.

But f is differentiable at $x_0$, so by Newton's approximation,

$$|g^{\prime }(f(x_0))E_f(x,x_0)|\le |g^{\prime }(f(x_0))|{ϵ}_1|x-x_0|$$

we also have

$$|E_g(f(x),f(x_0))|\le {ϵ}_2|f(x)-f(x_0)|={ϵ}_2|f^{\prime }(x_0)(x-x_0)+E_f(x,x_0)|$$

We can bound this from above using the triangle inequality:

$$\begin{array}{rr}|E_g(f(x),f(x_0))|& \le {ϵ}_2|f\text{'}(x_0)(x-x_0)|+{ϵ}_2|E_f(x,x_0)|\\ & \le {ϵ}_2|f\text{'}(x_0)||x-x_0|+{ϵ}_2{ϵ}_1|x-x_0|\end{array}$$

Now we can just choose ${ϵ}_1,{ϵ}_2$ small enough.

Limits of sequences

Main idea

Let $(x_n{)}_{n=1}^{\mathrm{\infty }}$ be a sequence taking values in $X\setminus \{x_0\}$ that converges to $x_0$. Then we want to write

$$\frac{g(f(x_n))-g(f(x_0))}{x_n-x_0}=\frac{g(f(x_n))-g(f(x_0))}{f(x_n)-f(x_0)}\cdot \frac{f(x_n)-f(x_0)}{x_n-x_0}$$

Now use the limit laws to conclude that the limit is $g^{\prime }(f(x_0))\cdot f^{\prime }(x_0)$. The problem is that $f(x_n)-f(x_0)$ can be zero even when $x_n\ne x_0$.

Proof

Let $(x_n{)}_{n=1}^{\mathrm{\infty }}$ be a sequence taking values in $X\setminus \{x_0\}$ that converges to $x_0$.

Define a function $\varphi :Y\to \mathbf{R}$ by

$$\varphi (y):=\{\begin{array}{cc}& y\ne f(x_0)\\ g\text{'}(f(x_0))& y=f(x_0)\end{array}$$

The idea is that we want to say $\frac{g(f(x_n))-g(f(x_0))}{f(x_n)-f(x_0)}$ is going to $g^{\prime }(f(x_0))$, so we just define it at the undefined points to already be at that limit.

Now we have

$$\frac{g(f(x_n))-g(f(x_0))}{x_n-x_0}=\varphi (f(x_n))\cdot \frac{f(x_n)-f(x_0)}{x_n-x_0}$$

for all $x_n$. (Why? Consider the cases $f(x_n)=f(x_0)$ and $f(x_n)\ne f(x_0)$ separately.)

Differentiability of $g$ at $f(x_0)$ says that if $(y_n{)}_{n=1}^{\mathrm{\infty }}$ is a sequence taking values in $Y\setminus \{y_0\}$ that converges to $f(x_0)$, then $\frac{g(y_n)-g(f(x_0))}{y_n-f(x_0)}\to g^{\prime }(f(x_0))$ as $n\to \mathrm{\infty }$. What if $(y_n{)}_{n=1}^{\mathrm{\infty }}$ is instead a sequence taking values in $Y$? Then we can say $\varphi (y_n)\to g^{\prime }(f(x_0))$ as $n\to \mathrm{\infty }$. To show this, let $ϵ>0$.

Now we can find $N\ge 1$ such that for all $n\ge N$, if $y_n\ne f(x_0)$, then $|\varphi (y_n)-g^{\prime }(f(x_0))|=|\frac{g(y_n)-g(f(x_0))}{y_n-f(x_0)}-g^{\prime }(f(x_0))|\le ϵ$.

But this means if $n\ge N$, then we have two cases: either $y_n\in Y$ and $y_n\ne f(x_0)$, in which case $|\varphi (y_n)-g^{\prime }(f(x_0))|\le ϵ$ as above, or else $y_n=f(x_0)$, in which case $\varphi (y_n)=g^{\prime }(f(x_0))$ so $|\varphi (y)-g^{\prime }(f(x_0))|=0\le ϵ$.

Differentiability of $f$ at $x_0$ implies continuity of $f$ at $x_0$, so this means that $f(x_n)\to f(x_0)$ as $n\to \mathrm{\infty }$. Since $f(x_n)\in Y$ for each $n\ge 1$, we can use $(f(x_n){)}_{n=1}^{\mathrm{\infty }}$ as our sequence in $Y$ to conclude that as $n\to \mathrm{\infty }$ we have $\varphi (f(x_n))\to g^{\prime }(f(x_0))$.

Now by the limit laws

$$\begin{array}{rr}\underset{n\to \mathrm{\infty }}{lim}\frac{g(f(x_n))-g(f(x_0))}{x_n-x_0}& =(\underset{n\to \mathrm{\infty }}{lim}\varphi (f(x_n)))(\underset{n\to \mathrm{\infty }}{lim}\frac{f(x_n)-f(x_0)}{x_n-x_0})\\ & =g\text{'}(f(x_0))f\text{'}(x_0)\end{array}$$

Since the sequence $(x_n{)}_{n=1}^{\mathrm{\infty }}$ was arbitrary, we can conclude that $\underset{x\to x_0;x\in X\setminus \{x_0\}}{lim}\frac{g(f(x))-g(f(x_0))}{x-x_0}=g^{\prime }(f(x_0))f^{\prime }(x_0)$.

$\frac{g(f(x_n))-g(f(x_0))}{f(x_n)-f(x_0)}\to g^{\prime }(f(x_0))$

TODO: Tao says that division by zero occurs when $f^{\prime }(x_0)=0$, which seems strange to me.