User:IssaRice/Chain rule proofs: Difference between revisions

From Machinelearning
No edit summary
Line 3: Line 3:
Since <math>g</math> is differentiable at <math>y_0</math>, we know <math>g'(y_0)</math> is a real number, and we can write
Since <math>g</math> is differentiable at <math>y_0</math>, we know <math>g'(y_0)</math> is a real number, and we can write


<math>g(y) = g(y_0) + g'(y_0)(y - y_0) + [g(y) - (g(y_0) + g'(y_0)(y-y_0))]</math> (there is no magic: the terms just cancel out)
<math display="block">g(y) = g(y_0) + g'(y_0)(y - y_0) + [g(y) - (g(y_0) + g'(y_0)(y-y_0))]</math>
 
(there is no magic: the terms just cancel out)


If we define <math>E_g(\Delta y) := g(y) - (g(y_0) + g'(y_0)(y-y_0))</math> we can write
If we define <math>E_g(\Delta y) := g(y) - (g(y_0) + g'(y_0)(y-y_0))</math> we can write


<math>g(y) = g(y_0) + g'(f(x_0))(y - y_0) + E_g(\Delta y)</math>
<math display="block">g(y) = g(y_0) + g'(f(x_0))(y - y_0) + E_g(\Delta y)</math>


Newton's approximation says that <math>|E_g(\Delta y)| \leq \epsilon|y-y_0|</math> as long as <math>|y-y_0|\leq \delta</math>.
Newton's approximation says that <math>|E_g(\Delta y)| \leq \epsilon|y-y_0|</math> as long as <math>|y-y_0|\leq \delta</math>.
Line 15: Line 17:
Since <math>f(x) \in Y</math> and <math>|f(x)-y_0|\leq \delta</math>, this means we can substitute <math>y = f(x)</math> and get
Since <math>f(x) \in Y</math> and <math>|f(x)-y_0|\leq \delta</math>, this means we can substitute <math>y = f(x)</math> and get


<math>g(f(x)) = g(y_0) + g'(f(x_0))(f(x) - y_0) + E_g(\Delta f)</math>
<math display="block">g(f(x)) = g(y_0) + g'(f(x_0))(f(x) - y_0) + E_g(\Delta f)</math>


Now we use the differentiability of <math>f</math>. We can write
Now we use the differentiability of <math>f</math>. We can write


<math>f(x) = f(x_0) + f'(x_0)(x - x_0) + [f(x) - (f(x_0) + f'(x_0)(x-x_0))]</math>
<math display="block">f(x) = f(x_0) + f'(x_0)(x - x_0) + [f(x) - (f(x_0) + f'(x_0)(x-x_0))]</math>


Again, we can define <math>E_f(\Delta x) := f(x) - (f(x_0) + f'(x_0)(x-x_0))</math> and write this as
Again, we can define <math>E_f(\Delta x) := f(x) - (f(x_0) + f'(x_0)(x-x_0))</math> and write this as


<math>f(x) = f(x_0) + f'(x_0)(x - x_0) + E_f(\Delta x)</math>
<math display="block">f(x) = f(x_0) + f'(x_0)(x - x_0) + E_f(\Delta x)</math>


Now we can substitute this into the expression for <math>g(f(x))</math> to get
Now we can substitute this into the expression for <math>g(f(x))</math> to get


<math>g(f(x)) = g(y_0) + g'(f(x_0))(f'(x_0)(x - x_0) + E_f(\Delta x)) + E_g(\Delta f)</math>
<math display="block">g(f(x)) = g(y_0) + g'(f(x_0))(f'(x_0)(x - x_0) + E_f(\Delta x)) + E_g(\Delta f)</math>


where we have canceled out two terms using <math>f(x_0) = y_0</math>.
where we have canceled out two terms using <math>f(x_0) = y_0</math>.
Line 33: Line 35:
Thus we have
Thus we have


<math>g(f(x)) = g(y_0) + g'(f(x_0))f'(x_0)(x - x_0) + [g'(f(x_0))E_f(\Delta x) + E_g(\Delta f)]</math>
<math display="block">g(f(x)) = g(y_0) + g'(f(x_0))f'(x_0)(x - x_0) + [g'(f(x_0))E_f(\Delta x) + E_g(\Delta f)]</math>


We can write this as
We can write this as


<math>(g\circ f)(x) - ((g\circ f)(x_0) + L(x - x_0)) = [g'(f(x_0))E_f(\Delta x) + E_g(\Delta f)]</math>
<math display="block">(g\circ f)(x) - ((g\circ f)(x_0) + L(x - x_0)) = [g'(f(x_0))E_f(\Delta x) + E_g(\Delta f)]</math>


where <math>L := g'(f(x_0))f'(x_0)</math>. Now the left hand side looks like the expression in Newton's approximation. This means to show <math>g\circ f</math> is differentiable at <math>x_0</math>, we just need to show that <math>|g'(f(x_0))E_f(\Delta x) + E_g(\Delta f)| \leq \epsilon|x - x_0|</math>.
where <math>L := g'(f(x_0))f'(x_0)</math>. Now the left hand side looks like the expression in Newton's approximation. This means to show <math>g\circ f</math> is differentiable at <math>x_0</math>, we just need to show that <math>|g'(f(x_0))E_f(\Delta x) + E_g(\Delta f)| \leq \epsilon|x - x_0|</math>.
Line 45: Line 47:
But f is differentiable at <math>x_0</math>, so by Newton's approximation,
But f is differentiable at <math>x_0</math>, so by Newton's approximation,


<math>g'(f(x_0))E_f(\Delta x)
<math display="block">g'(f(x_0))E_f(\Delta x)</math>


==Limits of sequences==
==Limits of sequences==

Revision as of 01:44, 28 November 2018

Using Newton's approximation

Since g is differentiable at y0, we know g(y0) is a real number, and we can write

g(y)=g(y0)+g(y0)(yy0)+[g(y)(g(y0)+g(y0)(yy0))]

(there is no magic: the terms just cancel out)

If we define Eg(Δy):=g(y)(g(y0)+g(y0)(yy0)) we can write

g(y)=g(y0)+g(f(x0))(yy0)+Eg(Δy)

Newton's approximation says that |Eg(Δy)|ϵ|yy0| as long as |yy0|δ.

Since f is differentiable at x0, we know that it must be continuous at x0. This means we can keep |f(x)y0|δ as long as we keep |xx0|δ.

Since f(x)Y and |f(x)y0|δ, this means we can substitute y=f(x) and get

g(f(x))=g(y0)+g(f(x0))(f(x)y0)+Eg(Δf)

Now we use the differentiability of f. We can write

f(x)=f(x0)+f(x0)(xx0)+[f(x)(f(x0)+f(x0)(xx0))]

Again, we can define Ef(Δx):=f(x)(f(x0)+f(x0)(xx0)) and write this as

f(x)=f(x0)+f(x0)(xx0)+Ef(Δx)

Now we can substitute this into the expression for g(f(x)) to get

g(f(x))=g(y0)+g(f(x0))(f(x0)(xx0)+Ef(Δx))+Eg(Δf)

where we have canceled out two terms using f(x0)=y0.

Thus we have

g(f(x))=g(y0)+g(f(x0))f(x0)(xx0)+[g(f(x0))Ef(Δx)+Eg(Δf)]

We can write this as

(gf)(x)((gf)(x0)+L(xx0))=[g(f(x0))Ef(Δx)+Eg(Δf)]

where L:=g(f(x0))f(x0). Now the left hand side looks like the expression in Newton's approximation. This means to show gf is differentiable at x0, we just need to show that |g(f(x0))Ef(Δx)+Eg(Δf)|ϵ|xx0|.

The stuff in square brackets is our "error term" for gf. Now we just need to make sure it is small, even after dividing by |xx0|.

But f is differentiable at x0, so by Newton's approximation,

g(f(x0))Ef(Δx)

Limits of sequences