User:IssaRice/Chain rule proofs - Revision history

IssaRice at 01:01, 31 May 2020

2020-05-31T01:01:41Z

← Older revision		Revision as of 01:01, 31 May 2020
Line 1:		Line 1:
	a differentiable function looks locally like a linear transformation. if you compose two differentiable functions, then it seems pretty obvious that locally, that composed map also looks like a linear transformation. but which linear transformation? well, you first apply the linear transformation that locally approximates the inner function. then you land in some target space, and you find what the outer function locally looks like at the place you land. and that's all the chain rule says.		a differentiable function looks locally like a linear transformation. if you compose two differentiable functions, then it seems pretty obvious that locally, that composed map also looks like a linear transformation. but which linear transformation? well, you first apply the linear transformation that locally approximates the inner function. then you land in some target space, and you find what the outer function locally looks like at the place you land. and that's all the chain rule says.

	So Folland actually says this on page 109: "Since the product of two matrices gives the composition of the linear transformations defined by those matrices, the chain rule just says that ''the linear ~~approximations~~ of a composition is the composition of the linear approximations''." But... putting this short paragraph away after a proof, and only after you've already discussed two versions of the chain rule in previous chapters, is in my opinion not trying hard enough to communicate the central message. This should be in bold flashing letters at the TOP of the FIRST discussion of the chain rule.		So Folland actually says this on page 109: "Since the product of two matrices gives the composition of the linear transformations defined by those matrices, the chain rule just says that ''the linear approximation of a composition is the composition of the linear approximations''." But... putting this short paragraph away after a proof, and only after you've already discussed two versions of the chain rule in previous chapters, is in my opinion not trying hard enough to communicate the central message. This should be in bold flashing letters at the TOP of the FIRST discussion of the chain rule.

	==Using Newton's approximation==		==Using Newton's approximation==

IssaRice at 01:00, 31 May 2020

2020-05-31T01:00:58Z

← Older revision		Revision as of 01:00, 31 May 2020
Line 1:		Line 1:
	a differentiable function looks locally like a linear transformation. if you compose two differentiable functions, then it seems pretty obvious that locally, that composed map also looks like a linear transformation. but which linear transformation? well, you first apply the linear transformation that locally approximates the inner function. then you land in some target space, and you find what the outer function locally looks like at the place you land. and that's all the chain rule says.		a differentiable function looks locally like a linear transformation. if you compose two differentiable functions, then it seems pretty obvious that locally, that composed map also looks like a linear transformation. but which linear transformation? well, you first apply the linear transformation that locally approximates the inner function. then you land in some target space, and you find what the outer function locally looks like at the place you land. and that's all the chain rule says.

			So Folland actually says this on page 109: "Since the product of two matrices gives the composition of the linear transformations defined by those matrices, the chain rule just says that ''the linear approximations of a composition is the composition of the linear approximations''." But... putting this short paragraph away after a proof, and only after you've already discussed two versions of the chain rule in previous chapters, is in my opinion not trying hard enough to communicate the central message. This should be in bold flashing letters at the TOP of the FIRST discussion of the chain rule.

	==Using Newton's approximation==		==Using Newton's approximation==

IssaRice at 03:50, 18 January 2020

2020-01-18T03:50:47Z

← Older revision		Revision as of 03:50, 18 January 2020
Line 1:		Line 1:
			a differentiable function looks locally like a linear transformation. if you compose two differentiable functions, then it seems pretty obvious that locally, that composed map also looks like a linear transformation. but which linear transformation? well, you first apply the linear transformation that locally approximates the inner function. then you land in some target space, and you find what the outer function locally looks like at the place you land. and that's all the chain rule says.

	==Using Newton's approximation==		==Using Newton's approximation==

IssaRice: /* Limits of sequences */

2018-12-01T06:54:47Z

Limits of sequences

← Older revision		Revision as of 06:54, 1 December 2018
Line 155:		Line 155:
	Differentiability of <math>g</math> at <math>f(x_0)</math> says that if <math>(y_n)_{n=1}^\infty</math> is a sequence taking values in <math>Y \setminus \{y_0\}</math> that converges to <math>f(x_0)</math>, then <math>\frac{g(y_n) - g(f(x_0))}{y_n - f(x_0)} \to g'(f(x_0))</math> as <math>n \to \infty</math>. What if <math>(y_n)_{n=1}^\infty</math> is instead a sequence taking values in <math>Y</math>? Then we can say <math>\phi(y_n) \to g'(f(x_0))</math> as <math>n\to\infty</math>. To show this, let <math>\epsilon > 0</math>.		Differentiability of <math>g</math> at <math>f(x_0)</math> says that if <math>(y_n)_{n=1}^\infty</math> is a sequence taking values in <math>Y \setminus \{y_0\}</math> that converges to <math>f(x_0)</math>, then <math>\frac{g(y_n) - g(f(x_0))}{y_n - f(x_0)} \to g'(f(x_0))</math> as <math>n \to \infty</math>. What if <math>(y_n)_{n=1}^\infty</math> is instead a sequence taking values in <math>Y</math>? Then we can say <math>\phi(y_n) \to g'(f(x_0))</math> as <math>n\to\infty</math>. To show this, let <math>\epsilon > 0</math>.

	Now we can find <math>N \geq 1</math> such that for all <math>n \geq N</math>, if <math>y_n \ne f(x_0)</math>, then <math>\|\phi(y_n) - g'(f(x_0))\| = \left\vert\frac{g(y_n) - g(f(x_0))}{y_n - f(x_0)} - g'(f(x_0))\right\vert \leq \epsilon</math>. (TODO: I think here we need to break off into two cases: one where there's an infinite number of <math>y_n</math> such that <math>y_n \ne f(x_0)</math>, and one where there's only a finite number so that eventually the sequence is all just <math>y_n = f(x_0)</math>. Only in the former case can we find <math>N</math>, by considering the subsequence that isn't equal to <math>f(x_0)</math>, but this is not a problem because in the latter case the sequence's tail is already at the place where we need it to be, so we don't even need to find <math>N</math>. The question is, is there some more elegant way to do this that doesn't break off into ~~case~~?)		Now we can find <math>N \geq 1</math> such that for all <math>n \geq N</math>, if <math>y_n \ne f(x_0)</math>, then <math>\|\phi(y_n) - g'(f(x_0))\| = \left\vert\frac{g(y_n) - g(f(x_0))}{y_n - f(x_0)} - g'(f(x_0))\right\vert \leq \epsilon</math>. (TODO: I think here we need to break off into two cases: one where there's an infinite number of <math>y_n</math> such that <math>y_n \ne f(x_0)</math>, and one where there's only a finite number so that eventually the sequence is all just <math>y_n = f(x_0)</math>. Only in the former case can we find <math>N</math>, by considering the subsequence that isn't equal to <math>f(x_0)</math>, but this is not a problem because in the latter case the sequence's tail is already at the place where we need it to be, so we don't even need to find <math>N</math>. The question is, is there some more elegant way to do this that doesn't break off into cases?)

	But this means if <math>n \geq N</math>, then we have two cases: either <math>y_n \in Y</math> and <math>y_n \ne f(x_0)</math>, in which case <math>\|\phi(y_n) - g'(f(x_0))\| \leq \epsilon</math> as above, or else <math>y_n = f(x_0)</math>, in which case <math>\phi(y_n) = g'(f(x_0))</math> so <math>\|\phi(y) - g'(f(x_0))\| = 0 \leq \epsilon</math>.		But this means if <math>n \geq N</math>, then we have two cases: either <math>y_n \in Y</math> and <math>y_n \ne f(x_0)</math>, in which case <math>\|\phi(y_n) - g'(f(x_0))\| \leq \epsilon</math> as above, or else <math>y_n = f(x_0)</math>, in which case <math>\phi(y_n) = g'(f(x_0))</math> so <math>\|\phi(y) - g'(f(x_0))\| = 0 \leq \epsilon</math>.

IssaRice: /* Limits of sequences */

2018-12-01T06:54:13Z

Limits of sequences

← Older revision		Revision as of 06:54, 1 December 2018
Line 155:		Line 155:
	Differentiability of <math>g</math> at <math>f(x_0)</math> says that if <math>(y_n)_{n=1}^\infty</math> is a sequence taking values in <math>Y \setminus \{y_0\}</math> that converges to <math>f(x_0)</math>, then <math>\frac{g(y_n) - g(f(x_0))}{y_n - f(x_0)} \to g'(f(x_0))</math> as <math>n \to \infty</math>. What if <math>(y_n)_{n=1}^\infty</math> is instead a sequence taking values in <math>Y</math>? Then we can say <math>\phi(y_n) \to g'(f(x_0))</math> as <math>n\to\infty</math>. To show this, let <math>\epsilon > 0</math>.		Differentiability of <math>g</math> at <math>f(x_0)</math> says that if <math>(y_n)_{n=1}^\infty</math> is a sequence taking values in <math>Y \setminus \{y_0\}</math> that converges to <math>f(x_0)</math>, then <math>\frac{g(y_n) - g(f(x_0))}{y_n - f(x_0)} \to g'(f(x_0))</math> as <math>n \to \infty</math>. What if <math>(y_n)_{n=1}^\infty</math> is instead a sequence taking values in <math>Y</math>? Then we can say <math>\phi(y_n) \to g'(f(x_0))</math> as <math>n\to\infty</math>. To show this, let <math>\epsilon > 0</math>.

	Now we can find <math>N \geq 1</math> such that for all <math>n \geq N</math>, if <math>y_n \ne f(x_0)</math>, then <math>\|\phi(y_n) - g'(f(x_0))\| = \left\vert\frac{g(y_n) - g(f(x_0))}{y_n - f(x_0)} - g'(f(x_0))\right\vert \leq \epsilon</math>. (TODO: I think here we need to break off into two cases: one where there's an infinite number of <math>y_n</math> such that <math>y_n \ne f(x_0)</math>, and one where there's only a finite number so that eventually the sequence is all just <math>y_n = f(x_0)</math>. Only in the former case can we find <math>N</math>, but this is not a problem because in the latter case the sequence's tail is already at the place where we need it to be, so we don't even need to find <math>N</math>. The question is, is there some more elegant way to do this that doesn't break off into case?)		Now we can find <math>N \geq 1</math> such that for all <math>n \geq N</math>, if <math>y_n \ne f(x_0)</math>, then <math>\|\phi(y_n) - g'(f(x_0))\| = \left\vert\frac{g(y_n) - g(f(x_0))}{y_n - f(x_0)} - g'(f(x_0))\right\vert \leq \epsilon</math>. (TODO: I think here we need to break off into two cases: one where there's an infinite number of <math>y_n</math> such that <math>y_n \ne f(x_0)</math>, and one where there's only a finite number so that eventually the sequence is all just <math>y_n = f(x_0)</math>. Only in the former case can we find <math>N</math>, by considering the subsequence that isn't equal to <math>f(x_0)</math>, but this is not a problem because in the latter case the sequence's tail is already at the place where we need it to be, so we don't even need to find <math>N</math>. The question is, is there some more elegant way to do this that doesn't break off into case?)

	But this means if <math>n \geq N</math>, then we have two cases: either <math>y_n \in Y</math> and <math>y_n \ne f(x_0)</math>, in which case <math>\|\phi(y_n) - g'(f(x_0))\| \leq \epsilon</math> as above, or else <math>y_n = f(x_0)</math>, in which case <math>\phi(y_n) = g'(f(x_0))</math> so <math>\|\phi(y) - g'(f(x_0))\| = 0 \leq \epsilon</math>.		But this means if <math>n \geq N</math>, then we have two cases: either <math>y_n \in Y</math> and <math>y_n \ne f(x_0)</math>, in which case <math>\|\phi(y_n) - g'(f(x_0))\| \leq \epsilon</math> as above, or else <math>y_n = f(x_0)</math>, in which case <math>\phi(y_n) = g'(f(x_0))</math> so <math>\|\phi(y) - g'(f(x_0))\| = 0 \leq \epsilon</math>.

IssaRice: /* Limits of sequences */

2018-12-01T06:53:09Z

Limits of sequences

← Older revision		Revision as of 06:53, 1 December 2018
Line 155:		Line 155:
	Differentiability of <math>g</math> at <math>f(x_0)</math> says that if <math>(y_n)_{n=1}^\infty</math> is a sequence taking values in <math>Y \setminus \{y_0\}</math> that converges to <math>f(x_0)</math>, then <math>\frac{g(y_n) - g(f(x_0))}{y_n - f(x_0)} \to g'(f(x_0))</math> as <math>n \to \infty</math>. What if <math>(y_n)_{n=1}^\infty</math> is instead a sequence taking values in <math>Y</math>? Then we can say <math>\phi(y_n) \to g'(f(x_0))</math> as <math>n\to\infty</math>. To show this, let <math>\epsilon > 0</math>.		Differentiability of <math>g</math> at <math>f(x_0)</math> says that if <math>(y_n)_{n=1}^\infty</math> is a sequence taking values in <math>Y \setminus \{y_0\}</math> that converges to <math>f(x_0)</math>, then <math>\frac{g(y_n) - g(f(x_0))}{y_n - f(x_0)} \to g'(f(x_0))</math> as <math>n \to \infty</math>. What if <math>(y_n)_{n=1}^\infty</math> is instead a sequence taking values in <math>Y</math>? Then we can say <math>\phi(y_n) \to g'(f(x_0))</math> as <math>n\to\infty</math>. To show this, let <math>\epsilon > 0</math>.

	Now we can find <math>N \geq 1</math> such that for all <math>n \geq N</math>, if <math>y_n \ne f(x_0)</math>, then <math>\|\phi(y_n) - g'(f(x_0))\| = \left\vert\frac{g(y_n) - g(f(x_0))}{y_n - f(x_0)} - g'(f(x_0))\right\vert \leq \epsilon</math>.		Now we can find <math>N \geq 1</math> such that for all <math>n \geq N</math>, if <math>y_n \ne f(x_0)</math>, then <math>\|\phi(y_n) - g'(f(x_0))\| = \left\vert\frac{g(y_n) - g(f(x_0))}{y_n - f(x_0)} - g'(f(x_0))\right\vert \leq \epsilon</math>. (TODO: I think here we need to break off into two cases: one where there's an infinite number of <math>y_n</math> such that <math>y_n \ne f(x_0)</math>, and one where there's only a finite number so that eventually the sequence is all just <math>y_n = f(x_0)</math>. Only in the former case can we find <math>N</math>, but this is not a problem because in the latter case the sequence's tail is already at the place where we need it to be, so we don't even need to find <math>N</math>. The question is, is there some more elegant way to do this that doesn't break off into case?)

	But this means if <math>n \geq N</math>, then we have two cases: either <math>y_n \in Y</math> and <math>y_n \ne f(x_0)</math>, in which case <math>\|\phi(y_n) - g'(f(x_0))\| \leq \epsilon</math> as above, or else <math>y_n = f(x_0)</math>, in which case <math>\phi(y_n) = g'(f(x_0))</math> so <math>\|\phi(y) - g'(f(x_0))\| = 0 \leq \epsilon</math>.		But this means if <math>n \geq N</math>, then we have two cases: either <math>y_n \in Y</math> and <math>y_n \ne f(x_0)</math>, in which case <math>\|\phi(y_n) - g'(f(x_0))\| \leq \epsilon</math> as above, or else <math>y_n = f(x_0)</math>, in which case <math>\phi(y_n) = g'(f(x_0))</math> so <math>\|\phi(y) - g'(f(x_0))\| = 0 \leq \epsilon</math>.

IssaRice: /* Using Newton's approximation */

2018-11-29T16:18:34Z

Using Newton's approximation

← Older revision		Revision as of 16:18, 29 November 2018
Line 21:		Line 21:
	<math display="block">\begin{align}g(f(x)) &= g(f(x_0)) + g'(f(x_0))(f'(x_0)(x-x_0) + o(x-x_0)) + o(f(x)-f(x_0)) \\ &= g(f(x_0)) + g'(f(x_0))f'(x_0)(x-x_0) + g'(f(x_0))o(x-x_0) + o(f(x)-f(x_0))\end{align}</math>		<math display="block">\begin{align}g(f(x)) &= g(f(x_0)) + g'(f(x_0))(f'(x_0)(x-x_0) + o(x-x_0)) + o(f(x)-f(x_0)) \\ &= g(f(x_0)) + g'(f(x_0))f'(x_0)(x-x_0) + g'(f(x_0))o(x-x_0) + o(f(x)-f(x_0))\end{align}</math>

	to complete the proof, we just need to show that the error <math>g'(f(x_0))o(x-x_0) + o(f(x)-f(x_0))</math> is <math>o(x-x_0)</math>.		to complete the proof, we just need to show that the error <math>g'(f(x_0))o(x-x_0) + o(f(x)-f(x_0))</math> is <math>o(x-x_0)</math>. The former term clearly is. The latter term is <math>o(f'(x_0)(x-x_0) + o(x-x_0))</math> so it is as well.

	===Proof===		===Proof===

IssaRice: /* Main idea */

2018-11-29T16:15:59Z

Main idea

← Older revision		Revision as of 16:15, 29 November 2018
Line 19:		Line 19:
	Thus if <math>x</math> is near <math>x_0</math>		Thus if <math>x</math> is near <math>x_0</math>

	<math display="block">\begin{align}g(f(x)) &= g(f(x_0)) + g'(f(x_0))(f'(x_0)(x-x_0) + o(x-x_0)) + o(y-f(x_0)) \\ &= g(f(x_0)) + g'(f(x_0))f'(x_0)(x-x_0) + g'(f(x_0))o(x-x_0) + o(y-f(x_0))\end{align}</math>		<math display="block">\begin{align}g(f(x)) &= g(f(x_0)) + g'(f(x_0))(f'(x_0)(x-x_0) + o(x-x_0)) + o(f(x)-f(x_0)) \\ &= g(f(x_0)) + g'(f(x_0))f'(x_0)(x-x_0) + g'(f(x_0))o(x-x_0) + o(f(x)-f(x_0))\end{align}</math>

			to complete the proof, we just need to show that the error <math>g'(f(x_0))o(x-x_0) + o(f(x)-f(x_0))</math> is <math>o(x-x_0)</math>.

	===Proof===		===Proof===

IssaRice: /* Using Newton's approximation */

2018-11-29T16:14:00Z

Using Newton's approximation

← Older revision		Revision as of 16:14, 29 November 2018
Line 8:		Line 8:

	Thus we get <math>g\circ f(x) \approx g\circ f(x_0) + g'(f(x_0))f'(x_0)(x-x_0)</math>, which is what the chain rule says.		Thus we get <math>g\circ f(x) \approx g\circ f(x_0) + g'(f(x_0))f'(x_0)(x-x_0)</math>, which is what the chain rule says.

			Slightly more formally:

			<math>f(x) = f(x_0) + f'(x_0)(x-x_0) + o(x-x_0)</math> when <math>x</math> is near <math>x_0</math>

			<math>g(y) = g(f(x_0)) + g'(f(x_0))(y - f(x_0)) + o(y-f(x_0))</math> when <math>y</math> is near <math>f(x_0)</math>

			<math>f</math> is continuous at <math>x_0</math> so <math>f(x)</math> is near <math>f(x_0)</math> whenever <math>x</math> is near <math>x_0</math>.

			Thus if <math>x</math> is near <math>x_0</math>

			<math display="block">\begin{align}g(f(x)) &= g(f(x_0)) + g'(f(x_0))(f'(x_0)(x-x_0) + o(x-x_0)) + o(y-f(x_0)) \\ &= g(f(x_0)) + g'(f(x_0))f'(x_0)(x-x_0) + g'(f(x_0))o(x-x_0) + o(y-f(x_0))\end{align}</math>

	===Proof===		===Proof===

IssaRice: /* Using Newton's approximation */

2018-11-29T05:30:40Z

Using Newton's approximation

← Older revision		Revision as of 05:30, 29 November 2018
Line 45:		Line 45:
	To get the bound for <math>\|E_g(f(x),f(x_0))\|</math> (using Newton's approximation), we need to make sure <math>\|f(x)-f(x_0)\|</math> is small. But by continuity of <math>f</math> at <math>x_0</math> we can do this.		To get the bound for <math>\|E_g(f(x),f(x_0))\|</math> (using Newton's approximation), we need to make sure <math>\|f(x)-f(x_0)\|</math> is small. But by continuity of <math>f</math> at <math>x_0</math> we can do this.

	<math display="block">\begin{align}\|E_g(f(x),f(x_0))\| &\leq \epsilon_2 \|f(x) - f(x_0)\| \\ &= \epsilon_2 \|f'(x_0)(x - x_0) + E_f(x,x_0)\| \\ &\leq \epsilon_2\|f'(x_0)\|\|x-x_0\| + ~~\epsilon_2~~\|x-x_0\| \\ &= \epsilon_2(\|f'(x_0)\| + 1)\|x-x_0\|\end{align}</math>		<math display="block">\begin{align}\|E_g(f(x),f(x_0))\| &\leq \epsilon_2 \|f(x) - f(x_0)\| \\ &= \epsilon_2 \|f'(x_0)(x - x_0) + E_f(x,x_0)\| \\ &\leq \epsilon_2(\|f'(x_0)\|\|x-x_0\| + \|x-x_0\|) \\ &= \epsilon_2(\|f'(x_0)\| + 1)\|x-x_0\|\end{align}</math>

	where again we are free to choose <math>\epsilon_2~~, \epsilon_3~~</math>.		where again we are free to choose <math>\epsilon_2</math>.

	TODO: can we do this same proof but without using the error term notation?		TODO: can we do this same proof but without using the error term notation?