Derivative of a quadratic form - Revision history

IssaRice: /* Using the chain rule */

2018-07-14T03:23:14Z

Using the chain rule

← Older revision		Revision as of 03:23, 14 July 2018
Line 95:		Line 95:

	We now have the <math>j</math>th component of the derivative, so the full derivative is <math>2y = 2Ax</math>.		We now have the <math>j</math>th component of the derivative, so the full derivative is <math>2y = 2Ax</math>.

			See [http://michael.orlitzky.com/articles/the_derivative_of_a_quadratic_form.xhtml] for something similar.

IssaRice: /* Straightforward method */

2018-07-14T03:22:42Z

Straightforward method

← Older revision		Revision as of 03:22, 14 July 2018
Line 23:		Line 23:
	Now distributing we have		Now distributing we have

	:<math>a_{jj}x_j^2 + \left(\sum_{i\ne j} a_{ji} x_i\right)x_j + \sum_{k\ne j} \left(a_{kj}x_k x_j + x_k \sum_{i\ne j} a_{ki} x_i\right) = a_{jj}x_j^2 + \left(\sum_{i\ne j} a_{ji} x_i\right)x_j + \left(\sum_{k\ne j}a_{kj}x_k\right) x_j + \sum_{k\ne j}x_k \sum_{i\ne j} a_{ki} x_i</math>		:<math>\begin{align}a_{jj}x_j^2 + \left(\sum_{i\ne j} a_{ji} x_i\right)x_j + \sum_{k\ne j} \left(a_{kj}x_k x_j + x_k \sum_{i\ne j} a_{ki} x_i\right) \\ = a_{jj}x_j^2 + \left(\sum_{i\ne j} a_{ji} x_i\right)x_j + \left(\sum_{k\ne j}a_{kj}x_k\right) x_j + \sum_{k\ne j}x_k \sum_{i\ne j} a_{ki} x_i\end{align}</math>

	It is now easy to do the differentiation. We obtain		It is now easy to do the differentiation. We obtain

IssaRice: /* Using the chain rule */

2018-07-14T01:45:33Z

Using the chain rule

← Older revision		Revision as of 01:45, 14 July 2018
Line 93:		Line 93:

	In the last equality we used the fact that <math>A</math> is symmetric.		In the last equality we used the fact that <math>A</math> is symmetric.

			We now have the <math>j</math>th component of the derivative, so the full derivative is <math>2y = 2Ax</math>.

IssaRice: /* Using the chain rule */

2018-07-14T01:42:24Z

Using the chain rule

← Older revision		Revision as of 01:42, 14 July 2018
Line 81:		Line 81:

	:<math>\frac{\partial z}{\partial x_j} = \underbrace{\frac{\partial z}{\partial x_1}\frac{\partial x_1}{\partial x_j} + \cdots + \frac{\partial z}{\partial x_n}\frac{\partial x_n}{\partial x_j}}_{\text{first half of terms}} + \underbrace{\frac{\partial z}{\partial y_1}\frac{\partial y_1}{\partial x_j} + \cdots + \frac{\partial z}{\partial y_n}\frac{\partial y_n}{\partial x_j}}_{\text{second half of terms}}</math>		:<math>\frac{\partial z}{\partial x_j} = \underbrace{\frac{\partial z}{\partial x_1}\frac{\partial x_1}{\partial x_j} + \cdots + \frac{\partial z}{\partial x_n}\frac{\partial x_n}{\partial x_j}}_{\text{first half of terms}} + \underbrace{\frac{\partial z}{\partial y_1}\frac{\partial y_1}{\partial x_j} + \cdots + \frac{\partial z}{\partial y_n}\frac{\partial y_n}{\partial x_j}}_{\text{second half of terms}}</math>

			The notation is confusing because <math>\frac{\partial z}{\partial x_j}</math> means different things on each side of the equation (since <math>x</math> is both the input variable and an intermediate variable).

	Looking only at the first half of the terms, <math>\frac{\partial x_k}{\partial x_j}</math> is <math>1</math> if <math>k=j</math> and <math>0</math> otherwise, so we keep only the <math>j</math>th term, where we see <math>\frac{\partial z}{\partial x_j} = y_j</math>.		Looking only at the first half of the terms, <math>\frac{\partial x_k}{\partial x_j}</math> is <math>1</math> if <math>k=j</math> and <math>0</math> otherwise, so we keep only the <math>j</math>th term, where we see <math>\frac{\partial z}{\partial x_j} = y_j</math>.

IssaRice: /* Using the chain rule */

2018-07-14T01:39:52Z

Using the chain rule

← Older revision		Revision as of 01:39, 14 July 2018
Line 88:		Line 88:
	Putting all the above together, we obtain		Putting all the above together, we obtain

	:<math>\frac{\partial z}{\partial x_j} = y_j + x_1 a_{1j} + \cdots + x_n a_{jn} = 2y_j</math>		:<math>\frac{\partial z}{\partial x_j} = y_j + x_1 a_{1j} + \cdots + x_n a_{nj} = 2y_j</math>

	In the last equality we used the fact that <math>A</math> is symmetric.		In the last equality we used the fact that <math>A</math> is symmetric.

IssaRice: /* Using the chain rule */

2018-07-14T01:39:35Z

Using the chain rule

← Older revision		Revision as of 01:39, 14 July 2018
Line 88:		Line 88:
	Putting all the above together, we obtain		Putting all the above together, we obtain

	:<math>\frac{\partial z}{\partial x_j} = y_j + x_1 a_{aj} + \cdots + x_n a_{jn} = 2y_j</math>		:<math>\frac{\partial z}{\partial x_j} = y_j + x_1 a_{1j} + \cdots + x_n a_{jn} = 2y_j</math>

	In the last equality we used the fact that <math>A</math> is symmetric.		In the last equality we used the fact that <math>A</math> is symmetric.

IssaRice: /* Using the chain rule */

2018-07-14T01:36:47Z

Using the chain rule

← Older revision		Revision as of 01:36, 14 July 2018
Line 80:		Line 80:
	Now the multivariable chain rule says:		Now the multivariable chain rule says:

	:<math>\frac{\partial z}{\partial x_j} = \frac{\partial z}{\partial x_1}\frac{\partial x_1}{\partial x_j} + \cdots + \frac{\partial z}{\partial x_n}\frac{\partial x_n}{\partial x_j} + \frac{\partial z}{\partial y_1}\frac{\partial y_1}{\partial x_j} + \cdots + \frac{\partial z}{\partial y_n}\frac{\partial y_n}{\partial x_j}</math>		:<math>\frac{\partial z}{\partial x_j} = \underbrace{\frac{\partial z}{\partial x_1}\frac{\partial x_1}{\partial x_j} + \cdots + \frac{\partial z}{\partial x_n}\frac{\partial x_n}{\partial x_j}}_{\text{first half of terms}} + \underbrace{\frac{\partial z}{\partial y_1}\frac{\partial y_1}{\partial x_j} + \cdots + \frac{\partial z}{\partial y_n}\frac{\partial y_n}{\partial x_j}}_{\text{second half of terms}}</math>

	Looking only at the first half of the terms, <math>\frac{\partial x_k}{\partial x_j}</math> is <math>1</math> if <math>k=j</math> and <math>0</math> otherwise, so we keep only the <math>j</math>th term, where we see <math>\frac{\partial z}{\partial x_j} = y_j</math>.		Looking only at the first half of the terms, <math>\frac{\partial x_k}{\partial x_j}</math> is <math>1</math> if <math>k=j</math> and <math>0</math> otherwise, so we keep only the <math>j</math>th term, where we see <math>\frac{\partial z}{\partial x_j} = y_j</math>.

IssaRice: /* Using the chain rule */

2018-07-14T01:34:32Z

Using the chain rule

← Older revision		Revision as of 01:34, 14 July 2018
Line 85:		Line 85:

	Now looking at the second half of the terms, <math>\frac{\partial z}{\partial y_k} = x_k</math> and <math>\frac{\partial y_k}{\partial x_j} = a_{kj}</math>.		Now looking at the second half of the terms, <math>\frac{\partial z}{\partial y_k} = x_k</math> and <math>\frac{\partial y_k}{\partial x_j} = a_{kj}</math>.

			Putting all the above together, we obtain

			:<math>\frac{\partial z}{\partial x_j} = y_j + x_1 a_{aj} + \cdots + x_n a_{jn} = 2y_j</math>

			In the last equality we used the fact that <math>A</math> is symmetric.

IssaRice: /* Using the chain rule */

2018-07-14T01:31:36Z

Using the chain rule

← Older revision		Revision as of 01:31, 14 July 2018
Line 83:		Line 83:

	Looking only at the first half of the terms, <math>\frac{\partial x_k}{\partial x_j}</math> is <math>1</math> if <math>k=j</math> and <math>0</math> otherwise, so we keep only the <math>j</math>th term, where we see <math>\frac{\partial z}{\partial x_j} = y_j</math>.		Looking only at the first half of the terms, <math>\frac{\partial x_k}{\partial x_j}</math> is <math>1</math> if <math>k=j</math> and <math>0</math> otherwise, so we keep only the <math>j</math>th term, where we see <math>\frac{\partial z}{\partial x_j} = y_j</math>.

			Now looking at the second half of the terms, <math>\frac{\partial z}{\partial y_k} = x_k</math> and <math>\frac{\partial y_k}{\partial x_j} = a_{kj}</math>.

IssaRice: /* Using the chain rule */

2018-07-14T01:29:06Z

Using the chain rule

← Older revision		Revision as of 01:29, 14 July 2018
Line 81:		Line 81:

	:<math>\frac{\partial z}{\partial x_j} = \frac{\partial z}{\partial x_1}\frac{\partial x_1}{\partial x_j} + \cdots + \frac{\partial z}{\partial x_n}\frac{\partial x_n}{\partial x_j} + \frac{\partial z}{\partial y_1}\frac{\partial y_1}{\partial x_j} + \cdots + \frac{\partial z}{\partial y_n}\frac{\partial y_n}{\partial x_j}</math>		:<math>\frac{\partial z}{\partial x_j} = \frac{\partial z}{\partial x_1}\frac{\partial x_1}{\partial x_j} + \cdots + \frac{\partial z}{\partial x_n}\frac{\partial x_n}{\partial x_j} + \frac{\partial z}{\partial y_1}\frac{\partial y_1}{\partial x_j} + \cdots + \frac{\partial z}{\partial y_n}\frac{\partial y_n}{\partial x_j}</math>

			Looking only at the first half of the terms, <math>\frac{\partial x_k}{\partial x_j}</math> is <math>1</math> if <math>k=j</math> and <math>0</math> otherwise, so we keep only the <math>j</math>th term, where we see <math>\frac{\partial z}{\partial x_j} = y_j</math>.