Backpropagation

Backpropagation has several related meanings when talking about neural networks. Here we mean the process of finding the gradient of a cost function.

Terminology confusion

• Some people use "backpropagation" to mean the process of finding the gradient of a cost function. This leads people to say things like "backpropagation is just the multivariable chain rule".
• Some people use "backpropagation" or maybe "backpropagation algorithm" to mean the entire gradient descent algorithm, where the gradient is computed using the multivariable chain rule.