Second-order iterative learning algorithm

Definition

A second-order iterative learning algorithm is an iterative learning algorithm where each iteration of the algorithm uses only second-order information about the preceding iteration.

Memoryless second-order iterative learning algorithm

A memoryless second-order iterative learning algorithm has the following form of rule for each iteration:

New parameter vector = Explicit function(Old parameter vector, estimated cost function, estimated gradient vector for cost function, estimated Hessian matrix for cost function)