Vipul: /* Memoryless first-order iterative learning algorithm */

2014-06-18T23:57:42Z

Memoryless first-order iterative learning algorithm

← Older revision		Revision as of 23:57, 18 June 2014
Line 7:		Line 7:
	A memoryless first-order iterative learning algorithm has the following form of rule for each iteration:		A memoryless first-order iterative learning algorithm has the following form of rule for each iteration:

	New parameter vector = Explicit function(Old parameter vector, estimated ~~gradient vector for~~ cost function, estimated cost function)		New parameter vector = Explicit function(Old parameter vector, estimated cost function, estimated gradient vector for cost function)

	A slight variant is:		A slight variant is:

Vipul: Created page with "==Definition== A '''first-order iterative learning algorithm''' is an iterative learning algorithm where each iteration of the algorithm uses only first-order information..."

2014-06-18T23:04:00Z

Created page with "==Definition== A '''first-order iterative learning algorithm''' is an iterative learning algorithm where each iteration of the algorithm uses only first-order information..."

New page

==Definition==

A '''first-order iterative learning algorithm''' is an [[iterative learning algorithm]] where each iteration of the algorithm uses only first-order information about the preceding iteration.

===Memoryless first-order iterative learning algorithm===

A memoryless first-order iterative learning algorithm has the following form of rule for each iteration:

New parameter vector = Explicit function(Old parameter vector, estimated gradient vector for cost function, estimated cost function)

A slight variant is:

New parameter vector = Explicit function(Old parameter vector, estimated gradient vector for cost function, estimated cost function, number of iterations so far)

Such an algorithm uses the number of iterations so far in determining the new parameter vector, but does not use any other information on the past history of iterations.

An example of this is [[gradient descent with constant learning rate]].

First-order iterative learning algorithm - Revision history

Vipul: /* Memoryless first-order iterative learning algorithm */

Vipul: Created page with "==Definition== A '''first-order iterative learning algorithm''' is an iterative learning algorithm where each iteration of the algorithm uses only first-order information..."