Model class selection - Revision history

Vipul: /* Parameters */

2016-05-07T20:06:48Z

Parameters

← Older revision		Revision as of 20:06, 7 May 2016
Line 13:		Line 13:
	For instance, a model for house price might say that it is of the following form, where <math>l</math> and <math>b</math> are the length and breadth of the house:		For instance, a model for house price might say that it is of the following form, where <math>l</math> and <math>b</math> are the length and breadth of the house:

	House price = <math>\theta_1 + \theta_2 l + \~~theta_2~~ b + \~~theta_3~~ lb</math>		House price = <math>\theta_1 + \theta_2 l + \theta_3 b + \theta_4 lb</math>

	where <math>\theta_1, \theta_2, \theta_3, \theta_4</math> are unknown real numbers, to be determined empirically using the training data. Once we have determined the parameter values, we will be able to predict the price of a house given knowledge of its length and breadth.		where <math>\theta_1, \theta_2, \theta_3, \theta_4</math> are unknown real numbers, to be determined empirically using the training data. Once we have determined the parameter values, we will be able to predict the price of a house given knowledge of its length and breadth.

Vipul at 19:37, 7 May 2016

2016-05-07T19:37:16Z

← Older revision		Revision as of 19:37, 7 May 2016
Line 3:		Line 3:
	==Definition==		==Definition==

	'''Model class selection''' or '''model type selection''' refers to the problem of selecting a suitable functional form (with possibly unknown parameters) describing how to predict the outputs in terms of a set of [[feature]]s. The model selection problem is generally the next step after [[feature selection]], which decides the set of appropriate features to choose.		'''Model class selection''' or '''model type selection''' refers to the problem of selecting a suitable functional form (with possibly unknown parameters) describing how to predict the outputs in terms of a set of [[feature]]s. The model class selection problem is generally the next step after [[feature selection]], which decides the set of appropriate features to choose.

	[[Cost function selection]] is sometimes viewed as part of model selection, and sometimes viewed as a separate step. A rationale for viewing cost function selection as part of model selection is that the choice of [[cost function]] reflects our understanding of the error distribution intrinsic to the model or to our measurement of features and outputs. A rationale against viewing cost function selection as part of model selection is that the error distribution can, however, be considered conceptually separate from the main task of describing the functional form of the model.		[[Cost function selection]] is sometimes viewed as part of model selection, and sometimes viewed as a separate step. A rationale for viewing cost function selection as part of model selection is that the choice of [[cost function]] reflects our understanding of the error distribution intrinsic to the model or to our measurement of features and outputs. A rationale against viewing cost function selection as part of model selection is that the error distribution can, however, be considered conceptually separate from the main task of describing the functional form of the model.

Vipul at 14:43, 16 August 2014

2014-08-16T14:43:45Z

← Older revision		Revision as of 14:43, 16 August 2014
Line 3:		Line 3:
	==Definition==		==Definition==

	'''Model selection''' refers to the problem of selecting a suitable functional form (with possibly unknown parameters) describing how to predict the outputs in terms of a set of [[feature]]s. The model selection problem is generally the next step after [[feature selection]], which decides the set of appropriate features to choose.		'''Model class selection''' or '''model type selection''' refers to the problem of selecting a suitable functional form (with possibly unknown parameters) describing how to predict the outputs in terms of a set of [[feature]]s. The model selection problem is generally the next step after [[feature selection]], which decides the set of appropriate features to choose.

	[[Cost function selection]] is sometimes viewed as part of model selection, and sometimes viewed as a separate step. A rationale for viewing cost function selection as part of model selection is that the choice of [[cost function]] reflects our understanding of the error distribution intrinsic to the model or to our measurement of features and outputs. A rationale against viewing cost function selection as part of model selection is that the error distribution can, however, be considered conceptually separate from the main task of describing the functional form of the model.		[[Cost function selection]] is sometimes viewed as part of model selection, and sometimes viewed as a separate step. A rationale for viewing cost function selection as part of model selection is that the choice of [[cost function]] reflects our understanding of the error distribution intrinsic to the model or to our measurement of features and outputs. A rationale against viewing cost function selection as part of model selection is that the error distribution can, however, be considered conceptually separate from the main task of describing the functional form of the model.

Vipul: Vipul moved page Model type selection to Model class selection

2014-08-16T14:43:30Z

Vipul moved page Model type selection to Model class selection

← Older revision	Revision as of 14:43, 16 August 2014
(No difference)

Vipul: Vipul moved page Model selection to Model type selection

2014-08-15T18:21:36Z

Vipul moved page Model selection to Model type selection

← Older revision	Revision as of 18:21, 15 August 2014
(No difference)

Vipul: /* Definition */

2014-06-25T17:08:18Z

Definition

← Older revision		Revision as of 17:08, 25 June 2014
Line 5:		Line 5:
	'''Model selection''' refers to the problem of selecting a suitable functional form (with possibly unknown parameters) describing how to predict the outputs in terms of a set of [[feature]]s. The model selection problem is generally the next step after [[feature selection]], which decides the set of appropriate features to choose.		'''Model selection''' refers to the problem of selecting a suitable functional form (with possibly unknown parameters) describing how to predict the outputs in terms of a set of [[feature]]s. The model selection problem is generally the next step after [[feature selection]], which decides the set of appropriate features to choose.

	[[Cost function selection]] is sometimes viewed as part of model selection, and sometimes viewed as a separate step. A rationale for viewing cost function selection as part of model selection is that the choice of cost function reflects our understanding of the error distribution intrinsic to the model or to our measurement of features and outputs. A rationale against viewing cost function selection as part of model selection is that the error distribution can, however, be considered conceptually separate from the main task of describing the functional form of the model.		[[Cost function selection]] is sometimes viewed as part of model selection, and sometimes viewed as a separate step. A rationale for viewing cost function selection as part of model selection is that the choice of [[cost function]] reflects our understanding of the error distribution intrinsic to the model or to our measurement of features and outputs. A rationale against viewing cost function selection as part of model selection is that the error distribution can, however, be considered conceptually separate from the main task of describing the functional form of the model.

	==Parameters==		==Parameters==

Vipul: /* Definition */

2014-06-25T17:08:01Z

Definition

← Older revision		Revision as of 17:08, 25 June 2014
Line 3:		Line 3:
	==Definition==		==Definition==

	'''Model selection''' refers to the problem of selecting a suitable functional form (with possibly unknown parameters) describing how to predict the outputs in terms of a set of [[feature]]s. The model selection problem is generally the next step after [[feature selection]], which decides the set of appropriate features to choose. ~~However, in some cases, the model can be selected~~		'''Model selection''' refers to the problem of selecting a suitable functional form (with possibly unknown parameters) describing how to predict the outputs in terms of a set of [[feature]]s. The model selection problem is generally the next step after [[feature selection]], which decides the set of appropriate features to choose.

	[[Cost function selection]] is sometimes viewed as part of model selection, and sometimes viewed as a separate step. A rationale for viewing cost function selection as part of model selection is that the choice of cost function reflects our understanding of the error distribution intrinsic to the model or to our measurement of features and outputs. A rationale against viewing cost function selection as part of model selection is that the error distribution can, however, be considered conceptually separate from the main task of describing the functional form of the model.		[[Cost function selection]] is sometimes viewed as part of model selection, and sometimes viewed as a separate step. A rationale for viewing cost function selection as part of model selection is that the choice of cost function reflects our understanding of the error distribution intrinsic to the model or to our measurement of features and outputs. A rationale against viewing cost function selection as part of model selection is that the error distribution can, however, be considered conceptually separate from the main task of describing the functional form of the model.

Vipul at 23:53, 18 June 2014

2014-06-18T23:53:59Z

← Older revision		Revision as of 23:53, 18 June 2014
Line 9:		Line 9:
	==Parameters==		==Parameters==

	A model typically involves a functional ''form'' but does not specify the actual function. Rather, it specifies the function in terms of unknown [[parameter]]s. These parameters need to be determined using the training data available to the model.		A model typically involves a functional ''form'' but does not specify the actual function. Rather, it specifies the function in terms of unknown [[model parameter]]s. These parameters need to be determined using the training data available to the model.

	For instance, a model for house price might say that it is of the following form, where <math>l</math> and <math>b</math> are the length and breadth of the house:		For instance, a model for house price might say that it is of the following form, where <math>l</math> and <math>b</math> are the length and breadth of the house:

Vipul: /* Relation between the number of parameters and the number of features */

2014-06-18T22:49:22Z

Relation between the number of parameters and the number of features

← Older revision		Revision as of 22:49, 18 June 2014
Line 23:		Line 23:
	* If the model is linear in the features, the intercept is zero (or a predetermined constant), and the coefficients for the features are freely varying and independent, then the number of parameters equals the number of features.		* If the model is linear in the features, the intercept is zero (or a predetermined constant), and the coefficients for the features are freely varying and independent, then the number of parameters equals the number of features.
	* If the model is linear in the features, the coefficients for the features are freely varying and independent, and the intercept is itself a separate parameter, then the number of parameters equals one more than the number of features.		* If the model is linear in the features, the coefficients for the features are freely varying and independent, and the intercept is itself a separate parameter, then the number of parameters equals one more than the number of features.
	* In general, if the model is linear in the ''parameters'', we can replace the existing set of features by a transformed set of derived features such that the model is linear in those derived features, and in that case, the number of derived features equals the number of parameters. This is the situation for [[linear regression]].		* In general, if the model is linear in the ''parameters'', we can replace the existing set of features by a transformed set of derived features such that the model is linear in those derived features, and in that case, the number of derived features equals the number of parameters. This is the situation for [[linear regression]]. For instance, the above example of house prices has 2 features (<math>l,b</math>) and 4 parameters (<math>\theta_1, \theta_2, \theta_3, \theta_4</math>). We can replace the set of features (<math>l,b</math>) by a set of derived features (<math>(1,l,b,lb)</math>) thereby getting an equal number of features and parameters.

Vipul at 22:43, 18 June 2014

2014-06-18T22:43:46Z

← Older revision		Revision as of 22:43, 18 June 2014
Line 6:		Line 6:

	[[Cost function selection]] is sometimes viewed as part of model selection, and sometimes viewed as a separate step. A rationale for viewing cost function selection as part of model selection is that the choice of cost function reflects our understanding of the error distribution intrinsic to the model or to our measurement of features and outputs. A rationale against viewing cost function selection as part of model selection is that the error distribution can, however, be considered conceptually separate from the main task of describing the functional form of the model.		[[Cost function selection]] is sometimes viewed as part of model selection, and sometimes viewed as a separate step. A rationale for viewing cost function selection as part of model selection is that the choice of cost function reflects our understanding of the error distribution intrinsic to the model or to our measurement of features and outputs. A rationale against viewing cost function selection as part of model selection is that the error distribution can, however, be considered conceptually separate from the main task of describing the functional form of the model.

			==Parameters==

			A model typically involves a functional ''form'' but does not specify the actual function. Rather, it specifies the function in terms of unknown [[parameter]]s. These parameters need to be determined using the training data available to the model.

			For instance, a model for house price might say that it is of the following form, where <math>l</math> and <math>b</math> are the length and breadth of the house:

			House price = <math>\theta_1 + \theta_2 l + \theta_2 b + \theta_3 lb</math>

			where <math>\theta_1, \theta_2, \theta_3, \theta_4</math> are unknown real numbers, to be determined empirically using the training data. Once we have determined the parameter values, we will be able to predict the price of a house given knowledge of its length and breadth.

			===Relation between the number of parameters and the number of features===

			The number of parameters need not equal the number of features in general. However, the following can be said:

			* If the model is linear in the features, the intercept is zero (or a predetermined constant), and the coefficients for the features are freely varying and independent, then the number of parameters equals the number of features.
			* If the model is linear in the features, the coefficients for the features are freely varying and independent, and the intercept is itself a separate parameter, then the number of parameters equals one more than the number of features.
			* In general, if the model is linear in the ''parameters'', we can replace the existing set of features by a transformed set of derived features such that the model is linear in those derived features, and in that case, the number of derived features equals the number of parameters. This is the situation for [[linear regression]].