Overfitting

Definition

Overfitting refers to a problem where a functional form or algorithm performs substantially better on the data used to train it than on new data drawn from the same distribution. It occurs when the parameters used to describe the functional form end up fitting the noise or random fluctuations in the training data rather than the attributes that are common between the training data and test data.

Overfitting typically occurs when the size (or diversity) of the training data is not substantially larger than the number of parameters.