Mean Squared Error (MSE)

Mean Squared Error (MSE) is one of the most common loss functions for numerical problems. It represents the average (squared) error in a regression problem, where the error is the distance from the actual value $y_i$ to the predicted value ${\hat{y}}_i$.

MSE is given by:

$$MSE=\frac{1}{N}\sum _{i=1}^N(y_i-{\hat{y}}_i{)}^2$$

One feature of MSE is that, due to the error values being squared, larger errors are penalized more (i.e. they contribute more to the overall MSE) than are smaller errors.

Note that since errors can be in the positive or negative direction (i.e. under-estimates or over-estimates), we need some way to incorporate the magnitude of the error into the loss function without accounting for the sign. Squaring the errors is a convenient way to do this. Another approach entails taking the absolute value.