Mean Squared Error

Definition

Mean squared error (MSE) is a loss function that measures the average of the squared differences between predicted and actual values, commonly used to train and evaluate regression models.

Overview

Mean squared error is calculated by taking the difference between each predicted value and its corresponding true value, squaring that difference, and averaging the squared differences across all examples. Squaring the error serves two purposes: it ensures all errors contribute positively regardless of direction, and it disproportionately penalizes larger errors compared to smaller ones, since the penalty grows quadratically rather than linearly with the size of the error. MSE has a convenient mathematical property: for linear models, minimizing MSE has a closed-form solution (ordinary least squares) and corresponds to finding the maximum likelihood estimate under the assumption that errors are normally distributed. It is also differentiable everywhere, making it straightforward to optimize with gradient-based methods like stochastic gradient descent. The square root of MSE, root mean squared error (RMSE), is often reported instead because it is in the same units as the target variable, making it more directly interpretable. MSE's sensitivity to large errors is both a strength and a weakness: it makes the model pay close attention to outliers, which is desirable when large errors are especially costly, but undesirable when the data contains noisy or erroneous outlier points that shouldn't dominate training. In such cases, alternatives like mean absolute error (MAE), which penalizes errors linearly, or Huber loss, which behaves like MSE for small errors and like MAE for large ones, are often preferred. MSE is the standard loss and evaluation metric for regression tasks such as price prediction, forecasting, and any continuous-valued output, distinct from cross-entropy loss, which is used for classification.

Key Concepts

• Computes the average of squared differences between predictions and actual values
• Penalizes larger errors disproportionately more than smaller ones
• Differentiable everywhere, making it well suited to gradient-based optimization
• Corresponds to maximum likelihood estimation under a Gaussian error assumption
• Root mean squared error (RMSE) expresses the metric in the target variable's original units
• Sensitive to outliers, which can dominate the loss if not handled carefully
• Standard loss function and evaluation metric for regression tasks
• Has a closed-form solution for linear regression (ordinary least squares)

Use Cases

Frequently Asked Questions

Related in AI Fundamentals