A fitted model rarely predicts every observation exactly. Residuals measure the discrepancies and help reveal whether the chosen model misses a systematic pattern. They are diagnostic information, not just errors to ignore.
A residual is observed minus predicted
For an observation and prediction the residual is
A positive residual means the observation lies above the model: the model underpredicted. A negative residual means the observation lies below it: the model overpredicted. Residuals have the same units as the response variable.
For a model predicting 72 points when a student scores 78, the residual is 6 points. Reversing the subtraction would reverse every sign and its interpretation.
On a graph, residuals are vertical differences at the same x-value, not shortest perpendicular distances to the fitted curve.
Example: calculate and interpret residuals
A model predicts plant height by where is weeks and height is centimeters. Observed heights at weeks 1, 2, and 3 are 6, 11, and 12 cm.
The predictions are 7, 10, and 13. Subtract to obtain residuals The model overpredicts weeks 1 and 3 by 1 cm and underpredicts week 2 by 1 cm.
A residual plot places input horizontally and residual vertically. Its horizontal zero line means perfect prediction. Using the same scale above and below zero makes both directions easy to compare.
Random scatter versus a missing pattern
A useful model often leaves residuals scattered around zero without a clear curve, trend, or changing spread. This suggests that the model captures the main systematic relationship.
If residuals form a U-shape, the model tends to miss in one direction at the ends and the opposite direction in the middle. A straight line may be inappropriate for a curved relationship. A fan shape, with spread increasing across x, suggests that prediction variability changes with x.
A good-looking residual plot does not prove the model is true. It is one piece of evidence, and small samples can hide patterns. Also, independence of observations depends on how data were collected, not simply on a plot looking random.
Example: detect curvature hidden by a strong trend
Suppose observations follow at equal to 1, 2, 3, 4, while a line predicts The observed values are 1, 4, 9, 16 and predictions are 0, 5, 10, 15. Residuals are
The positive-negative-negative-positive pattern suggests curvature: the line is too low at the ends and too high in the middle. Even a strong positive association between and does not mean a linear model captures the shape adequately.
The appropriate response is to consider a model consistent with the curvature and context, then inspect its residuals. Adding complexity solely to chase every data point can overfit noise.
The residuals are positive at the ends and negative in the middle, revealing curvature missed by the line.
The residuals are positive at the ends and negative in the middle, revealing curvature missed by the line.
Describe this figure
The residuals are positive at the ends and negative in the middle, revealing curvature missed by the line.
Marked points: filled marker at (1, 1); filled marker at (2, -1); filled marker at (3, -1); filled marker at (4, 1).
Outliers and influence are different
A point with a large residual is unusual relative to the model's prediction. A point with an extreme x-value can be influential because it may strongly affect the fitted equation. A high-leverage point can have a small residual and still substantially determine the line.
Investigate unusual points for recording mistakes, special conditions, or meaningful exceptions. Do not delete a point merely because removing it improves a fit statistic. Any exclusion needs a defensible reason tied to the data-generating situation.
For least-squares linear regression with an intercept, residuals sum to zero apart from rounding. That property does not establish a good fit: positive and negative errors can cancel while showing a clear systematic pattern.
Try it yourself
A model predicts 18, 24, and 30 for three observations whose actual values are 20, 21, and 31. Find the residuals. Which prediction overestimates, and by how much?
Answer: Residuals are 2, and 1. The middle prediction overestimates by 3 units. The others underestimate by 2 and 1 units respectively.
What to remember
Use observed minus predicted, interpret signs in context, and inspect patterns rather than only error size. Residuals assess how a model fails, which helps guide a better model.