Least-squares regression summarizes a linear relationship between two quantitative variables. It predicts a response from an explanatory variable by choosing the line that minimizes the sum of squared vertical residuals. The line is a model of association, not automatically a causal law.
Read the fitted equation in context
A regression line has form where is the predicted response, is slope, and a is intercept. The slope's units are response units per explanatory-variable unit.
For with measured in study hours and in score points, one additional study hour is associated with a predicted increase of 3 points. The intercept predicts 12 points at zero hours, but that interpretation is useful only if zero is meaningful and reasonably supported by the data range.
Do not read the equation backward without fitting a different regression. The least-squares line predicting from is generally not the algebraic inverse of the line predicting from .
Residuals measure prediction errors
A residual is observed response minus predicted response. A positive residual means the point lies above the line and the model underpredicted. A negative residual means the model overpredicted.
For a student with 5 study hours and score 31, the prediction is so the residual is 4 points. The residual is a vertical distance, not the shortest perpendicular distance to the line.
Least squares minimizes the sum of squared residuals. Squaring prevents cancellation and penalizes large discrepancies strongly, which also makes the fitted line sensitive to influential observations.
Correlation and slope describe different scales
Correlation measures the direction and strength of a linear relationship. It is unitless and lies between and 1. Slope has units and depends on the scales of both variables.
The formulas connecting summary statistics are
Thus the least-squares line passes through and slope has the same sign as correlation. Changing units can change slope while leaving correlation unchanged. A correlation near zero rules out a strong linear relationship, not every possible curved relationship.
Example: calculate a line from summaries
Suppose and The slope is and the intercept is Therefore
At the predicted response is 56. If the actual response is 52, the residual is . The line passes through as required, giving a quick consistency check on the calculations.
Interpret r-squared as explained variation
For simple linear regression with an intercept, is the proportion of variation in the response accounted for by the linear model using . In the example, so about 56.25% of the variation in is accounted for by its linear relationship with .
This is not the percentage of predictions that are correct, the percentage of points on the line, or the percentage of caused by . The remaining variation appears in residuals, but it need not arise from one identifiable source.
Read computer output and diagnostics
A coefficient table typically lists the intercept and x-coefficient estimates, standard errors, and t statistics or P-values. The coefficient attached to is the slope; do not confuse it with correlation or The residual standard deviation, often labeled , summarizes typical vertical prediction error in response units.
A residual plot should show no systematic curve or changing spread if a straight-line, constant-spread model is reasonable. Curvature suggests a missing nonlinear pattern; a funnel suggests changing variability. High-leverage points have unusual x-values and can strongly affect the line, especially when their responses do not follow the rest of the pattern.
Predictions far outside the observed x-range are extrapolations and may be unreliable even when is large. Association from observational data does not establish causation, because lurking variables can influence both quantities.
Try it yourself
A regression predicts At the observed is 71. Find the prediction and residual, and state the sign of correlation.
Answer: The prediction is 68 and the residual is so the model underpredicted. Correlation is negative because the slope is negative. Its magnitude cannot be recovered from slope alone without the variables' standard deviations.
What to remember
Interpret slope with units, calculate residual as observed minus predicted, and read and as linear-association summaries. Inspect residuals, influential points, and the data range before trusting a prediction.