Module 12 · Analyze

Correlation and regression

A scatter plot (Module 9) shows you there might be a relationship. Regression puts a number on it: a slope, an intercept, a measure of how much of the variation the relationship explains, and — just as important — a set of diagnostics that tell you whether to trust any of it. This module covers correlation and its traps, simple linear regression by hand and by script, residual analysis, an introduction to multiple regression with more than one predictor, and what R² does and does not tell you.

Learning objectives

Why this matters

Anscombe's quartet, introduced in Module 1 and revisited for its graphical lesson in Module 9, has one more lesson to give here, the most directly relevant to this module: all four datasets produce the identical simple regression line (slope near 0.500, intercept near 3.00) and the identical R² (0.67), yet only one of the four is a case where that line is actually the right model.[2] A regression's summary numbers — slope, intercept, R², even the slope's own p-value — cannot tell you this by themselves. Residual analysis is the tool that can, and NIST's e-Handbook is blunt about the limits of the summary statistics alone: "a high R² value does not guarantee that the model fits the data well."[1]

The scenario below is a constructed illustration, not a real incident.

A team investigating braze fillet width finds a moderate correlation with furnace temperature and, pleased with an R² over 0.45, recommends a temperature-only process control limit and closes the investigation. A more complete model, adding two more readily available process variables, very nearly doubles the explained variation — and reveals that one of the two new variables the team assumed mattered (flux mass) actually does not, once temperature and joint gap are accounted for. Both the celebration after the first model and the assumption behind the second variable were premature; only the fuller regression, checked properly, sorts out which relationships are real.

Correlation and its traps

The Pearson correlation coefficient r measures the strength of a linear relationship only, on a scale from −1 (perfect negative) to +1 (perfect positive), with 0 meaning no linear relationship — not necessarily no relationship at all. Three traps recur constantly in practice: r can be near zero for a strong curved relationship (Anscombe's set 2 again); a real correlation does not establish which variable causes which, or whether a third, unmeasured variable drives both; and computing r over a restricted range of one variable (only the parts of a process that happen to have run recently, say) can understate or overstate the true relationship across the variable's full range.

Simple linear regression

ŷ = b₀ + b₁x
b₁ = Sxy / Sxx = Σ(x−x̄)(y−ȳ) / Σ(x−x̄)²    b₀ = ȳ − b₁x̄ Least squares: the line that minimizes the sum of squared vertical distances from each point to the line. b₁ is the predicted change in y per unit change in x; b₀ is where the line crosses x = 0, which may or may not be a meaningful point depending on whether x = 0 falls inside the range of the data.

The slope's own standard error, t-statistic and p-value (Module 11's logic applied to one coefficient) test whether the slope is distinguishable from zero — whether there is a linear relationship at all, not whether the model is a good one.[3]

Residual analysis

A residual is the difference between an observed y and the model's fitted value at that x: e = y − ŷ. NIST's e-Handbook treats a plot of residuals against fitted values (or against x) as the primary check of a regression model, ahead of any summary statistic: it should show no pattern — no curve (a sign the true relationship is not linear), no funnel shape (a sign the variance is not constant), no trend over the order the data was collected (a sign of a missed time-based effect), and residuals that look reasonably close to a normal distribution.[1] Any of these patterns means the model's own assumptions do not hold, and the p-values and confidence intervals it reports are not trustworthy even if R² looks respectable.

Multiple regression

ŷ = b₀ + b₁x₁ + b₂x₂ + … + bₖxₖ Each coefficient bi is the predicted change in y per unit change in xi, holding every other predictor in the model constant — a genuinely different quantity from the slope of a simple regression of y on xi alone, because the simple slope does not control for the other predictors at all. Fitting is still least squares, solved from the normal equations XᵀXb = Xᵀy, where X is the design matrix (a column of 1s for the intercept, plus one column per predictor).

Adding a predictor to a model can never decrease R² (it can only help the least-squares fit, or do nothing), which makes plain R² a poor way to decide whether a new predictor is worth keeping. Adjusted R² corrects for this by penalizing extra predictors relative to the residual degrees of freedom, and is the more honest number to watch when comparing models with different numbers of predictors.

adj = 1 − (1 − R²) × (n − 1) / (n − k − 1) n = number of observations, k = number of predictors. If a predictor adds essentially nothing, adjusted R² barely moves or can even fall slightly, even though plain R² always ticks up by some small amount.

What R² does and does not tell you

R² is the proportion of the variation in y explained by the fitted model. It does not tell you the model's functional form is correct (Anscombe's set 2 again: a curved relationship can still produce a large R² from a straight-line fit, or a small one, depending on the data). It does not tell you the relationship is causal. It does not by itself tell you whether the effect is practically significant — a very large sample can make a tiny, unimportant slope statistically significant with a small R², and a small sample can leave a practically important relationship statistically inconclusive. And it never falls when a predictor is added, which is exactly why R² alone is not a valid way to compare a simple model against a fuller one; that comparison needs adjusted R² or the F-test on the additional predictors, not R² on its own.

Worked examples

Worked example 1: residual plots, Anscombe's quartet revisited

Real data (Module 1, S-C28), the same four datasets, examined here through their residuals.

Module 1 introduced these four sets; Module 9 looked at their scatter plots. Here, the same regression line is fitted to each (slope ≈ 0.500, intercept ≈ 3.00, R² ≈ 0.67 for all four) and the residuals — what the scatter plot's departure from that line actually looks like, quantified — are plotted against x.

Residual plots for Anscombe's four datasets Four small scatter plots of residual against x, one per Anscombe dataset, arranged in a two by two grid, all on the same horizontal and vertical scale with a dashed zero line. Set 1's residuals scatter with no visible pattern. Set 2's residuals trace a clean, symmetric downward-opening parabola, rising then falling as x increases. Set 3's residuals are all small and close to zero except one far above the zero line. Set 4's residuals are a vertical column of points at one x value plus a single point at a much larger x sitting almost exactly on the zero line. Set 1: residuals scatter with no pattern Set 2: a clear parabola Set 3: one large outlier Set 4: one point defines the whole line
Figure 1. Residuals (vertical axis, dashed line at zero) against x (horizontal axis), same scale in all four panels, for Anscombe's four datasets fitted with the identical regression line. The summary statistics cannot tell these four apart; the residual plots do it immediately.

Set 1's residuals scatter with no visible structure — a straight line is a reasonable model. Set 2's residuals trace a clean parabola: the true relationship is curved, and the linear fit's errors are not random noise, they are the shape of the curve the line missed. Set 3 is dominated by a single point (residual 3.241, far above every other point on that panel); without it the remaining ten points would fall almost exactly on a line, and the "0.67 R²" the summary statistics report is really one outlier's doing. Set 4 shows why: ten points sit at the same x with no information about a slope at all, and the entire regression line — slope, intercept, R² — is determined by the single point at x = 19, whose residual is essentially 0.0 because the line is, in effect, drawn straight through it and the other ten points' mean. None of this is visible in the slope, the intercept, or R² alone; all of it is visible in thirty seconds of looking at the residuals.

Worked example 2: simple vs multiple regression, braze fillet width

The data below is a constructed example, not a real production run.

Braze fillet width (mm) at the header joint, 24 joints, three candidate predictors: furnace temperature (°C), flux mass (mg), and joint gap (mm). First, temperature alone.

ŷ = -2.806 + 0.00670 × temperature r = 0.690, R² = 0.477, slope p-value 0.0002 (significant), residual s = 0.0390 mm.

Temperature alone explains R² = 0.477 of the variation in fillet width, about 48 %. Adding flux mass and joint gap:

Table 1. Multiple regression, fillet width vs temperature, flux mass, and joint gap (constructed data, n = 24).
TermCoefficientSEtp95% CI
Intercept-3.8690.663-5.830.00001−5.253 to −2.486
Temperature0.008530.001127.620.00000020.00619 to 0.01086
Flux mass0.006160.003901.580.1295−0.00197 to 0.01429
Gap-1.0830.212-5.100.00005−1.525 to −0.640
R² = 0.773    R²adj = 0.739    F(3, 20) = 22.69, p 0.000001    s = 0.0269 mm

Adding two predictors nearly doubles the explained variation, from about 48 % to 77 % (R² = 0.773). Both temperature and gap are highly significant, and gap's coefficient is negative — a wider gap means a narrower fillet, which matches the physical picture of a fixed amount of filler metal bridging a larger opening. Flux mass, though, has a p-value of 0.129: not statistically significant at the conventional 5 % level, and its 95 % confidence interval comfortably includes zero. A team that measured only flux mass and fillet width, and computed just their simple correlation, could easily have seen a moderate relationship and assumed flux mass mattered; in the full model, controlling for temperature and gap, that apparent effect mostly evaporates. This is precisely what "holding the other predictors constant" buys: a fairer read of whether each variable earns its place, not just how each one looks in isolation.

Show the 24 braze joints (temperature, flux mass, gap, fillet width)
Table A1. Braze fillet width (mm) with furnace temperature (°C), flux mass (mg) and joint gap (mm), 24 joints (constructed data). Worked example 2's simple regression uses the temperature and width columns only.
tempfluxgapwidthtempfluxgapwidth
598.412.660.0561.2469598.210.720.0551.1928
608.511.240.1331.2768602.313.10.111.2373
595.510.770.0641.232604.313.110.1451.1518
591.213.730.1221.1548602.410.580.1041.2116
596.210.210.0941.1649598.78.770.0521.241
604.411.880.11.2173598.812.990.1081.2256
605.611.260.1011.2633602.910.990.0981.2803
600.88.670.0671.2537608.411.340.1191.2618
596.29.010.0561.2166591.011.050.0621.1544
608.312.90.1141.2513591.88.220.1171.0924
608.611.140.061.3406605.211.080.1431.2045
598.712.070.0571.2416599.28.580.0721.1866

Worked example 3: a correlation that stratification reverses

The data below is a constructed example, not a real production run.

Setting: a powder coating line. Panels are sprayed through one of two spray nozzles, a standard nozzle (A) and a high-flow nozzle (B), and coating thickness is measured on each panel in micrometres. The high-flow nozzle lays down more material, so the line is run faster when it is fitted. Someone pulls 40 panels' records out of the system, pairs thickness against line speed, and fits a regression, without recording which nozzle sprayed which panel.

Table 2. Nozzle A: line speed (m/min) and coating thickness (µm), 20 panels (constructed data).
speedthicknessspeedthickness
24.441.723.242.2
24.541.219.942.1
22.139.921.543.1
20.343.325.838.6
18.44325.239.8
21.142.624.839.5
21.342.521.144
18.442.221.942.6
18.443.923.443.6
2639.318.544.8
Table 3. Nozzle B: line speed (m/min) and coating thickness (µm), 20 panels (constructed data).
speedthicknessspeedthickness
34.458.635.756.9
32.259.93060.6
3755.63458.1
30.56033.557.4
35.458.531.660.3
3755.832.657.5
31.859.536.455.3
37.256.132.557.6
3756.831.260.7
30.159.935.656.7

Pooling all 40 panels and fitting one line gives a strong, highly significant positive relationship:

Pooled (n = 40):   ŷ = 20.020 + 1.0762 × speed
r = 0.835,   R² = 0.697,   slope p-value 0.000000000,   residual s = 4.645 µm Read literally: every extra metre per minute of line speed buys about 1.08 µm of coating. An engineer who wanted a thicker coat and believed this number would speed the line up.

Now fit the same relationship separately within each nozzle:

Table 4. The same 40 panels, pooled and stratified by nozzle (constructed data).
FitnSlope (µm per m/min)rSlope p-valueMean speedMean thickness
Pooled, nozzle ignored401.07620.8350.6970.00000000027.9050.04
Nozzle A only20-0.5202-0.7630.5810.00009322.0142.00
Nozzle B only20-0.6339-0.8950.8000.00000010333.7958.09
The same 40 panels, pooled and stratified by nozzle Two scatter plots of coating thickness in micrometres (vertical axis) against line speed in metres per minute (horizontal axis), same scales in both. Left: all forty panels with the nozzle identity discarded, and one fitted line rising steeply from lower left to upper right. The points form two separated clouds, one at low speed and low thickness and one at high speed and high thickness, and the fitted line runs through the gap between them. Right: the same points with nozzle A drawn as circles and nozzle B as squares, each with its own fitted line; both of those lines slope gently downward, in the opposite direction to the pooled line. Pooled: thickness rises with speed Stratified by nozzle: it falls 20253035402025303540405060405060 nozzle A (circles) nozzle B (squares) line speed (m/min) line speed (m/min) coating thickness (µm)
Figure 3. Powder coating thickness against line speed for the same 40 panels, pooled (left) and stratified by spray nozzle (right). The pooled line and the two within-nozzle lines have opposite signs. Constructed data.

Both within-nozzle slopes are negative, both are statistically significant, and both agree with the physics: at a higher line speed a panel spends less time in front of the gun, so it receives less powder. The pooled slope has the opposite sign, and it is the more significant of the three. Nothing is wrong with the arithmetic in any of the three fits. What is wrong is the question the pooled fit answers.

The mechanism is visible in the last two columns of Table 4. Nozzle A runs at a mean speed of 22.01 m/min and lays 42.00 µm; nozzle B runs at 33.79 m/min and lays 58.09 µm. The pooled line is not measuring what line speed does to thickness at all. It is drawing a line between two nozzles that differ in both variables at once, and the nozzle is doing all the work. This is the continuous-data form of Simpson's paradox: a relationship within every subgroup that reverses when the subgroups are combined.

The defence is not statistical, it is procedural, and Module 9 already named it: stratify before you fit. Any variable that could plausibly shift both x and y — nozzle, machine, shift, batch of raw material, operator, gauge, season — is recorded with the data and used to split it. If the split changes the answer, the split was the answer. A lurking variable that was never recorded cannot be stratified on afterwards, which is why the operational definitions and data collection sheet of Module 5 matter more to a regression than any of the arithmetic in this module. The multiple regression of Worked example 2 is the other tool for the same problem: had the nozzle been recorded as an indicator variable and put in the model alongside speed, the fit would have reported the within-nozzle slope and the nozzle offset separately, which is the honest decomposition of Figure 3's right panel.

Common mistakes

  1. Fitting a regression without ever looking at the scatter plot first. Consequence: Anscombe's quartet, again — identical regression output, at least one of the four datasets is the wrong model for its data. Fix: Module 9's discipline applies here directly; plot before fitting.
  2. Trusting R² alone and skipping the residual plot. Consequence: a curved relationship, a funnel-shaped variance, or a time trend can all sit underneath a respectable-looking R² and go unnoticed. Fix: always plot residuals against fitted values before trusting any regression output.
  3. Reading a correlation or a regression coefficient as proof of causation. Consequence: acting on a relationship driven by a third, unmeasured variable, or by reverse causation, produces a "fix" that does not fix anything. Fix: causal claims need a designed experiment (Module 13) or strong process knowledge, not a regression coefficient by itself.
  4. Interpreting a multiple regression coefficient the same way as a simple regression slope. Consequence: reporting "a 1 mg increase in flux mass changes fillet width by b" when b is actually "holding temperature and gap constant" is a different, narrower claim, and conflating the two overstates what the model says. Fix: state the "holding other predictors constant" qualifier every time.
  5. Using plain R² to decide whether an added predictor is worth keeping. Consequence: R² can only rise or stay flat when a predictor is added, even a useless one, so it always looks like an improvement. Fix: compare adjusted R², or test the added predictor's own coefficient directly.
  6. Assuming a predictor matters because its simple correlation with y is large. Consequence: Worked example 2's flux mass has a real, moderate simple correlation with fillet width, but is not a statistically significant predictor once temperature and gap are in the model — its apparent effect was substantially explained by its association with the other two variables. Fix: check a predictor's significance in the full model you intend to use, not its marginal correlation alone.
  7. Extrapolating a fitted line or surface outside the range of the data used to fit it. Consequence: the intercept and slope describe the fitted range; nothing guarantees the same relationship holds outside it, and a process pushed to a new operating point can behave completely differently. Fix: state the range the model was fitted over, and treat any prediction outside it as an extrapolation, not a result.

Exercises

Exercise 1: wire-bond pull strength vs bond force and bond time

Constructed example, not a real production run.

Wire-bond pull strength (N), 20 bonds, two candidate predictors: bond force (g) and bond time (ms). Tasks. (a) A simple regression of pull strength on bond force alone gives R² = 0.876. Predict, before computing, whether adding bond time will raise R² even if bond time turns out to matter very little. (b) Fit the multiple regression and report both coefficients' p-values. (c) Does bond time earn its place in the model, statistically? Compare this result with Worked example 2's flux mass.

Show the worked solution

(a) R² must rise by at least a small amount regardless, because adding any predictor to a least-squares fit can never decrease R². The real question is whether it rises by enough to matter, and whether adjusted R² agrees.

(b) Force: coefficient 0.0949, p 0.000000000. Time: coefficient 0.0313, p 0.003. R² = 0.928, R²adj = 0.919.

(c) Yes: bond time's p-value (0.003) is well under 0.05, and adjusted R² rose along with plain R² (from the one-predictor model's adjusted R² of 0.869 to 0.919), not just plain R² alone. This is the opposite outcome from Worked example 2's flux mass: there, a second predictor's simple relationship with y turned out not to hold up once the other predictors were in the model; here, bond time's effect is real and statistically supported even after controlling for bond force. The two examples together are the point: adding a predictor always raises R² a little, but only checking its own coefficient and adjusted R² tells you whether that rise means anything.

Show the 20 wire bonds (force, time, pull strength)
Table A2. Wire-bond pull strength (N) with bond force (g) and bond time (ms), 20 bonds (constructed data). Exercise 1's simple regression uses the force and strength columns only.
forcetimestrengthforcetimestrength
45.430.08.1552.625.39.0
58.520.68.7246.117.38.1
34.324.76.839.927.57.29
58.534.69.3353.730.59.12
39.434.27.339.127.37.33
42.729.57.8343.633.38.31
54.825.88.9334.015.86.5
42.320.57.742.125.68.07
46.518.27.8536.124.27.18
30.834.47.0737.916.26.79

Exercise 2: a correlation of almost zero over a strong relationship

Constructed example, not a real production run.

Setting: lap-shear strength (MPa) of a structural adhesive joint against cure temperature (°C), 18 joints cured at temperatures spread across the supplier's stated 120 to 180 °C window, one joint per temperature, pulled to failure on a load frame reading to 0.01 MPa.

Table 5. Cure temperature (°C) and lap-shear strength (MPa), 18 joints (constructed data).
tempstrengthtempstrength
12822.2316523.8
12922.6516523.73
12921.9117023.01
13222.3117022.52
14023.2517021.63
14624.2917122.62
15123.9317621.54
15522.917721.42
16123.3217722.22

Tasks. (a) Fit a simple linear regression of strength on temperature and report the slope, r, R² and the slope's p-value. (b) On that evidence alone, what would you conclude about whether cure temperature affects strength? (c) Plot the residuals against temperature and say what the plot shows. (d) State what an r of nearly zero does and does not license you to say, and what you would do next.

Show the worked solution

(a) Slope -0.0070 MPa per °C, r = -0.146, R² = 0.0213, slope p-value 0.563, residual s = 0.875 MPa. The line is flat and the fit explains about 2 % of the variation.

(b) On the linear fit alone the temptation is to conclude "cure temperature does not affect strength, so we can cure anywhere in the window." That conclusion is wrong, and the exercise exists to make the error concrete. A p-value of 0.563 is not evidence of no effect; it is a failure to find a linear effect, and the null hypothesis being tested is about the slope of a straight line, nothing more (Module 11: never read a large p-value as proof of no difference).

(c) The residuals form a clean inverted parabola: negative at both ends of the temperature range and positive in the middle. That is the signature of a relationship with an optimum, and a straight line is the wrong model for it. Fitting temperature and its square as two predictors (Module 12's multiple regression, with temperature² simply computed as a second column) gives R² = 0.691, adjusted R² = 0.650, F(2, 15) = 16.79 with p = 0.000149, residual s = 0.5074 MPa, and both terms significant (temperature p = 0.000045, temperature² p = 0.000042). The quadratic's maximum sits at −b₁ / 2b₂ = 151.1 °C, near the middle of the window, and the residual standard deviation falls from 0.875 to 0.5074 MPa.

(d) An r of nearly zero licenses one statement: over this range, there is no useful straight-line relationship between these two variables. It does not license "there is no relationship", "temperature does not matter", or "cure anywhere in the window." What to do next: plot the raw scatter (Module 9's rule, which would have shown the curve before any arithmetic), then treat the cure temperature as a factor with an interior optimum, which is a response surface question rather than a regression one — Module 13's introduction to response surface methods is exactly the tool, and the practical answer for the process is to set the cure at the top of the curve and specify a window around it, not to leave the supplier's full range open.

Quiz

Ten questions. Score 70 % or more to mark the module complete on this device.

1. A correlation coefficient r near zero means
2. The primary purpose of a residual plot is to
3. In Worked example 1's residual plots, Anscombe's set 2 showed
4. In a multiple regression, a coefficient bᵢ is interpreted as
5. Adding a predictor to a regression model, even a useless one, causes R² to
6. Adjusted R² differs from plain R² because it
7. In Worked example 2, flux mass's p-value of 0.129 in the full model means
8. A regression coefficient, however statistically significant, does NOT by itself establish
9. Using a fitted regression to predict y at an x value well outside the range of the fitting data is
10. NIST's e-Handbook (4.4.4) states, on the limits of summary statistics, that
Answer key
  1. c. No strong linear relationship; a curve can still exist.
  2. b. Check whether the model's assumptions hold.
  3. d. A clear parabola.
  4. a. Change in y per unit x, holding others constant.
  5. c. Never decreases.
  6. b. Penalizes extra predictors.
  7. d. Not significant once temperature and gap are included.
  8. a. Causation.
  9. c. An extrapolation.
  10. b. A high R² does not guarantee a good fit.

Key takeaways

References

All web sources accessed 2026-09-09 unless noted.

  1. NIST/SEMATECH. e-Handbook of Statistical Methods, 4.4.4 "How can I tell if a model fits my data?". NIST. https://www.itl.nist.gov/div898/handbook/pmd/section4/pmd44.htm
  2. Anscombe, F. J. (1973). Graphs in statistical analysis. The American Statistician, 27(1), 17–21. Values as reproduced in Wikipedia and the R datasets package documentation (Module 1, S-E35, S-E36). https://www.sjsu.edu/faculty/gerstman/StatPrimer/anscombe1973.pdf
  3. NIST/SEMATECH. e-Handbook of Statistical Methods. NIST, 2012 (updated). https://www.itl.nist.gov/div898/handbook/
  4. Montgomery, D. C. (2019). Introduction to Statistical Quality Control, 8th ed. Wiley. https://www.wiley.com/en-us/Introduction+to+Statistical+Quality+Control,+8th+Edition-p-9781119399308 (catalogue-level; general regression reference, further reading)
  5. ASQ. Certified Six Sigma Green Belt (CSSGB) Body of Knowledge Map 2014–2022. ASQ, 2022. https://www.asq.org/cert/resource/pdf/certification/2022-CSSGB-BoK-Map.pdf (the Green Belt scope this module is written against; no single claim rests on it)