Module 1 · Foundations

Variation and basic statistics

Every tool in this course is a way of describing variation, and every one of them rests on a handful of ideas: a centre, a spread, a shape, and the difference between the parts you measured and the process that made them. This module builds those ideas from printed data with the arithmetic shown, so that when a piece of software prints a standard deviation you know exactly what it did.

Learning objectives

• Classify a measurement as continuous, a count, or an attribute, and say what each one allows you to compute.
• Compute the mean, median, range, sample variance and sample standard deviation of a dataset by hand, and explain why the standard deviation divides by n − 1.
• Read a histogram, choose a sensible bin width, and recognise skew, truncation and a second mode.
• Use the normal distribution to convert a distance from the mean into a fraction of parts, and recognise the characteristics for which the normal model is wrong.
• State the central limit theorem and show, on data and in a simulation, what it does to the means of subgroups.
• Distinguish a sample from the process that produced it, and an enumerative question from an analytic one.

Why this matters

In 1973 Francis Anscombe published four small datasets in The American Statistician. Each has eleven pairs of x and y. All four have the same mean of x, the same mean of y, the same variances, the same correlation, the same fitted straight line and the same R². Plotted, they are four different stories: a clean linear relationship, a curve, a line with one outlier, and ten identical x-values plus one point that decides everything.^[7][9] The full data are printed in Section 4 and every summary statistic is recomputed there; you will see the numbers agree to the second decimal while the pictures disagree completely.

Anscombe's point is the point of this module. Summary statistics are compressions. They are indispensable, because nobody can read 200 measurements, but they throw away the shape of the data, and the shape is often where the engineering is. A mean and a standard deviation describe a symmetric, single-peaked distribution well and describe a skewed, bounded or two-mode distribution badly, and a capability index computed from the wrong description predicts a defect rate that is simply wrong. So the module works in both directions: the arithmetic of the summaries, so that you can trust them, and the habit of looking at the data first, so that you know when not to.

The second idea that runs through the module is Deming's distinction between an enumerative study, which asks about the items in front of you, and an analytic study, which aims at action on the process that will make the items of the future.^[10] Almost everything you will do as an engineer is analytic. The sample of sixty shafts in Section 2 is not interesting as sixty shafts; it is interesting as evidence about the grinder, and that changes what the statistics are allowed to claim.

1. Types of data

The first question about any measurement is what kind of number it is, because that decides which summaries, which distribution and which control chart apply.

Continuous data carry far more information per part than the other two. A bore measured to 0.001 mm tells you how close it came to the limit; a plug gauge tells you only which side of it the bore fell on. When you have a choice of gauge, measure the value. A rule of thumb from the amount of information involved: a study on attribute data needs something like ten times as many parts as one on variables data to reach the same confidence about the process, which is why Modules 8 and 17 treat attribute data as the fallback rather than the norm.

2. Where the data sit: mean and median

The data below is a constructed example generated by a seeded script, not a real production run. Setting: a ground shaft of Ø8.000 ± 0.015 mm, measured with a micrometer reading to 0.001 mm, 60 consecutive parts in production order.

The median is the middle value when the data are sorted: with 60 values it is the average of the 30th and 31st, here 8.002 mm. The mean and the median agree to a few microns because the data are symmetric. They separate when the data are skewed, and the gap between them is the first and cheapest test of shape: in Section 6 a flatness dataset has a mean of 7.875 µm and a median of 7.2 µm, and that difference is the tail talking.

The mean is the right centre for a symmetric distribution and for anything that will be added up (weights, costs, times). The median is the right centre when a few extreme values would drag the mean somewhere no part actually sits. Report both when they differ, and then look at the histogram.

3. How far the data spread: range, variance, standard deviation

The range is the largest value minus the smallest, 8.012 − 7.994 = 0.018 mm for the shafts. It is simple, it is what a machinist sees, and it is what the R chart in Module 16 uses for small subgroups. Its weakness is that it grows with the sample size and depends on only two values.

The standard deviation uses every value. It is built in three steps: the deviation of each value from the mean, the average of the squared deviations, and the square root to get back to the original units.

Why n − 1 and not n

If you knew the true process mean µ, the average squared deviation from µ over n values would be an unbiased estimate of the process variance, and you would divide by n. You do not know µ; you use x̄, and x̄ is the one number that makes the sum of squared deviations of this sample as small as it can be. The deviations from x̄ are therefore, on average, a little smaller than the deviations from µ, and dividing by n underestimates the variance. Dividing by n − 1 corrects the bias exactly. The usual way of saying it is that one degree of freedom was spent estimating the mean.

For the shafts the two versions are s = 0.00444 mm with n − 1 and 0.00440 mm with n, a difference of less than 1 % at n = 60. At n = 5, the subgroup size of most X̄-R charts, the difference is about 12 %, which is why the choice matters on small samples and why spreadsheet functions come in two versions (in Excel, STDEV.S and STDEV.P). This course always means the n − 1 version when it writes s, and it says so whenever a within-subgroup estimate such as R̄/d₂ is used instead (Module 7).

Two more habits. The variance is in squared units and is the quantity that adds: if two independent sources of variation act on a part, their variances add, their standard deviations do not (Module 15 uses this for tolerance stacks, Module 4 for gauge error). And the standard deviation of the mean of n values, called the standard error, is s/√n; for the shafts 0.00057 mm. That √n is the central limit theorem in Section 8, and it is the reason a control chart on subgroup means has such narrow limits.

Calculator: descriptive statistics and histogram

Pre-loaded with the 60 shaft diameters and the drawing limits. Every value in the output table is one of the statistics defined above, computed from the data you see in the box.

4. Looking at the data: histograms, quartiles, and Anscombe

A histogram divides the range of the data into bins of equal width and draws a bar whose height is the number of values in each bin. It is the single most useful picture in this course and the first thing to draw with any new dataset. Three things to know about it.

• Bin width decides what you see. Too few bins and every distribution looks like a smooth hump; too many and the noise of individual values dominates. A starting rule this course uses is about √n bins, so 8 for 60 values and 14 for 200, then adjust by eye. Bins narrower than the gauge resolution produce comb-like gaps that mean nothing.
• Shape is the message. One peak and symmetric tails: the normal model may do. A long tail to one side: skew, probably from a physical bound (Section 6). A cliff at one edge: sorting or truncation, someone removed the parts beyond a limit. Two peaks: two streams mixed, two machines, two lots, two operators. A histogram cannot tell you which, but it tells you to go and find out.
• It hides time. A histogram of a process that drifted looks like a wide, normal-ish hump. Only a plot against time order shows the drift. Module 9 covers the run chart and Module 16 the control chart; the rule until then is that a histogram is read together with a time plot, never instead of one.

Quartiles give a compact numerical version of the shape. The first quartile Q1 has a quarter of the data below it, the third quartile Q3 three quarters; for the shafts Q1 = 7.998 and Q3 = 8.006 mm, so the middle half of the parts lie within 0.008 mm of each other. The box plot, which draws the quartiles as a box with the median inside and whiskers to the extremes, is Tukey's invention and is the tool of choice for comparing several groups side by side (Module 9).^[12] Several conventions exist for interpolating a quartile between two data values; the calculators in this course use the one that Minitab and Excel's QUARTILE.EXC use, so that your numbers agree with theirs. Skewness, a signed number that is zero for a symmetric distribution, is 0.03 for the shafts and 1.40 for the flatness data in Section 6; the calculator prints it, and a value beyond about ±1 is a strong hint that the normal model will fail.

Worked example 1: Anscombe's quartet, four datasets with one set of statistics

Real data, published by Anscombe in 1973.^[7] The values are as reproduced in two secondary sources,^[8][9] because the original article is available only as a scanned image; the transcription was checked by recomputing every property the paper reports. No units are given in the source.

Section 3's statistics, plus the correlation and fitted line that Module 12 explains, come out the same for all four sets to the precision Anscombe reported. The verification scripts and the regression engine behind Module 12's calculator produce Table 3 from the data above.

Read the four panels against Table 3. Set I is what the statistics describe. Set II is a curve, and a straight line through it is the wrong model however good its R². Set III is a perfect straight line with one point off it, and that one point has bent the fitted line away from the other ten. Set IV has no relationship at all among ten of its points; the eleventh creates the correlation single-handed. An engineer who receives Table 3 without Table 2 or the plots cannot tell which of the four processes is in front of them. This is why every analysis in this course starts with a picture, why Module 9 exists, and why the calculators draw before they compute.

5. The normal distribution and what it lets you do

Many continuous characteristics, when the process is stable and the variation comes from many small independent causes, have a histogram close to the familiar bell. The normal distribution is the mathematical form of that bell. It is defined by two numbers, its mean µ and its standard deviation σ, and its density is

What makes the normal useful is that the fraction of the distribution beyond any point depends only on the distance from the mean measured in standard deviations, the Z-value of Module 0: Z = (x − µ)/σ. There is no closed formula for that fraction,^[1] which is why textbooks carry a Z table, but every calculator in this course computes it directly to better than one part in ten million (the algorithm and its test against scipy are described in the code), so you never need the table. Three values are worth memorising:

Applied to the shafts: the mean is 8.0022 mm and s = 0.00444 mm, so the upper limit of 8.015 sits Z = (8.015 − 8.0022) / 0.00444 = 2.89 standard deviations above the mean, and the lower limit 3.87 below. Both are beyond 2.5σ, so a normal model puts well under 1 % of shafts outside the tolerance in total; the sample shows 0 of 60 outside. Module 7 turns that Z into a capability index. Module 0's Table 1 turns it into a sigma level.

Where the normal model stops

The normal distribution is a model, and three kinds of characteristic break it as a matter of physics, not of bad luck.

• Bounded characteristics. Flatness, runout, roughness, concentricity, leak rate, particle count, time to failure: all are zero or positive, and a good process sits close to zero. A symmetric bell cannot fit against a wall; the distribution piles up near the bound and trails off away from it. Section 6's example is one of these.
• Mixtures. Two machines, two cavities, two lots, two shifts, each roughly normal, produce together a distribution that is wider than either and may have two peaks. Stratification (Module 9) separates them.
• Sorted or truncated output. Parts that were 100 % inspected against a limit show a distribution cut off at that limit. The mean and standard deviation of what was shipped say little about what was made.

The test this course uses to ask "is a normal model reasonable?" is the Anderson-Darling test, whose p-value the calculator prints; Module 11 explains what a p-value is, and for now read a p below 0.05 as "the data depart from the normal shape by more than chance would explain, find out why". The shafts give p = 0.38. The flatness data in the next section give p below 0.001. A histogram and the skewness usually tell you the same thing before the test does, and the physical reason is what you act on.

6. The other distributions an engineer meets

Lognormal, and worked example 2: a bounded characteristic

Constructed data, not a real production run. Setting: the flatness of a milled face on an aluminium housing, measured on a CMM to 0.1 µm, 200 consecutive parts. Flatness cannot be negative and a perfect face would read zero.

The summary statistics: mean 7.875 µm, median 7.2 µm, s = 3.566 µm, minimum 2.1, maximum 25.2, Q1 = 5.425, Q3 = 9.675 µm, skewness 1.40. The mean is above the median, the upper quartile is further from the median than the lower, and the skewness is well above 1: three signs of a right-hand tail before you draw anything. The Anderson-Darling p-value is below 0.001. And a normal model with this mean and standard deviation would place its lower 3σ point at −2.8 µm, a negative flatness, which is the model telling you it does not apply.

A characteristic bounded at zero with a long right tail is often described well by the lognormal distribution: X is lognormal if ln X is normal.^[4] The recipe is to take the natural logarithm of every value and look again. For these 200 values, ln x has mean 1.970, standard deviation 0.435, skewness -0.05 and an Anderson-Darling p-value of 0.98: on the log scale the data are as normal as anyone could ask. The mean of the logs converted back, e^1.970 = 7.17 µm, is the geometric mean, and it sits near the median (7.2 µm) rather than the arithmetic mean (7.875 µm); for a skewed distribution the geometric mean or the median is the honest "typical part".

Module 8 shows what to do with capability for such a characteristic: fit the lognormal or transform, or better, report the fraction beyond the limit from the percentiles of the data themselves. The one thing not to do is to compute mean ± 3s and read a defect rate off it.

Weibull

The Weibull distribution has a shape parameter that lets one family cover distributions from steeply right-skewed to nearly symmetric, and a scale parameter that sets the size; both the Weibull and the lognormal are, in NIST's words, used extensively in reliability applications to model failure times.^[5][6] You will meet it in fatigue and life testing, in the cycles-to-failure of a spring or a switch, and occasionally as a fit for a bounded process characteristic (Module 8). Its arithmetic is not needed in this course; what is needed is to recognise it as the standard answer to "what distribution do lifetimes follow?".

Binomial and Poisson, and worked example 3: what a clean sample proves

Arithmetic on two standard distributions, computed by the verification scripts; not data. The settings are illustrative.

Attribute data follow the binomial distribution: if each of n parts independently has probability p of being defective, the number of defectives in the sample has mean np and standard deviation √(np(1 − p)), and the probability of exactly x defectives is C(n, x) p^x (1 − p)^{n − x}, where C(n, x) counts the ways of choosing which x parts.^[2] Counts of defects follow the Poisson distribution: if defects occur at an average rate λ per unit, the probability of exactly x on a unit is e^−λ λ^x / x!, and the mean and the variance are both λ.^[3] The p and np charts of Module 17 are built on the first, the c and u charts on the second, and Module 6's yield formula e^−DPU is the Poisson probability of zero defects.

Suppose a supplier's lot is running at 2 % defective and you pull a sample of 50. The expected number of defectives is np = 1.0, with standard deviation 0.99. The probabilities:

Read the first row. A sample of 50 from a lot that is 2 % defective contains no defectives 36.4 % of the time. "We checked 50 and found nothing" is therefore weak evidence that the lot is clean: it is what you would see more than a third of the time from a lot that is not. The probability of three or more, 1 − 0.922 = 0.078, is about 8 %, so three defectives in fifty is unusual but not remarkable at 2 %. Sample sizes for attribute decisions are large for exactly this reason. The Poisson columns tell the same story for defects: a board with an average of 1.5 defects comes out clean 22.3 % of the time, and shows three or more 19.1 % of the time.

7. Sample, population, and process

Statistics textbooks speak of a sample drawn from a population, and estimate the population's mean and standard deviation from the sample's. That picture fits a warehouse: 5,000 shafts in stock, 60 pulled at random, and the question "how many of the 5,000 are out of tolerance?" Deming called this an enumerative study: the population exists, is finite, and could in principle be measured completely.^[10]

An engineer's question is almost never that one. It is "what will the grinder make tomorrow, and what should we change?" The 60 shafts are not a sample from a population; they are a sample from a process in time, and the population they are supposed to represent, next week's output, does not exist yet. Deming's term is an analytic study: one that aims at action on the cause system to improve the product of the future.^[10] Two consequences follow, and the rest of the course keeps returning to them.

• No statistical formula can bridge from this sample to the future unless the process stays as it was. A confidence interval on the mean assumes the process has a mean, one value, that persists. If the grinder's wheel is wearing, there is no such value, and the interval is precise about nothing. This is why Module 7 will not report a capability index until a control chart shows the process was stable, and why Module 16 exists.
• How the sample was taken matters more than how big it is. Sixty consecutive shafts from one hour tell you about that hour. Sixty shafts spread over a week with the time recorded tell you whether the hours differ. Module 5 is about designing the sample so that the analytic question can be answered.

8. The central limit theorem

Take any distribution with a mean µ and a standard deviation σ, whatever its shape. Draw subgroups of n values and average each subgroup. The averages have their own distribution, and the central limit theorem says three things about it: its mean is µ; its standard deviation is σ/√n; and its shape approaches the normal as n grows, whatever the parent looked like.^[1] The third statement is the remarkable one, and it is the reason the X̄ chart works on data whose individual values are not normal at all.

Worked example 4: the theorem on the flatness data

The same constructed flatness data as worked example 2, taken in consecutive subgroups of 2, 5 and 10.

The 200 individual values are strongly right-skewed (skewness 1.40) with s = 3.566 µm. Averaging consecutive values gives 100 means of 2, 40 means of 5 and 20 means of 10. Table 7 compares each set of means with what the theorem predicts.

All three predictions hold on printed data. The mean of the means is the mean of the individuals, exactly, because averaging averages is averaging. The spread of the means shrinks as √n: the ratio column stays near 1, and it is not exactly 1 because 20 or 40 means give a rough estimate of a standard deviation. And the shape moves toward the normal: skewness falls from 1.40 to about zero by n = 5, and the Anderson-Darling test, which rejects the individuals decisively, finds nothing wrong with the means of 5 or 10. This is the theorem doing exactly what it promises, on a characteristic that could not be less normal.

Two cautions that Module 16 will repeat. The theorem is about the means, not the parts. A control chart of subgroup means will behave well on this flatness process; a histogram of individual parts against a specification will not, and a capability index that assumes normal individuals will be wrong. And the theorem needs the subgroups to be drawn from one distribution; if the process shifts between subgroups, the means carry the shift, which is precisely what the X̄ chart is designed to detect.

Simulator: draw subgroups from a skewed parent and watch the means

The simulator below draws subgroups from a parent distribution of your choice, plots the distribution of the subgroup means beside the parent, and reports their standard deviation against σ/√n. It also lets you add a shift and see how many subgroups a chart of means takes to notice it, which is Module 16's subject. The random numbers are seeded, so the same seed reproduces the same picture.

Common mistakes

1. Reporting a mean and standard deviation without a histogram. Consequence: Anscombe's four processes look identical, and a skewed or two-mode process is described as a bell. Fix: draw first; the calculator does it for you.
2. Dividing by n on a small sample. Consequence: a standard deviation about 12 % too small at n = 5, and a capability index inflated by the same amount. Fix: n − 1 for a sample; know which function your spreadsheet is using.
3. Adding standard deviations. Consequence: a tolerance stack or a gauge-plus-process error that is far too pessimistic. Fix: variances add; take the square root afterwards (Modules 4 and 15).
4. Applying the empirical rule to a bounded characteristic. Consequence: a predicted lower limit below zero and a defect rate that means nothing. Fix: recognise flatness, runout, roughness and leak rate as skewed by physics; use the log scale or the data's own percentiles (Module 8).
5. Treating a clean small sample as proof. Consequence: "50 checked, none found" is read as zero defectives when a 2 % lot would give the same result 36 % of the time. Fix: compute the binomial probability, and size attribute samples accordingly.
6. Using bins narrower than the gauge. Consequence: a comb-like histogram whose gaps are artefacts of the resolution. Fix: bins at least as wide as the resolution, about √n of them to start.
7. Reading a histogram as if time did not exist. Consequence: a drifting process looks like a wide, stable one. Fix: plot in time order first (Module 9), chart before you summarise (Module 16).
8. Answering an analytic question with an enumerative answer. Consequence: a confidence interval on last week's mean is presented as a forecast of next week's. Fix: show stability before extrapolating, and design the sample to cover the sources of change (Module 5).

Exercises

Exercise 1: descriptive statistics by hand

Constructed data, not a real production run. Setting: the tightening torque of a bolted joint, specified as 12.0 ± 1.0 N·m, measured with a torque analyser reading to 0.01 N·m, 30 consecutive joints.

Tasks. (a) Compute the sum, the mean and the median. (b) Compute the sum of squared deviations, the sample variance and the sample standard deviation, and the population version of the standard deviation; comment on the difference. (c) Find the minimum, maximum, range and quartiles, and describe the shape from these numbers alone. (d) Using a normal model with the sample mean and standard deviation, how many standard deviations away is each specification limit, and roughly what fraction of joints would be out of tolerance? How many of the 30 are?

Exercise 2: the central limit theorem on the shaft data

The constructed shaft diameters of Table 1, taken in twelve consecutive subgroups of five (parts 1 to 5, 6 to 10, and so on).

Tasks. (a) Compute the mean of the first subgroup by hand. (b) Compute all twelve subgroup means and their mean and standard deviation. (c) Compare the standard deviation of the means with s/√5 from Section 3. (d) Which of the two, the individual values or the subgroup means, would you expect to be closer to normal, and why does the answer matter for a control chart?

Quiz

Ten questions. Score 70 % or more to mark the module complete on this device.

Key takeaways

• Continuous data carry the most information; counts follow the Poisson, pass/fail the binomial. Measure the value when you can.
• The mean and the median agree for symmetric data and separate when there is a tail. Report both, then look.
• s divides the sum of squared deviations by n − 1 because the sample mean was estimated from the same data. Variances add; standard deviations do not.
• A histogram with about √n bins is the first picture of any dataset. Anscombe's quartet shows why the picture cannot be skipped: identical statistics, four different processes.
• The normal model gives 68.27 %, 95.45 % and 99.73 % within ±1σ, ±2σ and ±3σ, and it fails by physics for bounded characteristics, mixtures and sorted output. Flatness on a log scale is normal; flatness is not.
• A sample of 50 from a 2 % lot is clean 36 % of the time. Attribute evidence is weak per part.
• The central limit theorem makes subgroup means normal with standard deviation σ/√n whatever the parent; it protects the X̄ chart, not the capability index. And no formula bridges from a sample to the future without stability: that is Deming's analytic problem and the reason for control charts.

References

All web sources accessed 2026-09-09. Sources marked "secondary" were not read in the original by the course author; the claim is taken from the source shown. Books cited by catalogue record are cited only for their existence, edition and general subject.

1. NIST/SEMATECH. "1.3.6.6.1. Normal Distribution." e-Handbook of Statistical Methods. https://www.itl.nist.gov/div898/handbook/eda/section3/eda3661.htm
2. NIST/SEMATECH. "1.3.6.6.18. Binomial Distribution." e-Handbook of Statistical Methods. https://www.itl.nist.gov/div898/handbook/eda/section3/eda366i.htm
3. NIST/SEMATECH. "1.3.6.6.19. Poisson Distribution." e-Handbook of Statistical Methods. https://www.itl.nist.gov/div898/handbook/eda/section3/eda366j.htm
4. NIST/SEMATECH. "1.3.6.6.9. Lognormal Distribution." e-Handbook of Statistical Methods. https://www.itl.nist.gov/div898/handbook/eda/section3/eda3669.htm
5. NIST/SEMATECH. "1.3.6.6.8. Weibull Distribution." e-Handbook of Statistical Methods. https://www.itl.nist.gov/div898/handbook/eda/section3/eda3668.htm
6. NIST/SEMATECH. "8.1.6. What are the basic lifetime distribution models used for non-repairable populations?" e-Handbook of Statistical Methods. https://www.itl.nist.gov/div898/handbook/apr/section1/apr16.htm
7. Anscombe, F. J. "Graphs in Statistical Analysis." The American Statistician 27(1):17 to 21, 1973. https://www.sjsu.edu/faculty/gerstman/StatPrimer/anscombe1973.pdf (scanned copy; the front page was read, the data were transcribed from [8] and checked against the paper's stated properties)
8. Wikipedia. "Anscombe's quartet." https://en.wikipedia.org/wiki/Anscombe%27s_quartet (secondary: source of the transcribed values and the stated shared statistics)
9. R Core Team. "anscombe: Anscombe's Quartet of 'Identical' Simple Linear Regressions." R datasets package documentation. https://stat.ethz.ch/R-manual/R-devel/library/datasets/html/anscombe.html
10. Deming, W. E. "On Probability as a Basis for Action." The American Statistician 29(4):146 to 152, 1975. https://deming.org/wp-content/uploads/2020/06/On-Probability-As-a-Basis-For-Action-1975.pdf
11. Montgomery, D. C. Introduction to Statistical Quality Control, 8th ed. Wiley, 2019. https://www.wiley.com/en-us/Introduction+to+Statistical+Quality+Control,+8th+Edition-p-9781119399308 (further reading; no single claim on this page rests on it) (catalogue record; the standard textbook treatment of the material in this module, including the derivation of the n − 1 correction)
12. Tukey, J. W. Exploratory Data Analysis. Addison-Wesley, 1977. https://www.scirp.org/reference/referencespapers?referenceid=1482121 (catalogue record; origin of the box plot)

Further reading

• Montgomery [11], chapters 3 and 4, for the distributions in this module with their derivations and more worked examples.
• Deming [10] is short, readable, and the clearest statement of why process data are not a random sample from anything.