Module 14 · Improve

Lean improvement tools

Analyze (Modules 9 through 12) finds and confirms a cause. Improve turns that finding into a change that actually sticks. This module covers the standard Lean toolkit for doing that on a shop floor: 5S and standard work as the stable foundation everything else depends on, poka-yoke for mistake-proofing, SMED for cutting changeover time, kaizen events for running a focused improvement effort, and a Pugh matrix for choosing among competing concepts. The last section is the one that matters most: how to pilot a change and prove it worked with a hypothesis test and a control chart, not a before-and-after bar chart.

Learning objectives

• Explain why 5S and standard work are a precondition for the rest of this toolkit, not an optional first step.
• Describe poka-yoke's role in preventing or catching a defect at its source, and connect it to Module 10's PFMEA Detection rating.
• Apply SMED's internal/external setup distinction to cut a changeover time, and test the result with a two-sample t-test.
• Run a focused kaizen event and explain how it differs from an open-ended suggestion program.
• Build and interpret a Pugh matrix to select among improvement concepts.
• Prove a pilot improvement with the correct hypothesis test and a control chart of each period, not a before/after average alone.

Why this matters

In 2007, BusinessWeek reported on what happened when 3M applied Six Sigma discipline broadly across the company under CEO James McNerney, including to research: thousands of scientists trained as Black Belts, standardized stage-gate metrics applied to early-stage research, and, over the same years, the share of 3M's sales coming from products introduced in the previous five years falling from about a third to about a quarter.^[1] One 3M scientist told the magazine plainly: "Invention is by its very nature a disorderly process."^[1] The article does not claim Six Sigma alone caused the decline — other explanations were offered too — but it is a documented, widely cited case for a real distinction this module asks you to keep in mind throughout: 5S, standard work, and SMED are built for stabilizing and speeding up a process that is already supposed to produce the same result every time. Pointed at a repeatable production line, they are close to unambiguously good. Pointed at genuinely exploratory work, the same standardizing instinct can suppress the variation that the work actually needs.

The scenario below is a constructed illustration, not a real event.

Almost every Lean tool in this module can also fail for a much more mundane reason: a team runs a three-day kaizen event, moves some racks, times a changeover, feels good about it, and closes the event with a poster showing a bar chart — one bar for "before," one shorter bar for "after." Nobody checked whether the drop was bigger than ordinary day-to-day variation would produce anyway, and nobody checked whether either period was itself stable. Three months later, someone asks whether the change is still in effect, and nobody can say for certain, because the "before" and "after" bars were never anything more than two averages. Everything in this module's tool sections is standard and well documented. The last section, on proving an improvement, is where most real kaizen efforts actually go wrong.

5S and standard work

5S is a workplace organization method in five steps, usually given as Sort (remove what is not needed), Set in order (a place for everything, arranged for the sequence of use), Shine (clean and inspect), Standardize (make the first three a visible, repeatable routine), and Sustain (audit and maintain it).^[2] Treated as a one-time cleanup, 5S produces a tidy photo and nothing else; treated as Standardize and Sustain intend, it is a recurring discipline that keeps a workstation in a known state, which is what makes every other tool in this module possible to apply consistently.

Standard work is the companion discipline for the process itself rather than the workstation. The Lean Enterprise Institute defines it as establishing precise procedures for each operator's work in a production process, built from three elements: takt time (the rate at which product must be completed to meet demand), work sequence (the exact order of tasks within that time), and standard in-process stock (the minimum units needed to keep the sequence running smoothly).^[3] Standard work is also, quietly, a precondition for everything Modules 16 and 17 teach about control charts: a control chart's "common cause" variation is supposed to be the process's inherent noise, not the noise of three operators each doing the job a different way. Skip standard work, and a chart's control limits partly measure method variation you could have eliminated for free.

Poka-yoke

Poka-yoke (mistake-proofing) is Shigeo Shingo's term for a device or process design that makes a specific human error either physically impossible or immediately self-evident, rather than relying on inspection or attention to catch it afterward.^[4] Shingo describes three general mechanisms: a contact method (a physical feature, like an asymmetric locating pin, that only allows correct assembly), a fixed-value method (a counter or a kit that flags a missing or extra step by quantity), and a motion-step method (a sequence sensor that will not let the next step start until the current one is confirmed).^[4] This maps directly onto Module 10's PFMEA Detection rating: a poka-yoke that prevents the error is functionally driving Occurrence toward zero, while one that only flags the error after the fact is driving Detection toward 1 (catches it every time) — both lower RPN, but prevention is the stronger fix.

SMED: Single-Minute Exchange of Die

SMED, developed by Shingo largely from his consulting work in Toyota-group plants, is a method for cutting equipment changeover time.^[5] Its core move is a single distinction, applied rigorously: separate internal setup (steps that require the machine or furnace to be stopped) from external setup (steps that can be done while it is still running), then (1) convert as much internal work to external as physically possible — staging tooling, pre-heating a fixture, gathering fasteners, all done before the stop — and (2) streamline whatever internal work is left, typically with quick-release fasteners, guided locating features, and standardized settings instead of adjustment. Shingo reports changeovers cut from hours to single-digit minutes at several plants using this method; those specific figures come from his own book rather than an independently audited source, so they are presented here as Shingo's reported results, not independently verified facts.^[5] Worked example 1 below applies the same two-step method to a furnace-fixture changeover and checks the result with a hypothesis test rather than taking the improvement on faith.

Kaizen events

Kaizen, from Masaaki Imai's book of the same framing, is continuous, incremental improvement as everyone's ongoing responsibility, not a program run by a separate department.^[6] A kaizen event (or kaizen blitz) is a specific, time-boxed application of that idea: a small cross-functional team, a single clearly scoped process, typically three to five days, working from data already collected before the event starts rather than opening with a blank-page brainstorm. That last point is the one teams skip most often. An event that starts with "what do we think is wrong" instead of "here is what the data from the last month shows" tends to converge on whoever argues most persuasively in the room, which is a poor substitute for the root-cause work Module 10 already covers.

Solution selection: the Pugh matrix

Stuart Pugh's concept-selection matrix compares several candidate solutions against one reference concept (the current state, or the most straightforward candidate) across a shared set of criteria.^[7] Each candidate is scored, criterion by criterion, relative to the baseline: better (+), the same (S), or worse (−). The scores for a concept sum to a net total, and concepts can be ranked by that total.

Two cautions matter more than the arithmetic. First, the matrix is a facilitation tool for structuring a conversation among people who often disagree, not an algorithm that removes judgment from the decision: a close net score, or a criterion the team privately weights far more heavily than the others, both argue for discussion rather than mechanically picking the top row. Teams that want criteria weighted unequally can multiply each score by a weight before summing (a weighted Pugh matrix); this course uses the simpler unweighted, pure-summation version throughout. Second, a concept can win on a single important criterion and still lose overall once every criterion is counted — Worked example 2 below shows this on real numbers, and it is exactly the situation a matrix is built to surface.

Piloting and proving an improvement with data

A before-and-after comparison built from two averages, or worse, a bar chart with one bar per period, cannot distinguish a real change from ordinary sampling variation. Module 11 covers why in full: even a process with no true change produces two different-looking sample averages essentially every time it is measured twice. A defensible pilot does four things in order. First, pilot the change at a small, contained scale before committing to a full rollout — a single cell, line, or shift, not the whole plant. Second, collect before and after data under comparable conditions, ideally the same measurement system and the same range of normal operating conditions. Third, test the difference with the hypothesis test that matches the data (a two-sample t-test for a measured quantity, a chi-square or two-proportion test for a defect rate) and report an effect size or a practically meaningful number, such as a relative risk or a mean difference, alongside the p-value — a p-value alone says a difference is unlikely to be noise, not that the difference is large enough to matter. Fourth, check each period on its own with a control chart (Modules 16 and 17), because a period average can be pulled by one unusual day without the process's actual level having moved at all; a chart is what tells the two apart. Only after a pilot clears all four does a full-scale rollout, re-confirmed the same way, belong on the table — which is exactly what the capstone (Module 19) will do at full scale, building on the smaller pilot this module's Worked example 3 pilots first.

Worked examples

Worked example 1: furnace-fixture changeover time, before and after SMED

The data below is a constructed example, not a real production run. Setting: time in minutes to change the braze-furnace fixture between part numbers, timed to the nearest tenth of a minute, 16 changeovers before and 16 after separating internal from external setup steps (staging the next fixture and pre-heating it externally, and replacing a bolted mount with a quick-release clamp for the one step that has to happen with the furnace stopped).

Mean before 68.54 min (s = 11.80), mean after 26.77 min (s = 4.29), a difference of 41.78 min. Before testing, check the equal-variance assumption behind the standard pooled t-test: Levene's test gives p = 0.0084, well under 0.05, so the two periods' variances are not equal (the after period is both faster and more consistent, which makes sense — a quick-release clamp removes a source of adjustment-to-adjustment variation along with time). The textbook response is to prefer Welch's t-test, which does not assume equal variances: t = 13.31 on 18.9 degrees of freedom (Welch's own adjusted df, versus 30 for the pooled test), 95 % CI on the difference 35.20 to 48.35 min, p < 0.0001. Cohen's d = 4.70, an enormous effect size by any convention. Here the equal-variance violation changes which test is technically correct but not the practical conclusion, because the effect is so large that both the pooled test (p < 0.0001) and Welch's test agree; for a smaller, more marginal effect, checking this assumption before trusting the p-value can matter a great deal more than it does here.

Worked example 2: Pugh matrix, a furnace-fixture redesign

Constructed example, not a real selection workshop.

Module 10's root-cause work traced leak-test rejects partly to furnace-braze joint gap. A solution-selection workshop scores four fixture-redesign concepts against the current pin fixture on five criteria: how much the concept improves joint-gap repeatability, its effect on changeover time, its unit cost, operator ergonomics, and maintainability.

The modular quick-change concept wins on net score (+3), and notice what that number is doing: it is not the cheapest option (both it and the spring-loaded clamp cost more than baseline; the machined locating block is actually the only one that is cheaper), and it is not the only concept that improves gap repeatability (all four do — that was the point of the workshop). It wins because it is the only concept that improves on four of the five criteria at once, trading a higher unit cost for gains everywhere else. The ceramic insert is the instructive failure case: it does improve the one criterion the team most cared about (gap repeatability, since ceramic does not thermally distort at furnace temperature), but it nets to −2 once cost, ergonomics, and maintainability (brittle, harder to inspect, more care needed in handling) are all counted. A team that stopped at "does it fix the gap problem?" would have picked a net loser. This is not a reason to stop asking that question — it is a reason to also ask the other four.

Worked example 3: piloting the modular quick-change fixture

The data below is a constructed example, not a real production run. Setting: brazed aluminium heat-exchanger cores, 100 % helium leak test at final inspection, one lot per shift. The modular quick-change fixture from Worked example 2 was piloted on one cell for 14 shifts, compared against 15 shifts on that same cell immediately before the change. (The full-scale, 30-shift-per-period confirmation after a plant-wide rollout is the capstone, Module 19.)

Fifteen lots before the change ran 0.0441 (4.41 %) leakers on average, close to the Module 3 charter's 4.2 % baseline, over an average lot size of 105.7 cores. Fourteen lots after ran 0.0201 (2.01 %), over a smaller average lot size of 71.1 cores — smaller because a pilot cell runs less volume than the full line. Both periods, charted individually as p charts, show no rule violation: neither is drifting or unstable on its own. Notice something else about both charts, though: at these lot sizes and this rate, the 3-sigma lower control limit works out to 0.0 on every single point in both periods, simply because the rate is low enough that "mean minus three sigma" goes negative and gets floored at zero. That means the p chart's own 3-sigma rule cannot, by itself, flag the after period as a special-cause improvement over the before period — a chart of the after period alone would look identical whether the fixture helped or not. This is exactly why step three of piloting (above) calls for a hypothesis test in addition to the charts, not instead of them: the charts confirm each period is internally stable; the test is what actually compares the two periods to each other.

Treating this as a 2×2 table (leak or pass, before or after) and running a chi-square test of independence — equivalent to a two-proportion test — gives χ² = 10.524 on 1 degree of freedom, p = 0.0012: comfortably below 0.05, so the drop is not plausibly just sampling noise. Alongside the p-value, the practical-significance numbers: relative risk (after ÷ before) = 0.455, meaning the pilot cell's leak rate is a little under half what it was, and a risk difference of -2.41 percentage points. Cramér's V, a chi-square effect-size measure, comes out small (0.064) — worth flagging as a common trap on its own: V is small here mainly because leaks are rare in both periods (most of both tables is "Pass"), not because the change was unimportant. For a rare-event table like this one, relative risk and risk difference are the practically meaningful numbers; a small Cramér's V should not be read as "the effect did not matter." The pilot supports rolling the fixture out further, with the expectation — not yet the proof — that a full-scale, longer-running confirmation will show at least as large an improvement; Module 19 shows what that confirmation actually found.

Common mistakes

• Running a kaizen event with no baseline data. Consequence: there is nothing to compare "after" against except memory and impression, so the event's outcome cannot be proven or defended later. Fix: pull or collect baseline data (Module 5's rational subgrouping applies here too) before the event starts, not during it.
• Comparing two averages, or a two-bar chart, without a hypothesis test. Consequence: a real improvement can look unconvincing next to noisy data, or a difference that is really just sampling variation gets rolled out plant-wide and then quietly reverts. Fix: run the test that matches the data (Module 11) and report it with an effect size, not the averages alone.
• Applying 5S-style standardization to genuinely exploratory work. Consequence: 3M's own reported experience above. Fix: match the tool to the work; a first-of-a-kind build or an early-stage development effort is not the same kind of work as a repeatable production line, and does not automatically benefit from the same discipline.
• Treating a Pugh matrix's net score as the final decision instead of a facilitation aid. Consequence: a close score, or a criterion the team privately cares about far more than the others, gets rubber-stamped by the arithmetic instead of discussed. Fix: treat a close result as a prompt for discussion, and consider a weighted matrix when criteria are not equally important.
• Declaring a pilot successful from the period average alone, with no control chart of either period. Consequence: one unusually good (or bad) day can move a period's average without the process's true level having changed at all, as Worked example 3's discussion of the p chart's floored lower limit shows directly. Fix: chart both periods, not just the two summary rates.
• Blurring internal and external setup in SMED. Consequence: staging or preparation that could happen while the machine is still running gets left as part of the stop, so the changeover stays slow even after "doing SMED." Fix: time-and-motion-study one real changeover first and tag every single step internal or external before designing anything.
• Choosing a poka-yoke that only warns when a design that prevents the error outright is available. Consequence: an operator habituates to a light or a buzzer and starts ignoring it, so the warning stops functioning within weeks. Fix: prefer a contact or fixed-value method that makes the error physically impossible over one that only flags it after the fact (Module 10's Occurrence vs. Detection distinction again).

Exercises

Exercise 1: pallet-changeover time on a machining cell

Constructed example, not a real production run. Setting: time (min) to change the fixture pallet on a CNC machining cell, 10 changeovers before and 10 after fitting a locating pin that only accepts the pallet one way round (a contact-method poka-yoke that also happens to speed up the changeover, since the operator no longer has to check alignment by eye).

Tasks. (a) Compute both group means and standard deviations. (b) Run a two-sample t-test (pooled, equal-variance assumption) and report t, degrees of freedom, and the p-value. (c) Compute Cohen's d and compare its size to Worked example 1's. Why would you expect this pilot's effect size to be smaller than a full SMED redesign's?

Exercise 2: Pugh matrix for a connector poka-yoke

Constructed example, not a real selection workshop.

A team is choosing a device to stop an operator inserting a wiring connector backwards, scored against the current baseline (visual inspection only) on four criteria: how well it prevents the error, unit cost, its impact on cycle time, and the training burden it adds.

Tasks. (a) Compute the net score for each concept. (b) Which concept wins? (c) The optical sensor interlock has the same error-prevention score as the two winning concepts. Why does it still net out worst of the three real alternatives?

Quiz

Ten questions. Score 70 % or more to mark the module complete on this device.

Key takeaways

• 5S and standard work are the foundation the rest of this toolkit depends on: Standardize and Sustain turn a one-time cleanup into a repeatable state, and standard work's three elements (takt time, work sequence, standard in-process stock) are also a precondition for a control chart's common-cause variation meaning what it is supposed to mean.
• Poka-yoke that prevents an error outright (a contact or fixed-value method) is a stronger fix than one that only warns after the fact; in PFMEA terms, prevention drives Occurrence down, a warning only improves Detection.
• SMED separates internal setup (machine stopped) from external setup (can happen while running), converts as much internal work to external as possible, then streamlines what is left.
• A Pugh matrix's net score (count of + minus count of −, against a baseline scoring 0) is a facilitation tool for structuring a decision, not an algorithm that removes judgment; a concept can win on one important criterion and still lose overall.
• Lean's standardizing tools are close to unambiguously good on a repeatable production process and can actively hurt genuinely exploratory work — 3M's documented experience is a real, if contested, case for keeping that distinction in mind.
• Proving a pilot improvement takes a hypothesis test with an effect size, not just two averages, plus a control chart of each period on its own; a chart's own control limits (for example, a p chart's lower limit flooring at zero) can be structurally unable to flag an improvement by themselves, which is exactly why both are needed together.

References

All web sources accessed 2026-09-10 unless noted.

1. Hindo, B. "At 3M, a struggle between efficiency and creativity." BusinessWeek, 11 June 2007. https://effectuation.org/hubfs/Journal%20Articles/2016/06/3m-struggle-between-efficiency-and-creativity.pdf
2. Hirano, H. (1995). 5 Pillars of the Visual Workplace. Productivity Press. https://www.routledge.com/5-Pillars-of-the-Visual-Workplace/Hirano/p/book/9781563270475 (catalogue-level; the five steps)
3. Lean Enterprise Institute. Lexicon: "Standardized Work." https://www.lean.org/lexicon-terms/standardized-work/
4. Shingo, S. (1986). Zero Quality Control: Source Inspection and the Poka-yoke System. Productivity Press. https://www.routledge.com/Zero-Quality-Control-Source-Inspection-and-the-Poka-Yoke-System/Shingo/p/book/9780915299072 (catalogue-level; the three mechanisms)
5. Shingo, S. (1985). A Revolution in Manufacturing: The SMED System. Productivity Press. https://www.routledge.com/A-Revolution-in-Manufacturing-The-SMED-System/Dillon-Shingo/p/book/9780915299034 (catalogue-level; the internal/external distinction is confirmed at this depth, specific numeric changeover-time claims are Shingo's own reported figures, not independently audited, and are presented as such)
6. Imai, M. (1986). Kaizen: The Key to Japan's Competitive Success. McGraw-Hill. https://archive.org/details/kaizen00masa (catalogue-level)
7. Pugh, S. (1991). Total Design: Integrated Methods for Successful Product Engineering. Addison-Wesley. https://en.wikipedia.org/wiki/Stuart_Pugh (secondary: the concept-selection matrix method, via a summary source)