Agree on the Defect Before You Count It

Two people inspect the same work and report different defect counts. The supervisor wants to know who missed something. The improvement team wants a trend. Someone starts discussing retraining before anyone asks what each person counted.

Start there. What counts as a defect, and what does one mark on the report actually represent?

Think of two umpires working the same game with different strike zones. Both are honest. Both are working hard. The box score still means nothing. A defect count works the same way: until everyone calls the same zone, the trend line is noise with a nice chart.

A spreadsheet will make a percentage out of anything. It cannot tell you whether the people feeding it mean the same thing. Before you use a defect rate to pick a Lean project or judge a Six Sigma fix, find out how one look at a box turns into one mark on the sheet.

Telling everyone to use good judgment is not enough. What fails, how you check it, what you count, what you divide by and what happens when a box comes back all need to be clear enough that another trained person gets the same answer without calling you over.

Consider a fictional gift-box operation: the cosmetic printed border and all numbers are illustrative.

The team inspects 200 boxes. Twelve boxes each have one gap in the printed border. Four other boxes each have two separate gaps. Every remaining box meets the border specification. One inspector reports 16 defects. Another reports 20.

Both numbers describe something present in those same boxes. Sixteen boxes have at least one gap. Twenty separate gaps exist across those boxes. Calling both numbers “defects” leaves the report open to interpretation before anyone touches a formula.

Know what the customer asked for

First question: does that gap break what the customer asked for? Before you count, agree on what a bad box looks like, and check that it matches what the customer asked for. The team agreeing with each other does not make a box the customer rejected good. I would settle this before trusting a trend. A count is only useful if it tells you whether the customer is getting the product you promised.

For this fictional order, assume the approved customer appearance specification requires a continuous printed border with no gaps visible under its specified inspection conditions. The current inspection instruction identifies the applicable specification and revision. That is the basis for the decision.

On your floor, find the real requirement before you write the counting rule. It might be a customer spec, a drawing or an approved sample. Know which one rules. An old sample taped above a bench gets checked before anyone trusts it.

If two documents disagree, the person inspecting the box should not have to sort it out mid-shift. Ask the person who owns the spec. Hold the box until they answer. A deadline does not turn a guess into a good call.

Ask two questions: who can explain the spec, and who can change it? Those may be different people. Your team can rewrite how it records a gap. It cannot vote the standard down.

Define the observation before the tally

Now describe how the condition will be observed. For the gift boxes, the inspection instruction should tell the inspector which surface to examine and how to use the specified viewing conditions. If distance, lighting, orientation, or a viewing aid affects the decision, identify what the authorized method requires. If people inspect differently, you cannot tell whether the product changed or the inspection changed.

“Look it over” leaves room for one person to inspect under the station light while another tilts the lid beneath a flashlight. Even when both people care about quality, different methods can produce different observations. Confirm that the conditions at the workstation match the instruction. Writing a method that cannot be followed there creates another problem.

Give people usable examples tied to the requirement. For this cosmetic feature, approved photographs or reference samples can help show what a continuous border looks like and what a visible break looks like under the required conditions. Keep their identity and revision clear. A helpful picture still needs a known relationship to the approved standard.

The instruction also needs a route for a condition the examples do not settle. Record what was observed and send the item through the existing hold and disposition process. Do not force a questionable box into an accepted or rejected category simply because the reporting form has only two choices.

Then define the counting unit. For this example, the daily measure will be the percentage of inspected boxes with at least one qualifying border gap at their first inspection. Each box contributes once to the inspected count. Each affected box contributes once to the count of boxes failing this particular requirement.

Write that definition next to the measure. “Border defects” is too vague. “Boxes failing the border requirement at first inspection” tells the reader what the numerator contains. “Boxes inspected for that same feature at first inspection” tells the reader what belongs in the denominator.

For the fictional batch, 16 divided by 200 is 8 percent. The report should say that 8 percent of the inspected boxes failed this border requirement at first inspection. It should not claim that 8 percent failed every other quality check, or that the result describes boxes that were never inspected.

The team may also want to record the number of separate gaps for troubleshooting. That is a different count. Twelve single gaps plus eight gaps on the four remaining affected boxes gives 20 gaps. Twenty divided by 200 is 0.10 gaps per inspected box, or 10 gaps per 100 inspected boxes. That does not mean 10 percent of the boxes failed.

A box with two gaps contributes one affected box and two gaps. If both measures are useful, give them separate fields and names. Define what separates one gap from another before using the occurrence count. Otherwise, an inspector may count one damaged area while someone else divides it into several marks.

Keep repeated work visible without counting it twice

Next decide what happens when a box returns to the inspection station. Suppose all 16 affected boxes receive a replacement lid and pass a second inspection. The first inspection still found 16 affected boxes among 200. The additional work and its outcome belong in the record, but they do not erase the original result. You need to see how much work came back, even when the final product passed.

Do not add the 16 repeat inspections to the denominator of this first-inspection measure. Sixteen divided by 216 is about 7.4 percent. That lower percentage would come from mixing initial inspections with repeat visits. The original 200 boxes have not changed, and the first-inspection result remains 8 percent.

Likewise, if two people look at the same original gap, the second look does not create a second physical gap. Mark each box or its record so a second look ties back to the first one. A box number, a tally sheet, whatever you already trust. One gap gets one mark, no matter how many people look at it.

This does not require throwing away repeat-inspection information. Rework effort, additional inspections, and final disposition can matter a great deal. Record them so the operation can understand the work it performed. Keep the original question intact when calculating its answer.

Boxes still on hold count too. Show them as on hold. If the report goes out before they are decided, say so. Do not quietly call them good or drop them from the count to make the percentage come out clean.

Check whether people can use the definition

Before rolling out the form, have the people who will use it apply the proposed instructions to the same representative examples. Include clear cases and cases near the acceptance boundary. Have each person record a decision independently before discussing it. Otherwise, the first confident answer can hide the disagreement you need to find. This is where you find out whether the instructions work when you are not standing beside the inspector.

Compare the recorded decisions with the authorized reference decisions and with each other. Ask where the difference came from. Did someone use an outdated sample? Was the viewing condition wrong? Did one person count boxes while another counted gaps? Did the instruction leave a particular condition unresolved?

Correct the instruction, reference material, workstation condition, or training that caused the problem. Then have people apply the corrected method. A signature says someone signed. Watching the person make and record the decision gives you something useful to evaluate.

Know what this practice run can and cannot tell you. A short practice run shows where people disagree. It does not prove your gauge is right. People can agree with each other and still check against the wrong spec or all miss the same gap. Get help from quality before you bet a big decision on it.

Keep the collection form practical. It needs enough information to connect the observation to the product, requirement, inspection stage, count, and disposition. Test it during a short trial. Watch where people hesitate, add notes in the margins, or invent their own abbreviations. Those are places to inspect before the form becomes another source of interpretation.

Assign responsibility for keeping the specification reference, instruction, examples, and reporting fields current. When something changes, record what changed and when it took effect. Confirm that the people using it can apply the revised method. A revised document sitting in a folder does little for someone working from yesterday’s copy.

Also mark the change in the data record. If the definition or observation method changes, a lower reported rate may reflect that change. It does not establish that the product improved. Preserve the original records and explain the basis of any comparison. Do not present a change in counting practice as a production gain.

The immediate job is straightforward. Pick a defect measure that influences a real decision. Ask the people collecting it to identify the governing requirement, demonstrate the observation method, explain one recorded count, name the denominator, and show what happens when the same item returns. Follow one actual record through that process.

Where the answers differ, fix the counting method before you use those numbers to judge anyone or pick a fix. Keep inspecting and holding bad product the way you do today while you sort it out. A question about the numbers does not pause your job to protect the customer.

Then go fix what is making the gap. Clear counting rules help you aim the fix and see whether it worked. They do not repair a single print gap on their own.

Before asking whether the defect rate improved, make sure everyone can show what was counted and why. That is how a number becomes usable for the next decision.

Try it this week. Hand the same five boxes to two people, have each one count alone, then compare. If the numbers match, you have a measure. If they do not, you just found your first fix, and it cost you ten minutes. Want one idea like this to take to the floor every weekday? Read the Briefs.

Marc Bradburry
Big Bearded Ginger