Your quality system is a “cactus” 🌵 - and 4 other reasons why a whopping 78% of your NCs / defects / complaints don’t get reported!

In a recent study of 47 manufacturing plants, quality leaders used a “walkabout” method to understand how many quality events (NCs, defects, complaints, deviations, out-of-specs) could possibly be underreported.

Across 4 major segments (consumer, food, pharma, and machinery), they found underreported events at an average of 78%. And these weren’t just minor issues – a fifth of these (~16%) could be considered major/critical NCs, defects, or complaints.

So why does this matter? 📈

Quality events like deviations, defects and non-conformances are lagging indicators of process issues. In other words, when you root cause any of these events, you’ll typically find process issues at fault (eg. lack of training, wrong specifications, inadequate SOP, etc.)

The whole point of a Quality Management System is to achieve better outcomes by defining, documenting, measuring and improving processes. If a majority of the problems aren’t being identified, you’re missing on a huge improvement opportunity!

How do you know if your organization faces the same problem?

At the end of this article, you will find the DIY “walkabout” method that was employed. It’s a dipstick study – but there’s no alternative. Try it out – and if your results are different – reach out to me. I’d love to hear about it.

So why do 78% of quality events go unreported?

Reason 1: Your quality system is a “Cactus”

You knew this was coming – so let me get into the data. A surprising result came out of the study. It turns out that paper-based systems actually fared better than Excel or and eQMS/MRP for reporting defects and non-conformances.

When users from the shop floor, procurement and service were interviewed, the researchers found that these digital systems were adding considerable friction to the process. One production manager actually described the eQMS as a “cactus”.

These digital systems carry huge advantages over paper-based systems. But they presented major challenges such as “a long learning curve”, “too many mandatory fields”, “extremely slow”, and “frustrating to use”.

Without addressing these critical problems, you’re not going to be able to improve timely reporting of issues. I’ll talk more about this in the solutions section below.

Reason 2: It’s hard to access the system on the shop floor

Device restrictions and accessibility of the quality system on the shop floor was a critical factor in the under-reporting of events.

Meetings are, by definition, synchronous.  Specifically, they require multiple people to stop whatever else they were doing and focus on the audit.

By using information and communication technology properly, a large amount of the audit work can be made asynchronous.  This can actually increase the effectiveness of your audits by allowing people the time and space to complete certain tasks in their own time.

Reason 3: Wrong or misaligned goals

The survey showed that most companies had a stated goal to reduce quality events such as complaints, defects, and NCs. This sounds like a great goal to have! The problem is that it becomes a self-fulfilling goal. I want to see less defects -> means I want to report less defects.

It gets even worse when folks in production want to sell more (defects get in the way), sales want to sell more (complaints get in the way), procurement wants to buy on time (supplier NCs get in the way).

Even with the best intentions, folks tend to subconsciously ignore “tiny” problems when they’re not incentivized to create them.

As odd as this may sound, companies that didn’t have an openly stated zero-defect policy fared better at reporting all defects. A good trick to employ here is to create a separate class of defects and call them “observations” or “issues” so they don’t affect your defect goal.

Reason 4: Silos stop other teams from creating events

Silos exist in all companies. But companies can get more siloed for various reasons eg. size, complexity, geographical distribution, politics. Whatever the reason, the result is the same – it gets harder to engage other teams around quality processes.

It can be challenging to break down silos. Increasing engagement on quality processes like root causes, audits, and even disposal of NCs can help. Making it easier to participate can help. Ultimately, driving alignment towards a common goal can also help.

Reason 5: The auditor comes knocking

Audits are a very important part of a quality management system. Auditors, however, can unknowingly create the wrong kind of pressures.

In controlled industries like food processing, or pharmaceuticals, auditors will insist that every defect or non-conformance must be root caused. In addition, a trend of increasing defects or NCs may create a critical finding in the audit.

Possible solutions including creating a second class of non-conformance called deviation or issue. Make sure to upgrade these to defects / NCs when appropriate or your auditor will suspect you are sweeping problems under the rug.

Focus on root cause and CAPA for all major NCs / defects so that you can show improvement where it counts.

Potential solutions: So what should you do?

Consider doing the following:

1. Create a second class of NCs/Defects/Complaints – call them issues or observations. That helps with your goals and the stigma of creating events. It will also lower the ‘barrier’ for what is considered an event. Consider creating a financial incentive for the employee who creates the most observations every month.
   “It’s just a tiny observation – not a defect!”
2. Make your system easy to use and accessible. You don’t need to overhaul your entire eQMS. Consider rolling out a simple cloud-based tool that allows single-click creation of events from anywhere including the shop floor.
   “Wait – what happened to that cactus?”
3. Ensure that your system gets rid of painful validation rules and mandatory fields that make people detest it. You can’t get any of that data if the event goes unreported anyway.
   “You mean I can hold my latte and click this super button?”
4. Build engagement in quality processes by involving more people in the resolution of NCs and closing out CAPAs. Share the credit with the team every month. Get suppliers and customers involved as well by allowing them to collaborate on the creation and resolution of defects.
   “We’re doing a root cause – want to join? I’ve ordered pizza”
5. Form a plan for your auditor that allows you to explain your new strategy and show how increasing defects actually is beneficial to your customers.
   “You can’t fix it if you can’t measure it? Nice – I never thought of that”

Now, maybe this is a good time to see if you have a problem….

DIY: The ‘walkabout’ method for discovering underreported events

This is as simple as it sounds. As you are paying more attention to a particular area – you might see a bump in rejections or defects. That’s normal – in fact, it’s called the ‘Observer Effect’. Read more here…

But I’ve added some tips here to make sure you emerge successfully.

1. Gather a few weeks of baseline data from different parts of your org. How many complaints are you getting every month? What about non-conformances on the shop floor? Out-of-spec on raw materials and parts?
2. Every day for a week, select a different part of your organization – eg. procurement/incoming raw materials, packing, dispatch, customer service, sales. Speak to people outside your team, as well as quality personnel.
3. Conduct your interview:
   a. Don’t say “I’m investigating to see how many defects aren’t being reported”
   b. Do say “We’re doing an exercise to understand what kind of issues may be coming up regularly, but aren’t important enough to create a defect”
   c. Don’t say “Why aren’t you reporting these?”
   d. Do say “If we gave you a super-easy way to capture these issues, how many of them would you be able to report a day.” Describe how easy your new (but hypothetical) system will be.
   e. Don’t say “What’s wrong with our quality systems and processes”
   f. Do say “If you could make our systems and software better, which areas would you focus on”. Lean into the answers and ask open-ended questions like “That’s interesting, tell me more about…”
   g. Get specific. Try to check yourself, or get data for an entire day/week of what’s going unreported. Remember, some of these issues might seem unimportant.
   h. With customer-facing teams, it’s worth asking “which issues were you able to resolve without creating a customer complaint”
4. Do spot checks on the shop floor – you’ll find a lot of issues. Become OCD for a day.
5. Compare data to your baseline. For example, if you were creating 10 defects per week in production, and by this system you are creating 15 defects per week, then 33% of your defects are going unreported. Remember the observer effect above – so always compare to older baselines, not current week.

I’d love to hear what you learnt. Email me at ben.merton@unifize.com

‍