Quality metrics are essential for life science companies to monitor performance, ensure compliance, and drive continuous improvement. But many organizations struggle with collecting the right data, interpreting it effectively, and using it to make informed decisions. In this episode of The Life Science Rundown, The FDA Group’s Nick Capman spoke with Kimberly Wallbank about creating and maintaining a Quality scorecard that truly adds value to your organization.
Kimberly Wallbank is a seasoned quality management consultant with over 25 years of experience in the life sciences industry. She specializes in establishing, maintaining, and improving quality management systems for pharmaceutical and medical device companies. Kimberly's expertise spans various areas, including complaint handling, pharmacovigilance, auditing, and regulatory compliance.
Kimberly has held key positions at companies like Pfizer and Abbott Laboratories. She is also an Adjunct Professor, sharing her knowledge with aspiring professionals in the field. As a professional speaker, Kimberly frequently presents at industry conferences and events on complaint handling and risk management topics. Her approach focuses on aligning quality systems with business needs while ensuring compliance with FDA, EMEA, ISO, and other recognized standards.
Apple | Spotify | YouTube | Web + Others
Summary, Key Points, and Practical Takeaways
This interview has been edited for clarity and length.
Nick Capman: What are the biggest issues with quality metrics in the life sciences industry?
Kimberly Wallbank: The main issue I see is that companies often focus too heavily on throughput metrics without considering the quality of actions or root causes. For example, they might track how quickly deviations are closed or CAPAs are completed rather than examining why these issues are occurring in the first place.
There's often a visceral reaction to measure speed and efficiency, but this can overlook crucial quality indicators. A great example is in complaint handling. I've seen clients track the types of defects reported in complaints but then focus solely on closure rates during management reviews. Instead, they should be asking why they're receiving 300 complaints about difficult-to-open blister packs in four months and addressing the root cause.
How can quality professionals influence discussions around metric selection?
Start by asking "why" questions. When someone suggests collecting a certain metric, ask them why it's important and how it will be used. This approach often reveals whether the metric is truly valuable or just a paper exercise.
Also, try to associate quality metrics with financial impact. Every deviation means more time spent on products before they can be sold. Every complaint represents a potential loss of future sales. By framing metrics in terms of dollars, you can often get more buy-in from upper management.
How many metrics should a quality scorecard include?
I recommend starting with two or three key metrics and working your way up from there. It's crucial to reevaluate your metrics regularly. If you've solved a particular problem, such as late CAPAs, you might not need to keep tracking five or six related metrics. Perhaps you can whittle it down to one or two to keep a finger on the pulse.
Remember, your scorecard should be a living, breathing thing that evolves with your business. The issues you're facing today won't be the same in two years, so your metrics should change accordingly.
How often should metrics be reviewed, and how should that process work?
I recommend reviewing metrics monthly at a minimum. This allows you to regularly assess whether each metric is still providing value. However, the frequency can vary depending on your organization's needs. Some companies might benefit from weekly reviews of certain metrics, with a more comprehensive quarterly review of the entire scorecard.
The key is to make the review process meaningful. Ask questions like: Is this metric still relevant? What actions are we taking based on this data? Are there new areas we should be measuring?
How do you balance the value of a metric against the effort required to collect the data?
This is all about risk-based decision-making. You need to weigh the potential value of the information against the resources required to collect and analyze it. Sometimes, a short-term investment in data collection can lead to long-term gains in efficiency or quality.
For example, when I was a QA manager, I asked my team to track their time for three weeks. It was painful, but it revealed significant interruptions that were impacting productivity. With this data, we were able to implement changes that dramatically improved efficiency and work-life balance. The key is to have a clear purpose for data collection, set a specific timeframe, and communicate the goals clearly to your team.
What types of metrics should always be included in a quality scorecard?
There are a few categories of metrics that I believe are essential:
Regulatory compliance metrics: Anything with a strict agency deadline, such as pharmacovigilance reporting (MDRs, PBRERs, FAERs), must be measured consistently.
Procedural compliance metrics: If your procedures specify hard timelines (e.g., all deviations must be closed in 30 days), you should track these. However, be prepared to reassess these timelines if the data shows they're unrealistic.
Product quality metrics: Track your top product defects and customer complaints. This helps you focus on the most impactful quality issues.
Customer-facing metrics: Monitor issues that directly impact customer satisfaction and product usability.
Remember, the goal isn't just to track these metrics but to use them to drive discussions and improvements.
How can quality professionals determine when to add or remove metrics from their scorecard?
While there's no one-size-fits-all checklist, here are some questions to consider:
What problem are we trying to solve with this metric?
Is this a customer-facing issue that we need to address?
Does this metric help us prevent recalls or regulatory issues?
Can we easily collect this data with our current systems, or will it require significant new processes?
How does this metric align with our overall quality and business goals?
Also, consider the triage of your issues. You can't solve everything at once, so prioritize based on impact and feasibility.
What are some common challenges in implementing effective quality metrics?
One major challenge is ensuring that everyone understands the purpose behind each metric. From entry-level contributors to site heads, everyone should know why data is being collected and how it will be used.
Another challenge is balancing the need for structured data entry with the flexibility to capture nuanced information. Overly rigid systems can miss important details, while too much reliance on free text fields can make data analysis nearly impossible.
Lastly, involving the right stakeholders in metric discussions is crucial. I've seen too many meetings where high-level decisions are made without input from the people who will actually be collecting and working with the data.
Kimberly’s key takeaways:
Focus on quality over quantity when selecting metrics. Look beyond simple throughput metrics like closure rates for deviations or CAPAs to consider metrics that reveal root causes of issues, such as types of deviations from specific manufacturing processes. For example, Instead of just measuring how quickly complaints are closed, analyze the types of defects reported, such as difficulty opening blister packs.
Regularly review and update your scorecard. Assess metrics monthly at a minimum, with some companies benefiting from weekly reviews of critical metrics. Conduct comprehensive quarterly reviews of the entire scorecard and reevaluate your metrics regularly to ensure they're still providing value. Be prepared to remove metrics that are no longer relevant or add new ones as business needs change.
Align metrics with business goals and regulatory requirements. Always include metrics for processes with strict agency deadlines, such as pharmacovigilance reporting. Make sure you’re tracking metrics related to procedural compliance, such as timelines specified in SOPs. Track MDRs, PBRERs, and FAERs consistently to ensure regulatory compliance. Also, connect your quality metrics to financial impacts to gain buy-in from upper management and consider how metrics can support product development and continuous improvement efforts.
Involve the right stakeholders in metric selection and review. Don't make high-level decisions about metrics without input from the people who will actually be collecting and working with the data. Include individuals from various levels of the organization, from entry-level contributors to site heads. Make sure that people responsible for data collection and analysis are involved in discussions about metrics and communicate the purpose and intended use of each metric to all stakeholders.
Use a risk-based approach to prioritize metrics. Start with low-risk, high-value metrics before progressing to more complex applications; for example, begin with metrics related to employee training on SOPs and work instructions, which can significantly reduce man-hours while maintaining quality. Consider the potential value of the information against the resources required to collect and analyze it and prioritize your metrics based on their impact on patient safety, product quality, and regulatory compliance.
Make sure your data collection methods are efficient and accurate. Evaluate how easily data can be collected and analyzed with current systems before implementing new metrics. Be cautious of relying too heavily on free text fields, which can make data analysis challenging. Consider the structure of data entry in quality management systems to facilitate easy extraction and analysis. For example, when implementing a new metric, assess whether it can be easily captured with existing drop-down fields in your QMS, or if it will require significant new processes to collect the data.
The FDA Group helps life science organizations rapidly access the industry's best consultants, contractors, and candidates.
Need expert help planning and executing compliance projects or connecting with a quality and compliance professional? Get in touch with us. If we haven’t worked together yet, watch our explainer video below or head to our company introduction page to see if we could help you execute your projects now or in the future.
Our service areas:
Quality Assurance | Regulatory Affairs | Clinical Operations | Commissioning, Qualification, and Validation | Chemistry, Manufacturing, and Controls (CMC) | Pharmacovigilance | Expert Witness
Our engagement models:
Consulting Projects | Staff Augmentation | FTE Recruitment | Functional Service Program
Our podcast:
Apple | Spotify | YouTube | Web + Others
Share this post