[Federal Register Volume 87, Number 46 (Wednesday, March 9, 2022)]
[Notices]
[Pages 13295-13299]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 2022-04972]
-----------------------------------------------------------------------
DEPARTMENT OF HEALTH AND HUMAN SERVICES
Food and Drug Administration
[Docket No. FDA-2022-N-0075]
Food and Drug Administration Quality Metrics Reporting Program;
Establishment of a Public Docket; Request for Comments
AGENCY: Food and Drug Administration, HHS.
ACTION: Notice; establishment of a public docket; request for comments.
-----------------------------------------------------------------------
SUMMARY: The Food and Drug Administration (FDA or Agency) is announcing
the establishment of a docket to solicit comments on changes to FDA's
previously proposed quality metrics reporting program (QM Reporting
Program). This notice describes considerations for refining the QM
Reporting Program based on lessons learned from two pilot programs with
industry that were announced in the Federal Register in June 2018, a
Site Visit Program and a Quality Metrics Feedback Program, as well as
stakeholder feedback on FDA's 2016 revised draft guidance for industry
entitled ``Submission of Quality Metrics Data.'' FDA is interested in
responses to the questions listed in section III of this document, in
addition to any general comments on the proposed direction for the
program. This notice is not intended to communicate our regulatory
expectations for reporting quality metrics data to FDA but is instead
intended to seek input from industry to inform the future regulatory
approach.
DATES: Submit either electronic or written comments by June 7, 2022.
ADDRESSES: You may submit comments as follows. Please note that late,
untimely filed comments will not be considered. Electronic comments
must be submitted on or before June 7, 2022. The https://www.regulations.gov electronic filing system will accept comments until
11:59 p.m. Eastern Time at the end of June 7, 2022. Comments received
by mail/hand delivery/courier (for written/paper submissions) will be
considered timely if they are postmarked or the delivery service
acceptance receipt is on or before that date.
Electronic Submissions
Submit electronic comments in the following way:
Federal eRulemaking Portal: https://www.regulations.gov. Follow the instructions for submitting comments.
Comments submitted electronically, including attachments, to https://
[[Page 13296]]
www.regulations.gov will be posted to the docket unchanged. Because
your comment will be made public, you are solely responsible for
ensuring that your comment does not include any confidential
information that you or a third party may not wish to be posted, such
as medical information, your or anyone else's Social Security number,
or confidential business information, such as a manufacturing process.
Please note that if you include your name, contact information, or
other information that identifies you in the body of your comments,
that information will be posted on https://www.regulations.gov.
If you want to submit a comment with confidential
information that you do not wish to be made available to the public,
submit the comment as a written/paper submission and in the manner
detailed (see ``Written/Paper Submissions'' and ``Instructions'').
Written/Paper Submissions
Submit written/paper submissions as follows:
Mail/Hand delivery/Courier (for written/paper
submissions): Dockets Management Staff (HFA-305), Food and Drug
Administration, 5630 Fishers Lane, Rm. 1061, Rockville, MD 20852.
For written/paper comments submitted to the Dockets
Management Staff, FDA will post your comment, as well as any
attachments, except for information submitted, marked and identified,
as confidential, if submitted as detailed in ``Instructions.''
Instructions: All submissions received must include the Docket No.
FDA-2022-N-0075 for ``FDA Quality Metrics Reporting Program;
Establishment of a Public Docket; Request for Comments.'' Received
comments, those filed in a timely manner (see ADDRESSES), will be
placed in the docket and, except for those submitted as ``Confidential
Submissions,'' publicly viewable at https://www.regulations.gov or at
the Dockets Management Staff between 9 a.m. and 4 p.m., Monday through
Friday, 240-402-7500.
Confidential Submissions--To submit a comment with
confidential information that you do not wish to be made publicly
available, submit your comments only as a written/paper submission. You
should submit two copies total. One copy will include the information
you claim to be confidential with a heading or cover note that states
``THIS DOCUMENT CONTAINS CONFIDENTIAL INFORMATION.'' The Agency will
review this copy, including the claimed confidential information, in
its consideration of comments. The second copy, which will have the
claimed confidential information redacted/blacked out, will be
available for public viewing and posted on https://www.regulations.gov.
Submit both copies to the Dockets Management Staff. If you do not wish
your name and contact information to be made publicly available, you
can provide this information on the cover sheet and not in the body of
your comments and you must identify this information as
``confidential.'' Any information marked as ``confidential'' will not
be disclosed except in accordance with 21 CFR 10.20 and other
applicable disclosure law. For more information about FDA's posting of
comments to public dockets, see 80 FR 56469, September 18, 2015, or
access the information at: https://www.govinfo.gov/content/pkg/FR-2015-09-18/pdf/2015-23389.pdf.
Docket: For access to the docket to read background documents or
the electronic and written/paper comments received, go to https://www.regulations.gov and insert the docket number, found in brackets in
the heading of this document, into the ``Search'' box and follow the
prompts and/or go to the Dockets Management Staff, 5630 Fishers Lane,
Rm. 1061, Rockville, MD 20852, 240-402-7500.
FOR FURTHER INFORMATION CONTACT: Jean Chung, Center for Drug Evaluation
and Research, Food and Drug Administration, 10903 New Hampshire Ave.,
Bldg. 75, Rm. 6655, Silver Spring, MD 20993, 301-796-1874,
[email protected].
SUPPLEMENTARY INFORMATION:
I. Background
A. Quality Metrics
For pharmaceutical manufacturing, quality metrics are objective
means of measuring, evaluating, and monitoring the product and process
life cycle to proactively identify and mitigate quality risks; thereby
managing operations at higher levels of safety, efficacy, delivery, and
performance. Quality metrics are used throughout the drug and
biological product industry to monitor quality control systems and
processes and drive continuous improvement efforts in manufacturing.
Quality metrics are important because failure to update and innovate
manufacturing practices and lack of operational reliability (i.e.,
state of control) can lead to quality problems that have a negative
impact on public health.
The minimum standard for ensuring that a manufacturer's products
are safe and effective is compliance with current good manufacturing
practice (CGMP) requirements as outlined in current regulations and as
recommended in current policies (21 CFR parts 210 and 211 for drug
products and the International Conference on Harmonisation guidance for
industry entitled ``Q7 Good Manufacturing Practice Guidance for Active
Pharmaceutical Ingredients'' (September 2016); available at: https://www.fda.gov/regulatory-information/search-fda-guidance-documents/q7-good-manufacturing-practice-guidance-active-pharmaceutical-ingredients-guidance-industry). However, compliance with CGMP does not necessarily
indicate whether a manufacturer is investing in improvements and
striving for sustainable compliance, which is the state of having
consistent control over manufacturing performance and quality.
Sustainable CGMP compliance is difficult to achieve without a focus on
continual improvement.
An effective Pharmaceutical Quality System (PQS) ensures both
sustainable CGMP compliance and supply chain robustness. Quality
metrics data can contribute to a manufacturer's ability to develop an
effective PQS because metrics provide insight into manufacturing
performance and enable the identification of opportunities for updates
and innovation to manufacturing practices. Quality metrics also play an
important role in supplier oversight and can be used to inform the
oversight of outsourced activities and material suppliers as well as
appropriate monitoring activities to minimize supply chain disruptions.
Quality metrics data provided by establishments can also be useful
to FDA. These data can assist the Agency in developing compliance and
inspection policies and practices to improve the Agency's ability to
predict, and therefore possibly mitigate, future drug shortages, and to
encourage the pharmaceutical industry to implement innovative quality
management systems for pharmaceutical manufacturing. For example,
quality metrics data can be applied to FDA's risk-based inspection
scheduling, reducing the frequency and/or length of routine
surveillance inspections for establishments with metrics data that
suggest sustainable compliance. Additionally, the submission of quality
metrics data can provide ongoing insight into an establishment's
operations between inspections.
As part of FDA's shift towards a risk-based approach to regulation,
the Agency proposed to develop and implement a QM Reporting Program to
support its quality surveillance activities, as described in section
I.B of this notice. Under this program, FDA
[[Page 13297]]
intends to analyze the quality metrics data submitted by establishments
to: (1) Obtain a more quantitative and objective measure of
manufacturing quality and reliability at an establishment; (2)
integrate the metrics and resulting analysis into FDA's comprehensive
quality surveillance program; and (3) apply the results of the analysis
to assist in identifying products at risk for quality problems (e.g.,
quality-related shortages and recalls).
B. FDA Guidance for Industry on the Submission of Quality Metrics Data
In July 2015, FDA issued the draft guidance entitled ``Request for
Quality Metrics'' (80 FR 44973), which described a potential mandatory
program for product-based reporting of quality metrics. Under this
proposed program, manufacturers would have submitted four primary
metrics (lot acceptance rate (LAR), product quality complaint rate
(PQCR), invalidated/overturned out-of-specification rate (IOOSR), and
annual product review (APR) or product quality review on-time rate) and
three optional metrics (senior management engagement, corrective and
preventative action (CAPA) effectiveness, and process capability/
performance). Stakeholder comments on the guidance included concerns
regarding the burden associated with collecting, formatting, and
submitting data at a product level across multiple establishments;
technical comments on the proposed metrics and definitions; and legal
concerns regarding the proposed mandatory program. Stakeholder
commenters also suggested a phased-in approach to allow learning by
both industry and FDA.
In response to this feedback, FDA published a revised draft
guidance in November 2016 entitled ``Submission of Quality Metrics
Data'' (81 FR 85226). The 2016 guidance described an initial voluntary
phase of the QM Reporting Program, with participants reporting data
either by product or establishment, through an FDA submission portal.
FDA removed one of the four metrics from the 2015 draft guidance and
requested submission of the remaining three key metrics: (1) LAR to
measure manufacturing process performance; (2) IOOSR to measure
laboratory robustness; and (3) PQCR to measure patient or customer
feedback and proposed incentives for participation. This guidance also
described how FDA intended to utilize the submitted data. Stakeholder
comments on the guidance indicated that the FDA-standardized
definitions remained a challenge and incentives to participate in a
voluntary program needed to be strengthened (e.g., direct collaboration
with FDA to develop the program was an example of a strong incentive).
Commenters requested a better understanding of the value and utility of
the data to be submitted to FDA and how FDA would measure success of
the program. Commenters also expressed a preference for a pilot program
to gather industry input before implementing a widespread QM Reporting
Program.
C. Lessons Learned From FDA's Quality Metrics Pilot Programs
In Federal Register notices issued on June 29, 2018, FDA announced
the availability of two pilot programs, a Quality Metrics Site Visit
Program (83 FR 30751) and a Quality Metrics Feedback Program (83 FR
30748) for any establishment that has a quality metrics program
developed and implemented by the quality unit and used to support
product and process quality improvement.
The Quality Metrics Site Visit Program offered experiential
learning for FDA staff and provided participating establishments an
opportunity to explain the advantages and challenges associated with
implementing and managing a Quality Metrics program. For example,
participants provided feedback in the form of case studies to
demonstrate the differences between the metric definitions proposed in
the FDA draft guidances and definitions commonly used by industry for
the same metrics. They proposed changes to the definitions, justifying
why those changes (if any) would be needed. FDA toured the operations
of 14 establishments worldwide and engaged with establishments on
topics such as: How quality metrics data are collected, analyzed,
communicated (e.g., dashboards, business intelligence platforms), and
reported throughout the organization in a structured and centralized
manner; how management utilizes quality metrics data to monitor the
performance of their supply network; how management leverages metrics
to promote data-driven decisions; how an establishment implements and
monitors continuous improvements based on metrics; how various quality
metrics are defined; how actions were taken from observations resulting
from quality metrics data reviews; and how efforts to proactively
mitigate and prevent shortages are coordinated.
In the Quality Metrics Feedback Program, participating
establishments presented their quality metrics programs to FDA staff.
The presentations were followed by discussions and knowledge sharing
that focused on analytical strategies, exploratory data analyses, data
preparation and structure, and visualizations for communication, as
well as demonstrations on how FDA plans to analyze the data using
advanced analytical techniques (e.g., data/text mining, interactive
visualizations), sophisticated statistical methods (e.g., control
charts, time series analysis), and machine learning (e.g., predictive
analytics, natural language processing). In these discussions, FDA also
obtained feedback on industry's anticipated challenges in applying the
approach described in FDA's revised draft guidance. Participants had
the opportunity to submit their quality metrics data through an FDA
submission portal and provide feedback on their user experience. The
industry participants represented different sectors of the
pharmaceutical industry including innovator drug products, generic drug
products, nonprescription (also known as over-the-counter (OTC)) drug
products, and biological products.
The dedicated meetings with industry during the two pilot programs
that focused on data analytics resulted in the following key lessons
learned for FDA, which will inform the direction of the QM Reporting
Program:
1. Different industry sectors prefer different metrics due to their
individual operations and business dynamics needs. Therefore, it is
necessary to implement a program with sufficient flexibility when
choosing metrics. Identifying critical practice areas (e.g.,
manufacturing process performance) and allowing establishments to
select appropriate metrics from several options is a more feasible
approach.
2. Any metric chosen to be reported should be meaningful to the
practice area being measured, and the data collected on that metric
should be able to influence decision making about process improvements
and capital investments.
3. In some instances, a combination of metrics rather than a single
metric is preferred to assess a particular practice area.
4. The majority of participants prefer to report data at an
establishment level and have the capability to segment by product, but
some participants prefer product-level reporting due to their business
structure (e.g., a vertically integrated company).
5. Calculating LAR and PQCR based on the definitions in the 2016
revised draft guidance can result in mathematical discrepancies such as
rates over 100 percent or invalid calculations (i.e., dividing by
zero)). These discrepancies are caused by inherent variabilities from
real-time
[[Page 13298]]
operations (e.g., lots may not be dispositioned in the same quarter in
which they were started) or how denominators are defined for a
specified period of time.
6. While LAR and IOOSR are quality metrics that are routinely
monitored by establishments, they are not discerning metrics due to
limited variability over time or limited scope and can result in false
positives by highlighting nonexistent performance issues. Other metrics
should be identified as surrogates for manufacturing process
performance and laboratory robustness. Examples include, but are not
limited to, right-first-time rate, process capability, and adherence to
lead time.
7. The effectiveness of the quality system is a critical component
of a QM Reporting Program as evidenced by numerous establishments
collecting data around their PQS. Examples include metrics related to
the effectiveness of CAPA programs, repeat deviations, maintenance
programs, and timeliness.
8. Metrics related to quality culture are important indicators of
performance and reliability, but unlike other quality metrics, it is
difficult to capture quality culture at an establishment based on
numerical metrics alone. Both numerical key performance indicators
(KPIs) (e.g., APR timeliness and near misses) and qualitative summaries
(e.g., descriptions of management commitment or quality planning) can
be used to further understand quality culture.
9. FDA's analysis of the data submitted during the Quality Metrics
Feedback Program indicates that the use of statistical quality control
applications (e.g., statistical process control and process capability)
and machine learning/natural language processing are appropriate and
meaningful analytical strategies to assess quality metrics data
submitted by establishments.
II. Proposed Direction for an FDA QM Reporting Program
FDA has applied the lessons learned from the pilot programs and
other stakeholder feedback toward refining the QM Reporting Program
that was presented in the 2016 revised draft guidance. In this section,
we summarize a potential direction for the program, and in section III
we request input on specific aspects of this approach.
FDA believes that a change in the entities responsible for
collecting and submitting quality metrics data is not needed. ``Covered
establishments,'' as defined in the 2016 revised draft guidance, are
establishments engaged in the manufacture, preparation, propagation,
compounding or processing of a ``covered drug product'' (products
subject to an approved application under section 505 of the Federal
Food, Drug, and Cosmetic Act (FD&C Act) (21 U.S.C. 355) or section 351
of the Public Health Service Act; legally marketed pursuant to section
505G of the FD&C Act (21 U.S.C. 355h) (nonprescription drugs marketed
without an approved drug application); or marketed as unapproved
finished drug products) or an active pharmaceutical ingredient used in
the manufacture of a covered drug product. ``Covered establishments''
include contract laboratories, contract sterilizers, and contract
packagers.
FDA is considering changes to other aspects of the QM Reporting
Program. Stakeholders have indicated that different industry sectors
may prefer different quality metrics. To provide flexibility to
manufacturers, FDA would focus less on standardization of quality
metrics and definitions. Instead, FDA would identify practice areas
that are critical to ensure sustainable product quality and
availability and would permit manufacturers to select a metric(s) from
each practice area that are meaningful and enable establishments to
identify continual improvement opportunities. The metric definitions
would not specify how establishments calculate particular metrics.
Rather, the reporting establishment would select the most appropriate
metric(s) from each practice area and inform FDA how it was calculated.
Through the collective feedback gathered from pilot participants, FDA
has identified the following four general practice areas as appropriate
at this time for the QM Reporting Program: (1) Manufacturing Process
Performance, (2) PQS Effectiveness, (3) Laboratory Performance, (4)
Supply Chain Robustness. Examples of quality metrics associated with
each practice include the following:
1. Manufacturing Process Performance
Process Capability/Performance Indices (Cpk/Ppk): A
measure that compares the output of a process to the specification
limits and can be calculated as a proportion (e.g., total number of
attributes with Ppk greater than 1.33 divided by total number of
attributes where Ppk is used). It is important to consider standard
deviation measurements using a reasonable sample size.
LAR: A measure of the proportion of lots that were
accepted in a given time period. Examples of inputs that can be used to
calculate LAR include lots completed, lots dispositioned, lots
attempted, lots rejected, lots released, lots approved, abandoned lots,
and parallel/backup lots.
Right-First-Time Rate: A measure of the proportion of lots
manufactured without the occurrence of a non-conformance. Examples of
inputs that can be used to calculate a right-first-time rate include
number of deviations, lots dispositioned, lots attempted, number of
nonconformances, and lots approved in the first pass.
Lot Release Cycle Time: A measure of the amount of time it
takes for the lot disposition process. Lot release cycle time can be
calculated with an appropriate unit of measurement such as number of
hours or days.
2. PQS Effectiveness
CAPA Effectiveness: A measure of the proportion of CAPA
plan implemented and deemed effective (i.e., effectiveness
verifications closed as effective). Examples of inputs that can be used
to calculate CAPA effectiveness include number of CAPAs initiated,
CAPAs closed on time, CAPAs closed as ``effective,'' overdue CAPAs, and
CAPAs resulting in retraining.
Repeat Deviation Rate: A measure of the proportion of
recurring deviation measures. Examples of inputs that can be used to
calculate repeat deviation rate include total number of deviations and
number of deviations with the same assignable root cause.
Change Control Effectiveness: A measure of timeliness and
effectiveness of implemented changes to GMP facilities, systems,
equipment, or processes. Examples of inputs that can be used to
calculate this metric include on-time closure of the change, total
number of late effectiveness checks, total number of changes initiated,
number of changes that are initiated reactively versus proactively, and
total number of changes deemed effective.
Overall Equipment Effectiveness: A measure of operating
productivity, utilizing planned production time. Overall equipment
effectiveness can be calculated using inputs related to availability
(e.g., planned production time, operating time), performance (e.g.,
production capacity), and quality (e.g., production output that does
not result in acceptable product).
Unplanned Maintenance: A measure of the proportion of
maintenance time that was not planned or scheduled. Examples of inputs
that can be used to calculate this metric include total maintenance
hours and planned maintenance hours.
[[Page 13299]]
3. Laboratory Performance
Adherence to Lead Time: A measure of the proportion of
tests in the laboratory that are completed on time according to
schedule requirements. Adherence to lead time can be calculated, for
example, by tracking initiation and testing turnover time in release
and stability tests (i.e., the number of days between the start date
and completion date for quality control (QC)); tracking data review and
documentation; tracking final result reporting prior to batch
disposition; or comparing QC testing completion date against the target
date.
Right-First-Time Rate: A measure of the proportion of
tests conducted without the occurrence of a deviation. Right-first-time
rate as a metric for laboratory performance can be calculated, for
example, by tracking the invalid assay rate, the number of assays
invalidated due to human errors, or CGMP documentation errors during
review.
IOOSR: A measure that indicates a laboratory's ability to
accurately perform tests. Examples of inputs that can be used to
calculate this metric include total number of tests conducted and total
number of out-of-specification results invalidated due to an aberration
of the measurement process.
Calibration Timeliness: A measure of a laboratory's
adherence to inspecting, calibrating, and testing equipment for its
intended purposes as planned. This metric can be measured by tracking
calibration criteria and schedules.
4. Supply Chain Robustness
On-Time In-Full (OTIF): A measure of the extent to which
shipments are delivered to their destination containing the correct
quantity and according to the schedule specified in the order. This
metric can be calculated using inputs such as the number of orders
shipped, number of past due orders, or number of orders shipped within
tolerance.
Fill Rate: A measure that quantifies orders shipped as a
percentage of the total demand for a given period. Examples of inputs
that can be used to calculate this metric include total number of
orders shipped, the number of orders placed, and the number of orders
received.
Disposition On-Time: A measure of the proportion of lots
in which the disposition was carried out on time. Examples of inputs
that can be used to calculate this metric include the total number of
lots dispositioned and the total number of lots dispositioned on time.
Days of Inventory On-Hand: A measure of how a company
utilizes the average inventory available. It is the number of days that
inventory remains in stock.
Given that the majority of participants in the pilot programs
prefer to report data at an establishment level, FDA is considering an
approach for aggregating and reporting quality metrics data at the
establishment level, with the option to segment by manufacturing train,
product type, or product level (e.g., application number or product
family).
Once the data are submitted, FDA intends to analyze the information
with statistical and machine learning methods to provide useful
insights for inspection resource allocation. Examples include
examination of product trends and clusters; exploratory and time-series
analyses for signal identification, thereby monitoring the health of
the establishment over time; and utilizing quality metrics data as an
input into machine learning models to assist in determining an
establishment's overall PQS effectiveness.
III. Request for Comments
We are seeking comment on the following aspects of FDA's proposed
direction for its QM Reporting Program. We note that the questions
posed in this section are not meant to be exhaustive. We are also
interested in any other pertinent information that stakeholders and any
other interested parties would like to provide on FDA's QM Reporting
Program. FDA encourages stakeholders to provide the rationale for their
comments, including available examples and supporting information.
A. Reporting Levels
1. Do you agree that reporting should be aggregated at an
establishment level?
2. Would reporting at an establishment level facilitate submission
of quality metrics data by contract manufacturing organizations?
3. If you normally assess metrics by product family at an
establishment, what are useful definitions of ``product family'' from
your industry sector?
B. Practice Areas and Quality Metrics
1. If you think the general practice areas listed in section II of
this notice would not meet the objectives of FDA QM Reporting Program,
what other practice areas should FDA consider?
2. If FDA were to consider Quality Culture as one of the general
practice areas, what are the critical components of a robust quality
culture and can any of these components be measured quantitatively? If
so, how do you recommend quality culture information be captured as a
quantitative metric (e.g., near misses, APR on-time, binary response to
Quality Culture survey, or other numerical metrics/KPIs)?
3. Do you think that any of the examples of quality metrics
proposed by FDA would not be an appropriate measure for the designated
practice area?
4. What other metrics should FDA consider for a designated practice
area?
5. FDA is interested in an establishment's experience with
implementing process capability and performance metrics. For example,
how would you report Cpk and/or Ppk to FDA as part of the QM Reporting
Program (e.g., reporting Cpk and/or Ppk for certain products,
aggregated at the establishment level)?
6. A metric may need to be changed or adjusted by an establishment
to better monitor PQS effectiveness, inform appropriate business
strategy, or capture insightful trends, thereby driving continual
improvement behaviors. What criteria should be applied to justify
changing or modifying a quality metric (by either the establishment or
by FDA)? How frequently would you expect changes or modifications to be
needed?
7. When would you rely on multiple metrics versus a single metric
as an indicator when assessing a particular practice area (e.g., two
metrics are considered in combination because one metric influences the
other)? What combination of metrics have been meaningful and useful?
C. Other Considerations
1. Are there considerations unique to specific product categories
(e.g., generic drug products, OTC drug products, or biological
products) that should be addressed in the QM Reporting Program?
2. What would be the optimal reporting frequency for quality
metrics data submissions (e.g., monthly, quarterly, or yearly, and
segmented by quarter or month)?
3. In instances where a manufacturer is not able to extract
domestic data and its submission to FDA contains both U.S. and foreign
data, how can these data be submitted to FDA in a manner that would
still be informative?
4. Are there any other aspects of FDA's proposed direction for the
program that FDA should address in future policy documents?
Dated: February 28, 2022.
Lauren K. Roth,
Associate Commissioner for Policy.
[FR Doc. 2022-04972 Filed 3-8-22; 8:45 am]
BILLING CODE 4164-01-P