Mammography Quality Standards Act: X-ray Quality Improved, Access
Unaffected, but Impact on Health Outcomes Unknown (Testimony, 05/08/1998,
GAO/T-HEHS-98-164).
The overall impact of the Mammography Quality Standards Act on
mammography services has been positive, increasing the adherence of
mammography facilities to accepted quality assurance standards and
improving the quality of X-ray images. The impact on early detection of
breast cancer is less clear. Improving the quality of mammography images
should lead to more accurate interpretation by doctors and, therefore,
to more early detection of breast cancer. However, neither data nor
research methodologies are now in place to clearly establish these
links. Progress is being made to address these problems so that
improvements in cancer outcomes can be measured. Regarding concerns that
the additional burden of the act's quality assurance standards could
lead facilities to close rather than to upgrade, GAO found no indication
that the act has caused access problems.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: T-HEHS-98-164
TITLE: Mammography Quality Standards Act: X-ray Quality Improved,
Access Unaffected, but Impact on Health Outcomes
Unknown
DATE: 05/08/1998
SUBJECT: Breast cancer
Disease detection or diagnosis
Women
Medical equipment
Health care facilities
Inspection
Quality control
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO report. Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved. Major **
** divisions and subdivisions of the text, such as Chapters, **
** Sections, and Appendixes, are identified by double and **
** single lines. The numbers on the right end of these lines **
** indicate the position of each of the subsections in the **
** document outline. These numbers do NOT correspond with the **
** page numbers of the printed product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
** A printed copy of this report may be obtained from the GAO **
** Document Distribution Center. For further details, please **
** send an e-mail message to: **
** **
** **
** **
** with the message 'info' in the body. **
******************************************************************
GAO/T-HEHS-98-164
Cover
================================================================ COVER
Before the Subcommittee on Health and the Environment, Committee on
Commerce, House of Representatives
For Release on Delivery
Expected at 10:00 a.m.
Friday, May 8, 1998
MAMMOGRAPHY QUALITY STANDARDS ACT
- X-RAY QUALITY IMPROVED, ACCESS
UNAFFECTED, BUT IMPACT ON HEALTH
OUTCOMES UNKNOWN
Statement of Marsha Lillie-Blanton, Associate Director
Health Services Quality and Public Health Issues
Health, Education, and Human Services Division
GAO/T-HEHS-98-164
GAO/HEHS-98-164T
(108372)
Abbreviations
=============================================================== ABBREV
ACR - American College of Radiology
CIS - carcinoma in situ
FDA - Food and Drug Administration
MQSA - Mammography Quality Standards Act
MAMMOGRAPHY QUALITY STANDARDS ACT:
X-RAY QUALITY IMPROVED, ACCESS
UNAFFECTED, BUT IMPACT ON HEALTH
OUTCOMES UNKNOWN
============================================================ Chapter 0
Mr. Chairman and Members of the Committee:
I am pleased to be here today to discuss the findings of our work
over the last several years examining the impact of the Mammography
Quality Standards Act (MQSA). Quality mammography services are a key
tool in the early detection of breast cancer, significantly
increasing the possibility of survival for the estimated 180,000
women who are diagnosed with this devastating disease each year.
MQSA was enacted in 1992 in response to the growing incidence of
breast cancer and its associated mortality rates. MQSA established
minimum national quality standards for the nation's approximately
10,000 mammography facilities, as well as an accreditation and
inspection program to help ensure that these standards are met.
The act directed us to study the impact of MQSA's standards on (1)
the quality of mammography services, (2) early detection of breast
cancer, and (3) women's access to mammography services. Today, I
will focus on our findings on these issues, which are drawn from our
final report issued last October.\1
In summary, we found that the overall impact of MQSA on mammography
services has been positive, in that it has increased mammography
facilities' adherence to accepted quality assurance standards and
improved the quality of X-ray images. The impact on early detection
of breast cancer is less clear. Improving the quality of mammography
images should lead to more accurate interpretation by physicians and,
therefore, to improved early detection of breast cancer. However,
neither data nor research methodologies are now in place to clearly
establish these links. Progress is being made to remedy these
problems so that improvements in cancer outcomes can be measured.
Regarding concerns that the additional burden of MQSA quality
assurance standards could result in facilities deciding to close
rather than to upgrade, we found no indication that MQSA has caused
access problems.
--------------------
\1 Mammography Services: Impact of Federal Legislation on Quality,
Access, and Health Outcomes (GAO/HEHS-98-11, Oct. 21, 1997).
BACKGROUND
---------------------------------------------------------- Chapter 0:1
According to the Centers for Disease Control and Prevention, in 1995,
almost 60 percent of women over the age of 50 reported having
received a mammogram in the past year. The value of mammography in
reducing mortality is directly tied to its ability to detect cancer
at its earliest stages. Mammography is capable of detecting tumors
much smaller than those detected by other means. However, it is one
of the most technically challenging radiological procedures, and
ensuring the quality of the image and its interpretation is important
in detecting potential tumors. Figure 1 illustrates this point. It
shows two mammograms of the same patient who had a cancerous breast
tumor. As you can see, the picture on the left is not clear enough
to permit detection of the cancer site that is visible in the picture
on the right.
Figure 1: Low- and
High-Quality Mammography Images
of the Same Patient
(See figure in printed
edition.)
A mammogram that is of poor quality or incorrectly interpreted can
have considerable diagnostic consequences. If the mammogram is
incorrectly seen as indicating cancer (that is, a positive test
result), a woman may need to endure unnecessary and costly follow-up
procedures. A mammogram that is interpreted as normal when an
abnormality is actually present could result in the missed diagnosis
of early lesions and delayed treatment, which could cost a woman's
life.
MQSA contained a number of provisions designed to ensure the quality
of the image and its interpretation. Among other things, MQSA
provided that
-- the Food and Drug Administration (FDA) establish quality
standards for mammography equipment, personnel, and practices;
-- all mammography facilities, regardless of location or setting,
be accredited by an FDA-approved accrediting body once every 3
years and obtain a certificate from FDA in order to provide
mammography services after October 1, 1994; and
-- all mammography facilities have an annual evaluation by a
qualified medical physicist and an annual inspection by
FDA-approved inspectors that includes a test of image quality.
MQSA HAS HAD A POSITIVE EFFECT
ON THE QUALITY OF MAMMOGRAPHY
SERVICES
---------------------------------------------------------- Chapter 0:2
MQSA's quality standards and the related accreditation process have
had a substantial effect on improving the quality of services. FDA
substantially based its standards on an accreditation program
developed by the American College of Radiology (ACR), a private,
nonprofit professional association of radiologists. When MQSA
initially took effect, many mammography units did not meet the
standards. For example, between October 1, 1994, and August 1, 1995,
about 2,600 (35 percent) of the mammography units that sought ACR
accreditation (the main accrediting body used by FDA) initially
failed to meet accreditation requirements. While almost all of these
units were eventually granted full accreditation, they first had to
demonstrate that they had improved their quality assurance
activities. The accreditation process continues to result in more
facilities meeting quality assurance standards. For example, the
percentage of facilities that obtained ACR accreditation on the first
attempt increased from 66 percent in 1995 to 82 percent in 1997.
MQSA's inspection requirement gives FDA another means to ensure that
facilities comply with standards on a day-to-day operating level.
While accreditation is generally based on a review of written
application materials, annual inspections are conducted on site,
which allows inspectors to verify information provided during the
accreditation process. In January 1997 we reported on some initial
problems with FDA's inspection program, such as inconsistent
approaches in conducting inspections and reporting results, as well
as weaknesses in follow-up systems to ensure that facilities were
correcting deficiencies in a timely manner.\2 FDA has taken the
actions that we recommended in our report to remedy these operational
problems.
As FDA entered its third year of annual inspections, inspection
results continued to show improvement in facility compliance with
MQSA quality standards. As figure 2 shows, the percentage of
facilities found to have violations dropped in every violation
category during the first 3 years of the inspection program.
Figure 2: Comparison of
Mammography Facilities'
Inspection Results, Fiscal
Years 1995-97
(See figure in printed
edition.)
Level 1 violations are those that can have the most detrimental
effect on the quality of mammography services--for example, a
facility's having personnel who do not meet FDA's minimum
qualification standards. Level 2 violations are considered less
significant--for example, a facility's not having an evaluation of
equipment by a medical physicist during the past 14 months. FDA
considers both level 1 and level 2 violations to be significant
violations for which it requires the facility to submit written plans
for corrective actions. From fiscal year 1995 to fiscal year 1997,
the proportion of inspected facilities whose highest-level violation
was a level 1 dropped from 3 percent to 1 percent, and the proportion
whose highest-level violation was a level 2 dropped from 20 percent
to 12 percent.
Level 3 violations are more minor in nature--for example, a
facility's not having the required documentation of a quality control
test--and do not require a written response. Most facilities with
violations had only level 3 violations; these numbers also dropped
during the 3-year period. Consistent with the decrease in the number
of violations, the proportion of facilities with no violations at all
nearly doubled, from about 30 percent in 1995 to about 55 percent in
1997.
While these inspection results relate to the quality of mammography
services in general, one particular aspect of inspections can be tied
more directly to the quality of the image itself. The quality of the
image is tested through a "phantom image test" involving an X ray of
a plastic block that contains 16 test objects to determine how many
of the objects are visible on the resulting image produced by the
facility's mammography equipment.\3 Because FDA was already
conducting some phantom image testing before MQSA, it is possible to
make general comparisons of image quality before and after the law
took effect. In 1992, before the law was implemented, 11 percent of
the facilities failed the test. Since 1995, the failure rate has
been about 2 percent.
--------------------
\2 FDA's Mammography Inspections: While Some Problems Need
Attention, Facility Compliance Is Growing (GAO/HEHS-97-25, Jan. 27,
1997).
\3 The plastic block, which represents an average-sized compressed
breast, contains a wax insert that holds 16 test objects--6 fibrous
structures (fibers), 5 embedded microcalcifications (speck groups),
and 5 different-sized tumor-like masses that simulate growths that
could be cancerous.
DATA AND METHODOLOGY PROBLEMS
PREVENT MEASURING MQSA'S IMPACT
ON SAVING LIVES
---------------------------------------------------------- Chapter 0:3
Although MQSA can be linked to improved quality of mammography
images, it is difficult to say to what extent it has helped to
improve mammography interpretation or increase the frequency of early
detection of cancer which has been shown to save lives. Researchers
face several methodological challenges in clearly demonstrating these
links.
The first problem is that criteria have not been established to
measure how well images are being interpreted by radiologists. Some
experts have suggested that some type of peer review or proficiency
test would help improve accuracy, but FDA and others have argued that
such measures are too difficult and costly to implement and that each
facility should instead set up a system to monitor the performance of
its interpreting physicians.
The second problem is the overall lack of sufficient clinical data to
assess MQSA's contribution to mammography performance and resulting
cancer outcomes. For example, while various regional, state, and
local cancer registries collect clinical data on breast cancer cases,
including stage and tumor size, the data do not indicate whether the
cancer was detected by mammography.
A third and overarching problem is that no clear consensus exists on
appropriate performance and outcome measures for mammography.
Experts we talked with pointed out the following areas of analytical
and definitional complications.
-- Researchers do not agree on how to decide when a mammogram is a
false negative. For example, under a 12-month follow-up
scenario, if a mammogram is interpreted as negative and during
the 12 months after the mammogram a cancer is detected, then the
negative mammogram is considered a false negative. Some
researchers believe the follow-up period should be as little as
7 months, while others believe it should be a year or more. The
number of false negatives identified will be affected by the
length of the follow-up period that is used.
-- Because the ultimate goal of mammography is to reduce breast
cancer mortality, some studies have used mortality reduction as
a measure of mammography effectiveness. However, because women
with breast cancer generally survive longer than 5 years,
measuring changes in mortality reduction requires a long
follow-up time. Because MQSA has been in effect for only a few
years, not enough time has elapsed to permit use of mortality
reduction as a measure for assessing MQSA's effect on cancer
outcomes.
-- As an alternative to mortality reduction rates, some researchers
have used the early-stage cancer detection rate as an
intermediate measure of the effectiveness of mammography.\4 But
using this rate to measure the effectiveness of mammography,
also poses methodological difficulties. The most controversial
problem concerns whether or not cases of carcinoma in situ (CIS)
should be included when measuring mammography effectiveness.
While CIS is considered the earliest stage of breast cancer,
some experts view CIS as a very slow-growing, noninvasive tumor.
They contend that including CIS cases as early-stage cancer
overstates the benefit of mammography in reducing mortality.
Others argue that some CIS tumors can grow quickly and develop
into serious, even fatal, disease and that, therefore, CIS cases
should be considered early-stage cancer.
While these methodological problems are formidable, they do not
appear to be insurmountable, and, in fact, progress is being made
toward resolution. As authorized by MQSA, the National Cancer
Institute in 1994 established the Breast Cancer Surveillance
Consortium, which consists of nine federally funded research projects
at sites in most of the major geographical regions of the United
States. These projects collect data from affiliated area mammography
practices and link mammography data to cancer registry data. The
Consortium's goal is to ensure that the data collected can be
analyzed to address issues relating to mammography performance and
cancer outcomes. The Consortium has spent considerable time
developing a consensus on which data elements to collect in order to
obtain consistent and reliable data, and it is also working to
develop a consensus of common definitions for measurement. By the
year 2000, the Consortium expects to have sufficient data to analyze
mammography performance and cancer outcomes. Consortium efforts have
the potential to greatly strengthen the quality of information about
the impact of mammography on improving cancer outcomes. However,
they are unlikely to provide substantial information about MQSA's
specific effect because data are not available for a pre- and
post-MQSA comparison.
--------------------
\4 The most widely used approach to categorizing cancers involves the
use of staging classifications, with the size of the tumor being one
of the most important factors in determining the stage of the cancer.
MQSA HAS NOT LIMITED ACCESS
---------------------------------------------------------- Chapter 0:4
When MQSA was passed, the Congress was concerned that access to
mammography services might be limited because many providers might
choose to drop mammography services rather than upgrade facility
operations to comply with the standards. Our analysis of data
collected by FDA shows that of the approximately 10,000 facilities
offering mammography before MQSA, about 4 percent, or 404 facilities,
closed during the 12 months preceding MQSA's October 1994
implementation. Data for December 1994 to March 1997 showed a net
loss of 163 mammography facilities.
We found no evidence, however, that these closures adversely affected
access to mammography services. For example, almost all of the
closed facilities were located within 25 miles of another certified
mammography facility. Further, state health officials from those
states with the greatest numbers of closures generally told us that
many of the closed facilities were either low-volume providers that
did not generate enough revenue to cover the costs of meeting MQSA
requirements or poor-quality providers that could not meet
accreditation requirements. Officials in several states said that a
sizable number of the facilities identified as closed--more than 40
percent--continued to provide services in some form. For example,
some facilities continued to provide services by merging with another
facility or consolidating their equipment from satellite clinics.
Others contracted with mobile service providers to continue serving
their patients.
CONCLUSIONS
---------------------------------------------------------- Chapter 0:5
As the Congress considers reauthorization of MQSA, two points are
clear. First, overall, MQSA has had a positive impact on the quality
of mammography services and no adverse effect on women's ability to
obtain access to mammography. We believe it is reasonable to
attribute a large part of the quality improvement to MQSA processes
that enforced accreditation standards that were not previously met by
many facilities and to FDA's annual inspection process, which
provides a valuable, systematic means of helping ensure that these
higher standards are maintained.
Second, quantifying MQSA's effect on the accuracy of mammogram
interpretation and on the improvement of cancer outcomes is much more
problematic. Although data collection efforts now under way will
probably make it easier to monitor the quality of image
interpretation and the effects of mammography on cancer outcomes in
the future, the absence of pre-MQSA data means that analysts may not
be able to fully measure how the act itself has affected these two
areas.
-------------------------------------------------------- Chapter 0:5.1
Mr. Chairman, this concludes my prepared statement. I will be happy
to answer any questions that you or Members of the Subcommittee may
have.
*** End of document ***