Human Health Risk Assessment: EPA Has Taken Steps to Strengthen  
Its Process, but Improvements Needed in Planning, Data		 
Development, and Training (31-MAY-06, GAO-06-595).		 
                                                                 
Over 100,000 chemicals, pollutants, and toxic substances are used
in the United States and regulated by the Environmental 	 
Protection Agency (EPA). EPA uses risk assessment to determine	 
the health risk from exposure to these substances, collectively  
referred to as contaminants. In the last 12 years, independent	 
reviewers have examined this process and made recommendations for
how it could be improved. GAO was asked to (1) identify the	 
significant recommendations that have been made to improve human 
health risk assessment; (2) describe what EPA has done to modify 
its human health risk assessment process; (3) determine the	 
effects these past modifications have had on the preparation of  
risk assessments; and (4) identify any additional actions experts
believe EPA could take to improve its process, and the barriers  
it would face in doing so.					 
-------------------------Indexing Terms------------------------- 
REPORTNUM:   GAO-06-595 					        
    ACCNO:   A54902						        
  TITLE:     Human Health Risk Assessment: EPA Has Taken Steps to     
Strengthen Its Process, but Improvements Needed in Planning, Data
Development, and Training					 
     DATE:   05/31/2006 
  SUBJECT:   Chemicals						 
	     Contaminants					 
	     Environmental research				 
	     Evaluation methods 				 
	     Health hazards					 
	     Internal controls					 
	     Policy evaluation					 
	     Pollutants 					 
	     Risk assessment					 
	     Toxic substances					 

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO Product.                                                 **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
******************************************************************
GAO-06-595

     

     * Results in Brief
     * Background
     * Independent Reviewers Have Recommended Improvements to EPA's
          * Improve Planning and Review Process and Ensure an Appropriat
               * Improve Planning and Adopt an Iterative Risk Assessment Appr
               * Increase Peer Review and Stakeholder Involvement in the Risk
               * Ensure EPA Has the Skilled Workforce Needed to Conduct Quali
               * Improve Characterization of Risk and More Closely Link Risk
          * Review Existing Data, Identify Data Gaps, Prioritize Researc
          * More Clearly Indicate Use of Default Options and How They We
          * Better Evaluate Available Methods and Models and Incorporate
          * More Explicitly Analyze and Characterize the Sources of Unce
          * Enhance Analysis of Variability in Exposure Levels and Healt
          * Better Consider the Human Health Effects of Exposure to Mult
     * EPA Has Strengthened Many Facets of Its Risk Assessment Proc
          * EPA Has Enhanced Implementation of Key Aspects of Its Risk A
               * EPA Has Issued Many Agencywide and Program Office-Specific G
               * EPA Has Built Scientific Capacity through Increased Focus on
               * EPA Has Begun to Incorporate Planning Activities and Stakeho
               * EPA Has Refined Its Peer Review and Quality-Assurance Practi
          * EPA Has Enhanced Its Strategic Planning and Refocused Staff
               * EPA Restructured and Augmented Its Research Office
               * EPA Has Enhanced Strategic Research Planning
               * EPA Program Offices Have Established Collaborative Relations
               * EPA Maintains Databases of Information Related to Risk Asses
          * EPA Has Improved Its Choice and Communication of Default Opt
          * EPA Has Taken Steps to Enhance Its Modeling Capabilities and
          * EPA Is Introducing More Quantitative Characterization of Unc
          * EPA Better Analyzes and Communicates Variability in both Exp
          * EPA Has Begun to Consider the Combined Effects of Exposure t
     * EPA Risk Assessors Responding to Our Survey Reported That Pr
          * EPA Has Issued Numerous Guidance Documents That Have Been Ge
          * Collaboration Has Improved EPA's Ability to Conduct Risk Ass
          * EPA's Training Has Been Helpful, but Risk Assessors and Mana
               * Risk Assessors Have Enhanced Their Skills through Training i
               * Additional Training Is Needed
     * Enhanced Risk Assessment Planning, Improved Analysis and Rev
          * Enhance Planning by Increasing Focus on Data Needs and Invol
          * More Thoroughly Evaluate and Transparently Document Analytic
          * More Training Could Improve Risk Assessor, Risk Manager, and
          * EPA Faces Barriers to Improving Its Risk Assessment Process,
     * Conclusions
     * Recommendations for Executive Action
     * Agency Comments
     * GAO Contact
     * Staff Acknowledgments
     * GAO's Mission
     * Obtaining Copies of GAO Reports and Testimony
          * Order by Mail or Phone
     * To Report Fraud, Waste, and Abuse in Federal Programs
     * Congressional Relations
     * Public Affairs

Report to the Chairman, Committee on Environment and Public Works, U.S.
Senate

United States Government Accountability Office

GAO

May 2006

HUMAN HEALTH RISK ASSESSMENT

EPA Has Taken Steps to Strengthen Its Process, but Improvements Needed in
Planning, Data Development, and Training

GAO-06-595

Contents

Letter 1

Results in Brief 5
Background 8
Independent Reviewers Have Recommended Improvements to EPA's Risk
Assessment Process Since 1994 12
EPA Has Strengthened Many Facets of Its Risk Assessment Process Since
1994, and Efforts Are Ongoing 24
EPA Risk Assessors Responding to Our Survey Reported That Process
Modifications Have Helped Them Prepare Better Risk Assessments but That
Collaboration and Training Limitations Hamper Further Progress 51
Enhanced Risk Assessment Planning, Improved Analysis and Review, and Added
Training Could Further Improve EPA's Process, but Barriers Could Limit
Progress 62
Conclusions 67
Recommendations for Executive Action 69
Agency Comments 69
Appendix I Objectives, Scope, and Methods 71
Appendix II GAO Contact and Staff Acknowledgments 75

Figures

Figure 1: Typical Sequence of Risk Assessment and Risk Management
Processes 9
Figure 2: Helpfulness of Office-Specific Guidance 53
Figure 3: Risk Assessors That Generally or Strongly Agree That
Organizations Help Fill Data Needs 55
Figure 4: Areas in Which Training Has Moderately or Greatly Improved Risk
Assessor Knowledge and Skills 58
Figure 5: Types of Training Taken by Risk Assessors 59
Figure 6: Modifications in Training That Would Moderately or Greatly
Improve the Preparation of Risk Assessments 61

Abbreviations

ASPH Association of Schools of Public Health

ATSDR Agency for Toxic Substances and Disease Registry

BOSC Board of Scientific Counselors

Commission Presidential/Congressional Commission on Risk Assessment and
Risk Management

CREM Council for Regulatory Environmental Modeling

EPA Environmental Protection Agency

FQPA Food Quality Protection Act

GIS Geographic Information System

HEI Health Effects Institute

HPV High Production Volume

ILSI-RSI International Life Sciences Institute-Risk Science Institute

IRIS Integrated Risk Information System

NAS National Academy of Sciences

NCCT National Center for Computational Toxicology

NCEA National Center for Environmental Assessment

NCER National Center for Environmental Research

NERL National Exposure Research Laboratory

NHEERL National Health and Environmental Effects Research Laboratory

NHEXAS National Human Exposure Assessment Survey

NIEHS National Institute for Environmental Health Sciences

NRMRL National Risk Management Research Laboratory

OAQPS Office of Air Quality Planning and Standards

OMB Office of Management and Budget

OP Office of Pesticides

OPPT Office of Pollution Prevention and Toxics

ORD Office of Research and Development

OW Office of Water

STAR Science to Achieve Results

TRIM Total Risk Integrated Methodology

TSCA Toxic Substances Control Act

This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.

United States Government Accountability Office

Washington, DC 20548

May 31, 2006 May 31, 2006

The Honorable James M. Inhofe Chairman Committee on Environment and Public
Works United States Senate The Honorable James M. Inhofe Chairman
Committee on Environment and Public Works United States Senate

Dear Mr. Chairman: Dear Mr. Chairman:

Chemicals play an important role in people's everyday lives. Consumers use
products containing chemicals, such as cleansers and pesticides, and
companies use a variety of toxic substances as solvents or additives to
their industrial processes. Although these substances have beneficial uses
and are important in producing goods and services, some may adversely
affect human health. Over 100,000 chemicals, pollutants, and toxic
substances are used in the United States and regulated by the
Environmental Protection Agency (EPA). These substances, whether produced
in the United States or imported, include 187 hazardous air pollutants,
nearly 20,000 pesticide products on the market, and a growing number of
substances used in industrial processes-over 82,000 as of December 2005.
EPA uses a process known as risk assessment to estimate the health risk
from exposure to these substances, collectively referred to in this report
as contaminants.1 While highly technical by nature, risk assessments,
along with other relevant information, serve as a basis for regulatory
decisions that protect human health. EPA's risk assessments are shaped by
available scientific information and by provisions in major environmental
statutes, such as the Clean Air Act, and they affect a range of
stakeholders, including regulated industries, federal agencies,
environmental advocacy groups, academic and other researchers, and the
public. 21 2Chemicals play an important role in people's everyday lives.
Consumers use products containing chemicals, such as cleansers and
pesticides, and companies use a variety of toxic substances as solvents or
additives to their industrial processes. Although these substances have
beneficial uses and are important in producing goods and services, some
may adversely affect human health. Over 100,000 chemicals, pollutants, and
toxic substances are used in the United States and regulated by the
Environmental Protection Agency (EPA). These substances, whether produced
in the United States or imported, include 187 hazardous air pollutants,
nearly 20,000 pesticide products on the market, and a growing number of
substances used in industrial processes-over 82,000 as of December 2005.
EPA uses a process known as risk assessment to estimate the health risk
from exposure to these substances, collectively referred to in this report
as contaminants. While highly technical by nature, risk assessments, along
with other relevant information, serve as a basis for regulatory decisions
that protect human health. EPA's risk assessments are shaped by available
scientific information and by provisions in major environmental statutes,
such as the Clean Air Act, and they affect a range of stakeholders,
including regulated industries, federal agencies, environmental advocacy
groups, academic and other researchers, and the public.

1Although other regulatory agencies, such as the Food and Drug
Administration, also use risk assessments as part of regulatory decision
making, this report discusses only the risk assessment process used by
EPA.

2EPA's mission is to protect human health and to safeguard the natural
environment-air, water, and land-upon which life depends. This report
focuses only on EPA's efforts to protect human health.

In 1994, as part of a congressionally mandated review of the methods used
by EPA to estimate the risk of developing cancer from exposure to
hazardous air pollutants, the National Academy of Sciences (NAS) issued a
report entitled Science and Judgment in Risk Assessment. This report
focused primarily on hazardous air pollutants and provided an overview
assessment of EPA's risk assessment methods.3 In providing this overview,
the NAS committee identified several themes, largely focusing on the
quality and availability of essential information, that serve as a useful
framework for discussing the risk assessment process:

           o  Implementation. EPA faces certain overarching, institutional
           issues that affect its implementation of the risk assessment
           process.

           o  Data needs. Data necessary to complete scientifically plausible
           risk assessments are often unavailable on such topics as the
           actual levels of exposure to contaminants and how those exposures
           affect human health.

           o  Default options. In the absence of convincing scientific
           knowledge or data, EPA relies on assumptions, often conservative
           in nature, about such questions as how exposure to low doses of a
           contaminant affects human health.

           o  Method and model evaluation. The predictive accuracy of methods
           and models, such as those used to predict how a contaminant will
           be processed once it enters the body, is not always known.

           o  Uncertainty. The lack of precise knowledge about the type,
           likelihood, and extent of adverse effects from exposure to a
           contaminant results in uncertainty in risk assessment that can be
           reduced only by advances in scientific understanding or the
           collection of better data.

           o  Variability. Variability, in exposure or in the biological
           differences among humans that determine how exposure to
           contaminants affects health, can be better characterized with more
           data but cannot be reduced or eliminated.

           o  Aggregate and cumulative effects. Aggregate effects are the
           results from exposure to a single contaminant by any combination
           of means-including inhalation, contact with the skin, and
           ingestion; cumulative effects are the results from exposure to
           multiple contaminants by any combination of means.

           Science and Judgment in Risk Assessment was one of several reports
           by NAS that discussed risk assessment in general and made
           recommendations for ways to improve it.4 In addition, the
           Presidential/Congressional Commission on Risk Assessment and Risk
           Management issued a report in 1997 that reviewed risk assessment,
           as well as risk management, in regulatory programs.5 Furthermore,
           the Office of Management and Budget (OMB) issued guidance related
           to peer review and ensuring the quality, objectivity, utility, and
           integrity of information released to the public.6 More recently,
           in January 2006, OMB issued a proposed bulletin that advocates
           minimum standards for the scientific quality of risk assessments
           to enhance their technical quality and objectivity.

           In the context of these prior reviews, you asked us to (1)
           identify the significant recommendations to improve human health
           risk assessment that have been made since 1994; (2) describe what
           EPA has done to modify its human health risk assessment process
           over the same period; (3) determine the effects these past
           modifications have had on the preparation of risk assessments; and
           (4) identify any additional actions experts believe EPA could take
           to improve its risk assessment process in the future, and the
           barriers EPA would face in doing so.

           To identify significant recommendations to improve human health
           risk assessment since 1994, we reviewed EPA documents, including
           those produced by EPA's Risk Assessment Forum, Science Policy
           Council, Council for Regulatory Environmental Modeling, and
           Science Advisory Board, as well as each of EPA's program offices.
           We also reviewed our own reports and documents produced by NAS and
           the Presidential/ Congressional Commission on Risk Assessment and
           Risk Management. We also spoke with experts in the risk assessment
           field, who identified many of these documents in the course of our
           discussions and provided insight into some of the recommendations
           made therein. To describe what EPA has done to modify its human
           health risk assessment process, we interviewed program office
           managers from EPA's Office of Air and Radiation, Office of
           Pesticide Programs, Office of Pollution Prevention and Toxics,
           Office of Water, and Office of Research and Development (ORD).
           While we recognize that EPA's regional offices and the Office of
           Solid Waste and Emergency Response conduct risk assessments, for
           the most part their work focuses on site-specific assessments and
           thus was not included in this effort. Within ORD, we interviewed
           managers in two of EPA's laboratories (the National Health and
           Environmental Effects Laboratory and the National Exposure
           Research Laboratory) and three of EPA's research centers (the
           National Center for Environmental Assessment, National Center for
           Environmental Research, and National Center for Computational
           Toxicology). Furthermore, we attended various EPA and stakeholder
           group training sessions and meetings.

           To determine the effects these modifications have had on the
           preparation of risk assessments, we surveyed risk assessors from
           the program offices mentioned above. We sought their views on the
           usefulness of many of these modifications, as well as on aspects
           of EPA's risk assessment process, including guidance documents,
           training, organizational structure, and collaboration. The survey
           and results can be viewed at
           http://www.gao.gov/cgi-bin/getrpt?GAO-06-637SP . In addition, we
           interviewed experts in the risk assessment field and obtained
           their perspectives on the extent to which these modifications were
           helpful to EPA's risk assessment process. To identify additional
           actions experts believe EPA could take to improve its process and
           to identify barriers to such actions, we contacted experts
           representing a range of stakeholders in the process. Specifically,
           we contacted risk assessment scientists; toxicologists; scientific
           advisers to EPA; state officials; and representatives from
           regulated industries, government agencies, and environmental
           advocacy groups who have an expertise in risk assessment. We used
           an iterative process (often referred to as the "snowball sampling"
           technique) to identify these knowledgeable experts and selected
           for interviews those who would provide us with a broad and
           balanced range of perspectives on EPA risk assessment practices.
           We used a standard set of questions to interview each of these
           experts to ensure we consistently discussed each aspect of EPA
           risk assessment policies and practices. (See app. I for a more
           detailed description of the methodology we employed.) In addition,
           we consulted with NAS's Board on Environmental Studies and
           Toxicology at its June 2005 meeting to obtain input on the areas
           in which EPA has made the most progress in improving its risk
           assessment practices and areas EPA will need to focus on in the
           future. We used the experts present at this meeting as a starting
           point for our snowball sampling technique used to identify
           subsequent experts. We conducted our work from February 2005
           through March 2006 in accordance with generally accepted
           government auditing standards.

           Results in Brief
			  
			  Independent reviewers, including NAS, have made a number of
           significant recommendations to improve EPA's human health risk
           assessment process since 1994 that can be grouped and discussed in
           terms of the seven themes originally identified by NAS in 1994.
           First, reviewers have said that EPA should improve its overall
           implementation of the process with such steps as more thorough
           planning of its risk assessments and greater use of independent
           reviews by scientists (called peer reviews). Second, to improve
           the quantity and quality of the data upon which risk assessments
           are based, reviewers have recommended that EPA better utilize the
           scientific data it has and identify additional data needs on the
           potential adverse effects from exposure to contaminants, and
           prioritize and support research to meet the data needs identified.
           Third, because EPA lacks data to fully assess the health risk from
           exposures, reviewers have recommended that when the agency relies
           on assumptions-known as default options-it should, among other
           things, more clearly indicate when it relies on default options
           and how it chooses them. Fourth, to improve the accuracy of EPA's
           risk assessments, reviewers have stated that EPA should better
           evaluate the methods and models used in its analysis and
           incorporate newer, more powerful tools when appropriate. Fifth, to
           address the inevitable uncertainties associated with gaps in
           scientific knowledge and general unknowns about model and data
           accuracy, EPA should, according to reviewers, more explicitly
           analyze and characterize the sources of uncertainty in its risk
           assessments and, when possible, discuss the uncertainties both
           descriptively (qualitatively) and numerically (quantitatively).
           Sixth, reviewers have also recommended that EPA enhance its
           analysis of variability in levels of exposures to contaminants, as
           well as differences in individual reactions to exposure. Finally,
           because people are typically exposed to a mixture of contaminants
           through a variety of means, such as contact with skin and
           breathing air, reviewers have recommended that EPA improve how it
           considers the effects of combinations of contaminants and all
           possible means of exposure.

           EPA has strengthened many aspects of its risk assessment process
           since 1994, and improvement efforts are ongoing. First, EPA has
           improved implementation of its risk assessment process by, for
           example, beginning to improve risk assessment planning and
           creating scientific leadership positions, such as the Office of
           the Science Advisor. Second, EPA has initiated actions to develop
           missing or incomplete scientific data on the potential adverse
           effects from exposure to contaminants. For example, EPA officials
           told us they have begun to use the planning and review processes
           to determine what data are needed and communicate these needs to
           both EPA and outside researchers. Third, EPA has cited the need
           for risk assessments to be more transparent about their use of
           default options. For example, EPA has recently issued guidance
           describing default options that are appropriate for certain
           purposes and directing risk assessments to disclose the default
           options used in a particular risk assessment. Fourth, EPA is
           enhancing its use of models and embracing new methodologies. For
           example, EPA has established a group-the Council for Environmental
           Regulatory Modeling-to review models and provide guidance in model
           selection. Furthermore, some program offices now use more advanced
           models to predict how a contaminant will be processed once it
           enters the body. Fifth, EPA generally characterizes uncertainty
           descriptively, but has begun to incorporate quantitative
           techniques into its uncertainty analyses. For example, EPA
           typically uses a tiered approach to conducting uncertainty
           analysis, starting as simply as possible, describing uncertainty
           qualitatively, and sequentially employing more sophisticated
           analysis, such as probabilistic analysis, as warranted. Sixth, EPA
           has made progress in describing variability due to differences in
           both the exposure and the susceptibility of exposed individuals to
           contaminants. For example, when determining a level of exposure
           that is unlikely to be harmful, EPA includes adverse effects for
           people who might be at increased risk because of their age or the
           state of their health. Finally, EPA has begun to take steps to
           consider the combined effects of exposure to multiple contaminants
           through multiple means of exposure. For example, EPA has developed
           a framework to assess the combined, or cumulative, risk and has
           directed its program offices to include cumulative risks when
           planning major risk assessments.

           EPA risk assessors believe the modifications EPA has made over the
           past 10 years-particularly issuing additional guidance-have
           generally helped them improve risk assessments, but collaboration
           and training could be improved. Most EPA risk assessors believe
           that agencywide guidelines and policy and reference documents have
           helped them prepare risk assessments by, for example, providing
           useful frameworks for evaluating potential harm from chemicals,
           and have resulted in greater consistency among risk assessments
           prepared by different offices within EPA. Furthermore, EPA's
           collaboration with external researchers, including other federal
           research entities, academia, and industry, has improved EPA's
           ability to conduct risk assessments by providing expertise and
           research not always available within the agency, according to risk
           assessors. In addition, risk assessors reported that internal
           collaboration among EPA offices is moderately to very effective,
           but could be improved. For example, a few risk assessors commented
           that the program offices' knowledge of the regulatory context in
           which research will be used helps the researchers structure their
           work. However, risk assessors also reported that collaboration is
           hindered within EPA by, for example, conflicting priorities among
           the various offices and the poor communication between some of
           them. Finally, while risk assessors said improved training has
           also helped them gain relevant skills and knowledge, over 70
           percent of the risk assessors responding to our survey stated that
           more in-depth or relevant training would improve their ability to
           prepare risk assessments. In addition, they believe training for
           risk assessors and managers in specific technical and scientific
           areas, such as emerging scientific issues and the use of newer
           models, is lacking.

           While the experts we spoke with said the modifications EPA has
           made over the past 10 years have been beneficial overall, they
           identified additional actions EPA could take to improve its risk
           assessment process, recognizing that EPA may face barriers to
           doing so. Specifically, experts said EPA could improve the
           planning it undertakes prior to starting a risk assessment by
           better focusing on what data are needed for the assessment and by
           involving stakeholders early in this planning process. For
           example, several experts said that increased involvement with a
           broad range of stakeholders early in the planning process would
           help identify alternative methods and models and obtain
           stakeholder concurrence with the agency's approach. In addition,
           experts said EPA could more thoroughly evaluate methods and
           models, transparently document its analytic choices, and enhance
           internal review. For example, several experts said that EPA should
           more transparently communicate which default assumptions were used
           in risk assessments, why the defaults were chosen, and what
           judgments EPA was making when it employed certain methods.
           Finally, experts said EPA could provide additional training for
           risk assessors, managers, and stakeholders on all elements of the
           risk assessment process, such as how to use and apply models and
           how to interpret data from emerging scientific fields. While these
           efforts would further improve the risk assessment process, experts
           pointed out inherent barriers that EPA may face in carrying them
           out, such as the scientific complexity of risk assessment, the
           difficulty of obtaining and applying data, and a cultural
           resistance to deviating from established methods. For example,
           several experts said that EPA's risk assessments have grown more
           technically challenging and require risk assessors and managers to
           have different skills from what they had in the past.

           To further improve the risk assessment process, GAO recommends
           that EPA enhance early planning of each risk assessment, identify
           and communicate data needs to the research community, and support
           development and implementation of in-depth training for risk
           assessors and managers.

           We provided EPA with a draft of this report for review and
           comment. EPA neither agreed nor disagreed with our findings and
           recommendations. However, the agency provided specific comments to
           improve the report's technical accuracy, which we incorporated as
           appropriate.

           Background
			  
			  Risk assessment, as used in public health and environmental
           settings, is inherently a complex and highly technical process
           that provides a systematic scientific description of potential
           adverse effects from exposure to contaminants. Risk assessments
           are conducted to estimate whether and how much harm can be
           expected from exposure to a given contaminant or mixture of
           contaminants and to help determine whether the harm is significant
           enough to require regulatory or other corrective action. The
           adverse effects from exposure can be acute, such as respiratory
           illnesses or poisonings, or chronic, such as cancer or
           reproductive or genetic abnormalities. The result of a risk
           assessment is a statement of the likelihood that an exposed
           population will be harmed and to what extent. EPA's human health
           risk assessment process generally consists of the four stages
           recommended in 1983 by the National Academy of Sciences (NAS): (1)
           hazard identification (determining whether a substance could cause
           adverse affects and what those adverse affects would be), (2)
           dose-response assessment (determining the relationship between the
           magnitude of exposure to a contaminant and the probability and
           severity of adverse effects), (3) exposure assessment (identifying
           the extent to which exposure is predicted to occur), and (4) risk
           characterization (combining the information from the preceding
           analyses into a conclusion about the nature and magnitude of
           risk).

           Risk assessment plays a critical role in EPA's regulatory
           decisions, but the characterization of risk that results from a
           risk assessment is only one piece of information used to inform
           decisions on regulatory standards for protecting human health.
           This analysis is integrated with other information, such as
           economic information on the costs and benefits of mitigating the
           risk, technological information on the feasibility of managing the
           risk, and the concerns of various stakeholders. The combined
           analysis informs agency officials who ultimately make regulatory
           decisions. This overall process is generally called risk
           management. (See fig. 1.) According to NAS, the risk assessment
           process should be conceptually distinguished from how the results
           of that process are used in the risk management process. However,
           in practice, maintaining this separation is difficult because the
           two processes are fundamentally linked by the complementary needs
           of the risk assessors and the risk managers. Specifically, risk
           assessors need to be aware of the context in which the assessment
           will be used and to communicate their key findings, as well as
           their confidence in them, in a way that risk managers can
           understand and apply. Risk managers need to ensure that the risk
           assessment has been properly performed and can be integrated with
           other information to make and justify a sound regulatory decision.

           Figure 1: Typical Sequence of Risk Assessment and Risk Management
           Processes

           EPA's human health risk assessment process is limited to a great
           extent by the amount of scientific data available on contaminants
           and by the level of scientific understanding about how
           contaminants act in the body to produce adverse effects. Data are
           often unavailable, and science cannot always provide definitive
           answers. The complex and sophisticated analytic tools used in risk
           assessment cannot overcome basic scientific unknowns. As a result,
           EPA's risk assessments almost always include assumptions about
           potential adverse effects. Some assumptions are conservative-that
           is, they are intended to help ensure that the agency does not
           underestimate health risks. Some critics of EPA's risk assessment
           practices believe EPA uses assumptions that are unjustifiably
           conservative, given new scientific data and methods, and thereby
           produces estimates that overstate actual risks. Other
           stakeholders, however, criticize some agency assumptions for not
           being conservative enough in the face of scientific uncertainties.
           These criticisms often result in legal challenges to EPA's
           decisions.

           EPA's human health risk assessment process is also greatly
           dependent on its degree of knowledge about the population's level
           of exposure to contaminants. This knowledge includes the extent to
           which people are exposed to potentially harmful contaminants in
           their daily lives, the chemicals to which they are most often
           exposed, the levels of such exposures, how exposures change over
           time, and the sources of exposure. Risk assessors, researchers,
           and policymakers must often rely on estimates of human exposure
           that are often derived from data showing the extent the chemicals
           are found in the air, water, food, or other environmental media
           and assumptions about how and at what rate the body absorbs the
           chemicals it contacts. In addition to estimates from models,
           extrapolations from experiments involving animals, and
           measurements of chemicals in the environment, EPA also relies on
           more direct methods to measure exposure and more accurately assess
           exactly how much of a contaminant has been absorbed in the body.
           For example, EPA uses population activity models-models based on
           actual human behavior, such as the time spent outdoors or, for
           children, the amount of time spent on the floor-to better estimate
           an individual's true exposure.

           Risk assessment activities at EPA are carried out by both the
           agency's Office of Research and Development (ORD)-its principal
           scientific and research arm-and its program and regional offices,
           including the Office of Air and Radiation, Office of Pesticide
           Programs, Office of Pollution Prevention and Toxics, Office of
           Solid Waste and Emergency Response, and Office of Water. ORD
           carries out all steps of highly complex, precedent-setting risk
           assessments for specific contaminants, such as dioxin. In
           addition, ORD often has responsibility for the first two steps of
           the risk assessment process-hazard identification and
           dose-response assessment-in support of the program offices. In
           such cases, the last two steps-exposure assessment and risk
           characterization-are the responsibility of the various program
           offices. Three notable exceptions exist to this division of
           responsibility. The Office of Pesticide Programs and the Office of
           Pollution Prevention and Toxics often conduct all steps of risk
           assessments independently, in part, because the relevant statutes
           for these program offices place strict time frames on decision
           making and the confidential nature of data provided to the agency
           under these programs. Furthermore, the Office of Water also does
           all of the stages for some of the assessments for purposes of the
           Safe Drinking Water Act.

           EPA's approach to risk assessment varies across program offices,
           often as a result of different regulatory and legal requirements.
           For example, a branch of the Office of Pollution Prevention and
           Toxics assesses data submitted by industry applicants on
           approximately 2000 new chemicals annually under the Toxic
           Substances Control Act (TSCA) of 1976. TSCA generally requires EPA
           to evaluate the chemicals within 90 days, but does not require all
           applicants to conduct laboratory tests on the potential hazards
           and risks of the chemicals. In contrast, the Clean Air Act
           requires EPA to establish National Ambient Air Quality Standards,
           to review the scientific basis for those standards at least every
           five years, and to revise the standards as appropriate. As part of
           this process, ORD summarizes the most current scientific
           information on the pollutant in question. ORD's findings and
           conclusions are then combined with other exposure and risk
           analyses to determine what, if any, revisions should be made to
           the standards.

           Risk assessment has been the center of numerous reports, analyses,
           and regulations over the years. One of the earliest was the
           National Academy of Sciences' 1983 report Risk Assessment in the
           Federal Government: Managing the Process, often referred to as the
           "Red Book" because of the color of its cover. Subsequently, NAS
           released several related reports, including Science and Judgment
           in Risk Assessment (1994), Understanding Risk: Informing Decisions
           in a Democratic Society (1996), and Strengthening Science at the
           U.S. Environmental Protection Agency (2000). In addition,
           Congress, via the Clean Air Act Amendments of 1990, created the
           Presidential/Congressional Commission on Risk Assessment and Risk
           Management (Commission) and required it to investigate the policy
           implications and appropriate uses of risk assessment and risk
           management in various regulatory programs designed to protect
           people from cancer and other chronic health effects that may
           result from exposure to hazardous substances. The Commission
           published its two-volume final report in 1997.

           In addition, the Office of Management and Budget (OMB) has issued
           guidelines, reports, and bulletins that have affected the practice
           of risk assessment. For example, in October 2001, OMB issued its
           Guidelines for Ensuring and Maximizing the Quality, Objectivity,
           Utility, and Integrity of Information Disseminated by Federal
           Agencies, which required agencies to issue their own guidelines to
           ensure the quality of information being disseminated. Furthermore,
           in September 2003, OMB issued a report to Congress on the costs
           and benefits of federal regulation. As part of this effort, OMB
           sought public comment on the practice of risk assessment, many of
           which were directed at, and passed along to, EPA. OMB also issued
           the Final Information Quality Bulletin for Peer Review in December
           2004 that provided guidance to federal agencies on what
           information is subject to peer review and defined a planning
           process for peer review. Most recently, on January 9, 2006, OMB
           released a draft bulletin on procedures agencies should use to
           conduct risk assessments and information that should be included.
           Public comment on this draft will be accepted until June 15, 2006.

           Independent Reviewers Have Recommended Improvements to EPAï¿½s Risk
			  Assessment Process 
			  
           Since 1994Independent reviewers, including the National Academy of Sciences
           (NAS) and the Presidential/Congressional Commission on Risk
           Assessment and Risk Management have made a number of significant
           recommendations to improve EPA's human health risk assessment
           process since 1994. These recommendations cover a range of
           actions, including improving planning of what will be required to
           complete a particular risk assessment and what key stakeholders
           need to be involved, supporting development of new scientific data
           on the potential adverse effects from exposure to the contaminant
           under review, and enhancing analysis of the potential risk and its
           uncertainties. All of them can be summarized and discussed in
           terms of the seven themes-implementation, data needs, default
           options, method and model evaluation, uncertainty, variability,
           and exposure to multiple contaminants and routes of
           exposure-reported in NAS's report Science and Judgment in Risk
           Assessment.7

           Improve Planning and Review Process and Ensure an Appropriately
           Skilled Workforce to Conduct Risk Assessment

			  Independent reviewers made a number of recommendations to EPA to
           improve the overall implementation of its risk assessment process.
           The reviewers recommended that EPA improve its planning of risk
           assessments before beginning the process and adopt an iterative
           risk assessment approach. Reviewers also stressed that EPA should
           increase its use of risk assessment reviews by independent
           scientists, known as peer reviews; increase stakeholder
           involvement in the risk assessment process; and ensure it has the
           workforce needed to conduct quality risk assessments. Finally, the
           reviewers recommended that EPA improve the way it characterizes
           the risk it finds and strengthen the link between risk assessment
           and risk management.

           Improve Planning and Adopt an Iterative Risk Assessment Approach

           In 1996, NAS reviewers stressed the importance of planning from
           the beginning of a risk assessment to incorporate the perspectives
           and knowledge of interested and affected parties.8 Because of the
           large number of stakeholders interested in the results of a risk
           assessment, it is important to ensure that the risk assessment
           addresses all of the stakeholders' needs. NAS pointed out that one
           way to do this is to be aware of the stakeholders' concerns from
           the outset and incorporate them into the analysis and
           characterization of risk. By involving stakeholders early, risk
           assessors can ensure that they ask the right questions, make
           appropriate assumptions, and determine the best way to summarize
           information, thereby potentially making the resulting message more
           credible to these parties.

           In addition to improved planning, NAS suggested in 1994, and the
           Commission concurred, that EPA develop the ability to conduct risk
           assessments iteratively-that is, begin with a screening analysis
           to ascertain the potential risk and refine that analysis as
           appropriate. Such an approach would start with relatively
           inexpensive screening techniques and move to more
           resource-intensive data gathering and modeling as the particular
           situation warranted. To guard against the possibility of
           underestimating risk, these screening techniques should be
           constructed to err on the side of caution where there is
           uncertainty.

           Increase Peer Review and Stakeholder Involvement in the
           Risk Assessment Process

			  Each of the NAS reports we reviewed, as well as the report by the
           Commission, recommended ways EPA could improve its peer review
           process and better involve stakeholders in risk assessment. These
           independent reviewers said peer review is critical to evaluate the
           accuracy and appropriateness of technical information, scientific
           observations, and interpretations used in regulatory decision
           making. The Commission observed that peer review can also help in
           the planning stages of a risk assessment to put a problem in
           context and draw on the knowledge of experienced researchers,
           public health officials, and scientists.

           In particular, NAS recommended in 2000 that EPA change its peer
           review policy to ensure greater independence of these reviews from
           the control, or appearance of control, of the program managers.9
           EPA's Science Advisory Board had expressed concern about potential
           conflicts of interest because agency policy allowed the same
           individual to serve as both project manager for a particular work
           product and peer review leader.10 The NAS committee concurred with
           this concern.

           In addition, NAS's 1994 report recommended that EPA collaborate
           more with outside parties to improve the overall risk assessment
           process. For example, EPA could use external advisory groups, such
           as its Science Advisory Board, to help ensure that risk assessment
           decisions use the best science and incorporate full public
           discussion and participation from the scientific community. Such
           collaboration could also include a process for public review and
           comment. The Commission also said that agencies should use
           advisory groups, composed of stakeholders, to periodically
           evaluate the use of technical information. In addition, the
           Commission recommended that agencies establish clear, written
           guidelines for peer review and match the use of peer review to the
           importance of the decision to be made.

           In 1996, NAS commented on the importance of stakeholder
           involvement in the risk assessment process. NAS suggested that the
           risk assessment process involve the spectrum of interested and
           affected parties, including decision makers and specialists in
           risk analysis, at each step of the process. According to NAS, such
           involvement will help ensure that as much important, relevant
           knowledge as possible enters the process and that the process
           remains open and inclusive. NAS also mentioned another benefit of
           involving these stakeholders in the process: the participation of
           a spectrum of people helps ensure that the process will be framed
           correctly and the resulting risk characterization will be
           accurate, balanced, and informative.

           Ensure EPA Has the Skilled Workforce Needed to Conduct
           Quality Risk Assessments
			  
			  Each of the NAS reports we reviewed contained recommendations
           stressing the importance of an organization's ability to ensure
           that it has the expertise and leadership needed to conduct risk
           analyses. In addition, according to current workforce models,
           agencies need to ensure that they have effective training and the
           capability to identify what skills and competencies the employees
           and the organization need. In 1994, NAS recommended that EPA
           acquire additional expertise, as needed, to better interpret data
           and reach sound conclusions about the risks to human health from a
           contaminant. For example, many of the conclusions regarding the
           potential risks from a particular contaminant rely upon risk
           assessment models based on animal data or, when available, human
           evidence from epidemiological studies. The quality of the risk
           assessment will depend upon how well these data are interpreted to
           predict health effects in human populations. NAS recommended that
           EPA acquire staff with specialized skills in fields such as
           toxicology to successfully complete this type of analysis.

           NAS also recommended in 2000 that EPA continue to place high
           priority on ORD's graduate fellowship and postdoctoral program.
           The report stated that to achieve scientific and technical
           excellence, EPA must first attract, retain, and properly support a
           dedicated professional staff. NAS also noted that while EPA has
           many outstanding scientists and engineers, ORD's workforce is
           aging and many staff may retire in the relatively near future.
           Periodic hiring freezes, combined with high competition from the
           private sector, had made it difficult for ORD to recruit the new
           talent needed to sustain and enhance the research workforce. NAS
           concluded that ORD's graduate fellowships and postdoctoral
           programs brought a stream of fresh scientific and technical talent
           into EPA's research program and helped train future researchers in
           environmental science, engineering, and other disciplines. NAS
           urged EPA to continue to place a high priority on these programs.

           In 2000, NAS made a series of recommendations related to
           scientific leadership and talent at EPA. Specifically, NAS
           recommended that EPA establish a new position-deputy administrator
           for science and technology-to address concerns that science at EPA
           was not perceived to be strong and that many EPA scientists
           believed their scientific knowledge and resources were not
           effectively used. Traditionally, EPA's most senior science
           official had been the assistant administrator for research and
           development, but that official lacked agencywide responsibility or
           authority to oversee the scientific and technical basis for
           regulatory and policy decision making. In addition, EPA's program
           offices were not required to follow scientific advice from ORD.
           Consequently, the NAS panel concluded that EPA needed an
           appropriately qualified science official at a sufficiently high
           level with both the authority and responsibility for agencywide
           scientific performance.

           Improve Characterization of Risk and More Closely Link Risk
			  Assessment with the Needs of Risk Managers

           Finally, reviewers recommended ways EPA could improve its
           characterization of risk and better link its risk assessment and
           risk management processes to harmonize the scientific aspects of
           risk assessment and the broader policy objectives of risk
           management. NAS's 1996 report, which primarily focused on risk
           characterization and the role it plays in risk assessment,
           stressed that successful risk characterization can result in
           better and more widely accepted risk assessment and risk
           management decisions. The Commission echoed NAS's position and
           recommended that risk characterizations should include information
           useful to everyone in the risk management process, such as
           information on who is at risk, how they might be affected, what
           the severity of an adverse effect might be, and how confident the
           risk assessors are about their predictions.

           Reviewers also suggested that EPA better link its risk assessment
           and risk management processes. EPA had tried to separate these
           processes entirely to avoid the possible perception that EPA made
           scientific judgments on the risk posed by a contaminant on the
           basis of its willingness to regulate the substance rather than on
           the science. However, NAS pointed out that while some degree of
           judgment is necessary, the science-policy judgments EPA makes in
           the course of risk assessment should more clearly be informed by
           the agency's risk management priorities and goals. According to
           NAS, better linkage between risk assessment and risk management
           will help ensure that the risk assessments will be more accurate
           and relevant to risk managers. The Commission concurred, stating
           that risk assessment should be guided by an understanding of the
           issues important to risk managers' decisions about how to protect
           public health and the environment.

           Review Existing Data, Identify Data Gaps, Prioritize Research Needs,
           Foster Development of New Data, and Improve Strategic Research
           Planning and Collaboration
			  
			  In 1994, NAS found, in general, that EPA did not have the full
           range of data needed to conduct adequate risk assessments to
           estimate the health risks associated with most contaminants-for
           example, data related to the effect a contaminant has on the body
           and the extent of exposure to the contaminant. These data are
           essential to ensure the accuracy and precision of the risk
           assessment. In addition, NAS recommended that EPA increase its
           collaboration with researchers and improve the transparency of its
           research priorities.

           To help address EPA's dearth of data, NAS recommended, among other
           things, that EPA review its existing databases, such as the
           Integrated Risk Information System (IRIS), and identify data
           gaps.11 As part of its database review, NAS recommended that EPA
           compile an inventory of the chemical, toxicological, and
           epidemiological literature in the databases to determine what gaps
           exist. NAS further suggested that EPA prioritize its future
           research on the basis of the significance of the missing data to
           risk assessments, convey its data needs to other public and
           private researchers, such as the Agency for Toxic Substances and
           Disease Registry,12 and develop incentives to expedite the
           generation of needed data by these entities.

           Furthermore, in 2000, NAS recommended, among other things, that
           EPA expand the multiyear research planning by its Office of
           Research and Development (ORD). For example, ORD had developed
           plans for research on topics such as particulate matter, endocrine
           disruptors,13 and drinking water. The NAS committee pointed out
           that these planning efforts will contribute to research program
           continuity and the achievement of strategic goals. In addition,
           NAS recommended that the plans be both continued for
           problem-driven research-research targeted at a particular
           environmental problem-and expanded to include core
           research-broader, more generic research that will help improve
           current and future understanding of a scientific issue. Research
           should be balanced between these two types, since core research
           will help ORD better understand and anticipate environmental risks
           and will enable ORD to provide better problem-specific research
           and technical assistance to the agency.

           NAS has also made recommendations to EPA in the area of research
           accountability. Specifically, NAS recommended that EPA improve the
           documentation and transparency of the decision-making process ORD
           uses to set research and technical assistance priorities and
           allocate funds. For example, NAS pointed to an EPA Science
           Advisory Board finding that ORD's lack of transparency in its
           process for setting research priorities made it difficult to
           evaluate the adequacy of proposed budgets. NAS also recommended
           that EPA expand on its efforts to create an inventory of science
           projects and programs across EPA by documenting and publishing a
           comprehensive and detailed inventory. The inventory should include
           information such as goals and objectives of each project,
           milestones, schedules, and staff allocations and should be used to
           ensure that science activities are properly coordinated through
           the agencywide science planning and budgeting process and are
           appropriately peer reviewed.

           Once the research strategy has been established, research
           collaborations can help execute it. NAS suggested that EPA should
           recognize the limits of its own research capabilities and develop
           a strategy to obtain outside support in research areas where ORD
           is not pre-eminent. Specifically, NAS recommended EPA enhance its
           research collaborations by developing and implementing a strategy
           to stimulate, acquire, and apply the results of research conducted
           or sponsored by other federal agencies, state agencies,
           universities, and industry in this country and abroad. NAS said
           such collaboration is especially important given the budget
           constraints EPA and the federal government face and the breadth of
           knowledge required to conduct the full range of risk assessments.
           In addition, NAS recommended that EPA develop additional
           mechanisms to promote and facilitate research interactions among
           grantees in EPA's Science to Achieve Results (STAR) grant program
           and ORD research staff.14 For example, NAS suggested that grant
           applicants could identify in their proposals how their research
           might complement or supplement ongoing or planned research in the
           ORD laboratories. NAS also recommended that EPA increase its
           efforts to disseminate ORD's research products, explain their
           significance, and assist others inside and outside the agency in
           applying the research. NAS noted that while EPA's policy and
           regulatory work receives a great deal of public attention, the
           agency's science research typically receives a similar degree of
           attention only when the scientific basis for a decision is
           questioned. They also pointed out that even internal EPA offices
           may be unaware of important benefits from ORD's research program.

           More Clearly Indicate Use of Default Options and How They Were Chosen
			  
			  When EPA needs to rely on assumptions-known as default
           options-because it lacks data to fully assess the potential health
           risks from exposure to a contaminant, reviewers said EPA should,
           among other things, more clearly indicate that it relied on these
           defaults and how they were chosen. EPA's defaults, which are based
           on general scientific knowledge and policy judgment, are, for the
           most part, conservative.15 One example of a commonly used default
           option is that laboratory animals are a surrogate for humans in
           assessing health risks: An adverse effect in the animals is taken
           as evidence of a chemical's potential to harm humans.

           Independent reviewers agreed with EPA's use of default options as
           a reasonable way to cope with the lack of available data. However,
           NAS recommended that EPA be clearer about the scientific and
           policy basis for each default option and when it uses a default.
           Furthermore, reviewers said that EPA should fully explain under
           what conditions they would depart from these default options.
           According to NAS, giving greater formality to the criteria for a
           departure would result in greater guidance for the public and
           lessen the possibility of undocumented departures that would
           undercut the scientific credibility of a risk assessment. The
           Commission concurred, stressing that the defaults used in risk
           assessments and the uncertainty associated with their results
           should be clearly identified and justified.

           Better Evaluate Available Methods and Models and Incorporate More
           Advanced Tools When Appropriate
			  
			  The overall accuracy of a risk assessment largely depends on the
           validity of the various analytic methods and models EPA uses to
           assess the toxicity of, and exposure to, a particular contaminant.
           However, according to NAS, EPA often does not clearly understand
           the extent to which the methods or models it chooses accurately
           predict the toxicity or exposure to a contaminant. For example,
           because of limited scientific information on how contaminants
           actually work in the body to produce adverse effects in humans,
           EPA frequently uses a method that relies on studies involving
           laboratory animals to understand the toxicity of a substance. The
           concentrations of the contaminant introduced into the animals as
           part of these studies are higher and administered for shorter
           periods of time than humans would normally experience.
           Consequently, to determine the expected response in people, EPA
           extrapolates the response from laboratory animals to humans. While
           such extrapolations are useful to predict potential harmful
           effects of a contaminant, different analytic methods may better
           predict the effect.

           In 1994, NAS recommended, among other things, that EPA evaluate
           the accuracy of its methods and models for assessing toxicity and,
           when appropriate, incorporate more advanced tools. For example,
           models that are based on the underlying mechanisms at the cellular
           or molecular level can more accurately estimate the dose of the
           contaminant that would have an adverse biological effect on a
           specific part of the body. Regarding the risk of cancer from a
           particular contaminant, NAS recommended that EPA continue to both
           use methods involving animal studies to evaluate the possibility
           of adverse effects in humans and explore, when appropriate,
           mechanistic models. Furthermore, to better convey the cancer risk
           associated with a particular substance, NAS recommended that EPA
           develop a classification scheme that provides narrative statements
           regarding the hazards posed by carcinogens and a descriptive
           evaluation of the strength and nature of the evidence used to
           estimate the substance's potential for causing cancer.

           Regarding exposure, NAS noted that EPA had traditionally
           characterized exposure according to two criteria: exposure of the
           total population and exposure of a specified highly exposed
           subpopulation. While these two criteria can help assess whether
           any particular exposure might occur above a regulatory threshold,
           only considering the highly exposed subpopulation is likely to
           overestimate the exposures of most of the population.
           Consequently, NAS recommended, and the Commission concurred, that
           EPA consider the entire range of a population's exposure, rather
           than just the exposures of a highly exposed subpopulation. The
           Commission also recommended that EPA identify and evaluate highly
           exposed populations separately. Moreover, NAS recommended that EPA
           use population-activity models-models based on actual human
           activity patterns-to better estimate an individual's true exposure
           and expand efforts to use personal monitoring data to better
           understand actual exposures and variances across the population.

           More Explicitly Analyze and Characterize the Sources of Uncertainty in
           Risk Assessments
			  
			  To address the inevitable uncertainties associated with gaps in
           scientific knowledge and general unknowns about model and data
           accuracy, reviewers recommended that EPA more explicitly analyze
           and characterize the sources of uncertainty in its risk
           assessments. Numerous gaps in scientific knowledge exist regarding
           the health effects of various contaminants, such as the exact
           amount of exposure to a particular contaminant that can cause an
           adverse effect or the biological effect of a contaminant on the
           body. In addition, knowledge is often lacking about which model or
           method might be most appropriate to estimate risks to human health
           from a particular contaminant. NAS recommended that EPA develop
           guidelines for how to analyze and report the different types of
           uncertainty, both for the overall assessment and for the different
           stages of the risk assessment, such as hazard identification and
           exposure assessment. For example, during hazard identification,
           uncertainty can be related to the quality, type, and results of
           scientific studies; however, during exposure assessment,
           uncertainty can be related to model choice or available data.

           NAS also recommended in 1994 that EPA conduct uncertainty analysis
           of the risk estimate and present the identified uncertainties as
           explicitly, accurately, and fully as feasible. Analysis of the
           effects of uncertainties can help inform EPA decision makers and
           the public about the extent of uncertainty associated with the
           risk assessment. The analysis can also show where additional
           research might resolve major uncertainties and where it might not.
           NAS recommended that the uncertainty analysis be presented both
           descriptively (qualitatively) and, where possible, numerically
           (quantitatively). For example, some sources of uncertainty, such
           as those related to estimating exposures, can be reduced through
           the use of more advanced statistical methods. Other types of
           uncertainty, such as those associated with extrapolating data from
           animal testing to predict the effect on humans, are more difficult
           to quantify. The Commission concurred and recommended that risk
           characterizations include narrative descriptions of the primary
           reasons for uncertainty, as this information is likely to be more
           understandable and useful than quantitative estimates or model
           results.

           Enhance Analysis of Variability in Exposure Levels and Health Risks to
           Exposed Individuals
			  
			  Variability, which refers to the natural diversity in a
           population, can be better understood or described, but not
           reduced. To address the two main categories of variability-one
           related to differences in levels of individual exposures to
           contaminants and the other related to differences in individual
           reactions to exposure-reviewers recommended that EPA enhance its
           analysis of both types and carefully state in each risk assessment
           what assumptions it made about what is and is not accounted for.
           Specifically, variability related to different exposures depends
           on the various concentrations of a contaminant as it disperses in
           the environment, different breathing rates, and different food
           consumption and personal activity patterns. For example, infants
           and children generally consume more fruits, vegetables, and fruit
           juices per body weight than adults, and some people, such as
           agricultural workers, are more exposed to pesticides through
           breathing and skin contact. The Commission recommended that risk
           assessments identify groups of people who are likely to have
           higher exposures to contaminants and consult these groups in the
           early stages of an assessment to obtain information about all
           known sources of exposure.

           Reviewers also recommended that EPA revise the way it estimates
           how long a person is exposed to a contaminant. NAS and the
           Commission recommended that EPA move away from estimates of
           exposure based on a hypothetical "maximally exposed individual,"
           who was assumed to be the person at greatest risk in a worst-case
           scenario, because these estimates do not account for a number of
           other factors that may affect exposure patterns and rates, such as
           the time the person spends indoors or going to work. Furthermore,
           estimates based on the hypothetical maximally exposed individual
           likely overestimate the exposures of most of the population and
           underestimate the exposures of subpopulations, such as
           agricultural workers, who may be more highly exposed than the
           general population. While EPA's 1992 exposure assessment
           guidelines suggest the use of ranges and high-end exposure
           estimates chosen from the high end of those ranges, according to
           NAS, EPA had not sufficiently documented the reliability of such
           estimates when data are limited.

           The second type of variability-differences in human
           susceptibility-is related to inherent differences among people,
           such as age, physiologic characteristics, lifestyle, genetics,
           sex, and ethnicity. Reviewers found that EPA's approach for
           reducing risks associated with chemical exposures generally did
           not include information on differences in individual
           susceptibility or encourage gathering evidence to identify these
           differences. The reviewers recommended, among other things, that
           EPA consider this "interindividual" variability and adopt a
           default option for differences in susceptibility among humans. In
           addition, NAS recommended that EPA assess risks to infants and
           children whenever it appears that their risks might be greater
           than those of adults. For example, the developing brains of
           infants and young children have an increased susceptibility to
           contaminants that harm the nervous system, such as lead. NAS
           specifically recommended that EPA sponsor research to examine the
           causes and extent of interindividual variability in susceptibility
           to cancer and the possible connection between susceptibility and
           age, race, ethnicity, and sex. The Commission generally concurred,
           adding that risk assessments should also identify especially
           susceptible subpopulations, such as people with asthma who may
           have an increased responsiveness to allergens and respiratory
           irritants. The Commission also stated that, where possible,
           available information about the range of the population's
           susceptibility should be considered and used in place of default
           assumptions.

           Better Consider the Human Health Effects of Exposure to Multiple
           Contaminants and Routes of Exposure
			  
			  Reviewers recommended that EPA improve how it considers the
           effects of combinations of contaminants (cumulative exposure) and
           all possible paths of exposure to a single contaminant (aggregate
           exposure). People are typically exposed to a mixture of
           contaminants through a variety of pathways, such as contact with
           skin or eating food, each of which might be associated with an
           increased probability of one or more health effects. However, most
           risk assessments address a single contaminant and often focus on a
           single pathway of exposure, such as inhalation. As a result, NAS
           recommended that EPA should consider all possible exposure
           pathways. For example, when assessing risk from mercury, EPA
           should consider the risk to residents from inhaling mercury
           emitted from a nearby industrial smoke stack, as well as the
           possibly greater health risk of consuming mercury that has
           accumulated in the tissue of fish that are caught and eaten
           locally after mercury from the smoke stack was deposited into
           water.16

           When assessing the risks of exposure to chemical mixtures, most
           risk assessments estimate the risks from individual contaminants,
           then calculate the combined risk by simple addition. However, this
           method ignores potentially synergistic interactions that may make
           the effects more damaging to human health than anticipated or
           antagonistic interactions that may make the effects less damaging
           than anticipated. Consequently, this method could either under- or
           overestimate the total risk. NAS said that simple addition of the
           risks from multiple contaminants may be appropriate for
           screening-level risk estimates. However if a more refined
           quantitative estimate is needed, EPA should consider using
           statistical procedures to combine the risks from exposures to
           multiple contaminants, which would help produce a more
           comprehensive estimate of risk. The Commission concurred but
           stressed that combining risks may not always be feasible; the risk
           analyses for exposure to each contaminant may not be compatible
           because the risk assessments may differ in accuracy. Further, the
           Commission recommended that for risk assessments involving
           exposures to low concentrations of multiple chemicals, the risks
           from each exposure should be added in the absence of information
           on exactly how the chemicals affect the body. However, if the
           multiple chemicals affect the body in different ways-for example,
           if one chemical affected development while another affected the
           nervous system-the impact of each chemical on the body should be
           considered independently and not added together.

           EPA Has Strengthened Many Facets of Its Risk Assessment Process
           Since 1994, and Efforts Are Ongoing

           EPA has modified its human health risk assessment process since
           1994 in several ways. First, the agency enhanced implementation of
           its risk assessment process by, for example, issuing guidance and
           realigning staff resources. In addition, EPA has taken steps to
           identify the scientific data it has on the potential adverse
           effects from exposure to various contaminants and has established
           collaborative relationships with external researchers to foster
           the development of needed additional data. Furthermore, EPA has
           begun to improve its use of default options, enhance its modeling
           capabilities, and explore new methodologies. EPA has also begun to
           characterize uncertainty quantitatively and analyze and
           communicate variability more thoroughly. Finally, EPA is more
           often considering the combined effects of exposure to multiple
           contaminants through multiple pathways. Most of these efforts are
           ongoing and can be discussed in terms of the themes presented in
           the previous section.

           EPA Has Enhanced Implementation of Key Aspects of Its Risk
           Assessment Process
			  
			  EPA has taken a number of steps to improve implementation of its
           risk assessment process. Specifically, EPA has developed guidance
           and policy documents at the agency and program office levels;
           built scientific capacity; modified components of its approach to
           risk assessment; and refined its peer review and quality-assurance
           practices.

           EPA Has Issued Many Agencywide and Program Office-Specific Guidance and
           Policy Documents to Improve Risk Assessment Practices
			  
			  The guidance and policy documents EPA has issued over the past
           decade were intended to help staff develop and use risk
           assessments and to provide basic information to the public about
           EPA's risk assessment methods. While some of these documents have
           remained unchanged over time, many have been revised, or will be
           revised, as science, knowledge, and analytic methods have
           improved. Among these documents are the guidelines issued by EPA's
           Risk Assessment Forum, a committee of senior EPA scientists
           established to promote agencywide consensus on risk assessment
           issues, which cover such topics as neurotoxicity,17 exposure
           assessment, and carcinogenic risk assessment. Each of the original
           five guideline documents created in 1986 has been updated at least
           once, and some are slated to be revised again. For example, the
           exposure guidelines were revised in 1992 and revisions are
           currently being planned. In addition, the forum has issued two
           entirely new sets of guidelines since 1994-Guidelines for
           Reproductive Toxicity Risk Assessment and Guidelines for
           Neurotoxicity Risk Assessment-and has developed a number of
           policies on, among other things, risk characterization, peer
           review, and evaluating risk to children. EPA has also issued
           interim policy memorandums and position papers on scientific
           issues such as genomics and endocrine disruption.

           Similarly, much of EPA's agencywide guidance issued since 1994 has
           undergone revision and has its origins in earlier policy
           documents. For example, EPA's Risk Characterization Handbook,
           issued in 2000, has its roots in the 1995 Policy for Risk
           Characterization. The handbook stresses that risk characterization
           should be transparent, clear, consistent, and reasonable. EPA's
           policy on the use of peer review was originally issued in 1994 and
           was followed up by issuance of peer review handbooks in 1998 and
           again in 2000.18 The peer review policy was recently updated in
           January 2006. EPA has also issued technical guidance, such as its
           review of the processes to estimate a daily or continuous exposure
           to humans that is likely to be without appreciable adverse effects
           during a lifetime.

           In addition to the EPA-wide guidance, many of the program offices
           have also issued guidance documents that support their particular
           risk assessment efforts. In general, the office-specific guidance
           documents provide risk assessors with analytic tools and exposure
           scenarios pertinent to the statutory responsibilities of the
           office. For example, the Office of Pollution Prevention and Toxics
           created guidance in the form of an analytic tool to screen
           chemicals in the absence of data, which frequently occurs because
           of the lack of a requirement for industry to develop extensive
           data on new chemicals or new uses of existing chemicals. In
           addition, the Office of Pesticide Programs has issued
           pesticide-specific guidance documents. For example, to help ensure
           consistency in pesticide chemical risk assessments, the Office of
           Pesticide Programs issued guidance for developing residential
           exposure assessments and developed a template for making and
           documenting registration eligibility decisions.

           EPA Has Built Scientific Capacity through Increased Focus on Scientific 
			  Leadership, Greater Reliance on Research Advisory Groups, and
           Development of Future Scientific Talent
			  
			  EPA enhanced its scientific leadership through the creation of the
           position of science advisor in 2002 and the Office of the Science
           Advisor in 2003, the increased reliance on research advisory
           groups composed of senior EPA scientists and external experts, and
           the continuation of its research fellowship programs. The
           overarching responsibility of the science advisor is to coordinate
           and oversee the scientific activities of the program offices at
           EPA to ensure the best use of science. The Office of the Science
           Advisor provides further leadership by establishing specific
           mechanisms to ensure that scientific results, combined with
           technical evaluation and peer review, play a prominent role in
           regulatory decisions and that EPA staff interpret and enforce
           regulations consistent with the science supporting them.

           In addition to enhancing its scientific leadership, EPA has also
           increased its reliance on research advisory groups since 1994. The
           Science Policy Council and the Risk Assessment Forum play key
           roles in advancing the practice of risk assessment at EPA. The
           council reviews the adequacy of existing policies, establishes
           science policy as needed, and coordinates EPA efforts related to
           methods, modeling, risk assessment, and environmental technology.
           The Science Policy Council staff facilitate ad hoc work groups,
           encourage communication and consensus building within the agency,
           and participate in technical work-group activities and
           deliberations.

           The Risk Assessment Forum is a standing committee of senior EPA
           scientists established to promote agencywide consensus on
           difficult and controversial risk assessment issues and to ensure
           that this consensus is incorporated into guidance. According to an
           agency official, the forum is designed as a venue where staff can
           meet and discuss common risk assessment issues across program
           offices. One of the forum's main contributions to risk assessment
           at EPA has been the issuance of a series of risk assessment
           guidelines. The forum is currently working on new guidelines, such
           as one related to adverse effects on the immune system. When more
           specificity is needed on an existing guideline, the forum issues
           companion pieces, known as "purple books" because of the color of
           their cover, that provide additional or updated information.

           The Board of Scientific Counselors (BOSC) provides objective and
           independent advice, information, and recommendations about ORD's
           research program to ORD's assistant administrator. BOSC is
           composed of scientists and engineers from academia, industry, and
           environmental organizations who are recognized as experts in their
           fields. In 1998, BOSC completed a peer review of ORD's
           laboratories and centers.19 BOSC completed a second review of the
           laboratories and centers in 2002 and 2003 that identified key
           accomplishments of the laboratories and centers, as well as areas
           for future improvement. In addition, after EPA's Office of the
           Science Advisor issued its 2004 staff paper,20 it asked BOSC to
           host a workshop for EPA staff and other interested stakeholders,
           such as industry, environmental groups, and researchers, to
           provide feedback to refine EPA's current practices and to suggest
           alternative approaches for specific aspects of risk assessment.

           EPA has also worked to foster scientific excellence and enhance
           the skills of its existing workforce through its graduate and
           postdoctoral fellowship programs. One such program, the EPA/ORD
           Postdoctoral Fellowship Program, began in 1998, and, as of May
           2003, 205 individuals had participated in the program. One benefit
           of the program is that it helps provide a ready pool of talented
           candidates for EPA vacancies. In fact, according to a study of the
           program conducted by the National Council for Science and the
           Environment, nearly half of the former postdoctoral participants
           had taken permanent positions at EPA. The Science to Achieve
           Results (STAR) fellowship program is designed to encourage masters
           and doctoral students to pursue careers in an environmental field.
           The STAR fellowship program has provided new environmental
           research in the biological and health sciences, two fields related
           to the development of human health risk assessment. EPA has also
           partnered with the Association of Schools of Public Health (ASPH)
           to offer 1-year placements of graduates in EPA laboratories,
           centers, and program offices to work on public health issues. In
           the announcement of opportunities for the 2006 ASPH Fellows
           Program, a dozen fellowships are being offered in areas related to
           human health risk assessment. For example, the National Center for
           Environmental Assessment is offering a fellowship to develop
           health assessments for various chemicals, and the National Center
           for Environmental Research is offering a position for someone
           interested in working on developing models or analyzing
           uncertainty in risk assessments.

           EPA Has Begun to Incorporate Planning Activities and Stakeholder
			  Input into Its Risk Assessment Approach
			  
			  To improve planning, which is a part of all risk assessments to
           some degree, EPA has issued various guidance documents and held
           workshops for staff. One of EPA's earliest related guidance
           documents,21 released in 1997, was designed to help risk assessors
           and risk managers plan and document the scope of risk assessments
           and to consider input from appropriate stakeholders and experts,
           especially in those assessments involving the effects of
           combinations of contaminants.22 EPA followed up this guidance with
           workshops to help staff apply it in risk assessments. In January
           2002, EPA issued a handbook to reflect some of the lessons learned
           from implementation of the 1997 guidance to make risk assessments
           more useful to decision makers and other stakeholders. These
           lessons, conveyed through case studies, include that planning can
           be particularly valuable when the assessment is complex,
           controversial, or precedent setting, and that explaining
           uncertainty to stakeholders can help develop trust, credibility,
           and support for the decision-making process.

           The 2000 Risk Characterization Handbook also strongly advocates
           the use of planning and presents a number of topics for both risk
           managers and risk assessors to consider, such as identifying the
           stakeholders in the process; scope of the effort; relevant
           management goals, issues, and policies; available data; and data
           needs. For example, during planning, risk assessors, risk
           managers, and stakeholders need to identify the key data gaps and
           discuss how best to fill them, such as whether to use existing
           data or conduct additional short- or long-term tests to evaluate
           exposure and effects. The Office of Air and Radiation recognized
           the need for planning and developed planning guidance as part of
           its Air Toxics Risk Assessment Reference Library, issued in 2004.
           EPA acknowledged in its 2004 staff paper that it needs to continue
           to stress the importance of concerted and conscious planning with
           risk assessors and risk managers before a risk assessment is
           started. According to EPA, risk assessors need to outline early in
           the development of a risk assessment what will and will not be
           addressed and how they will develop the risk assessment.

           Stakeholders and the public play a key role in the planning, as
           well as at later stages in the development of a risk assessment.
           Stakeholders, at various levels and in various forms, can help
           ensure better understanding of the risk assessment results and may
           promote support for the selected risk reduction strategies.
           Program offices involve stakeholders in various ways. For example,
           the branch of the Office of Air Quality Planning and Standards
           (OAQPS) responsible for setting certain air quality standards for
           six principal pollutants solicits input from stakeholders in the
           planning phase of its periodic updates to the standards it sets.23
           In addition, the public may officially comment on draft air
           quality standards once they are publicly released. The Office of
           Water pursues stakeholder and public involvement that includes
           working with the environmental community, industry, trade
           associations, risk assessor organizations, states, and bordering
           countries. In addition, the office's periodic reviews of water
           quality standards and other nonregulatory actions, such as health
           advisories, are all open processes that allow for public input on
           various stages of the analysis.

           For risk assessments involving the reregistration of pesticides,
           the Office of Pesticide Programs (OPP) established a process that
           provides several opportunities for public participation.24
           Depending on the potential health risks posed by a pesticide
           product, the public has anywhere from one to four separate
           opportunities to comment. For example, if risk assessors estimate
           that the product poses little risk to human health, the public
           will have one opportunity to comment before OPP decides whether to
           approve the pesticide product. For higher-risk products, the
           public will have as many as four opportunities to comment. The
           first opportunity to comment occurs after OPP has completed a
           preliminary risk assessment. This preliminary assessment contains
           all of the elements of a risk assessment and has undergone
           internal review, but is not yet finalized. Notice of the
           opportunity to comment is distributed to people who have elected
           to sign up for such notifications, as well as through a "notice of
           availability" published in the Federal Register. The public can
           also comment on risk assessments prepared by the Office of
           Pesticide Programs through the office's Science Advisory
           Panel-which holds periodic public meetings on pesticide-related
           risk assessment issues, such as methods to assess skin sensitivity
           to exposure to pesticides or models used to estimate dietary
           exposures.

           EPA has also adopted an iterative approach to many of its risk
           assessments. An iterative approach begins with a screening
           assessment and progressively grows in depth and scope in relation
           to the estimated risks to human health. When a screening
           assessment identifies a potential for a nontrivial risk, EPA
           decides if pursuing that risk is appropriate based on its current
           priorities and available resources. If EPA decides to pursue the
           risk, a more detailed, refined risk assessment is performed. The
           degree of refinement is based on the type of decision, the
           available resources, and the needs of the risk manager. After
           refinement of the estimate, EPA reviews it to see if it will be
           sufficient to answer the questions posed. Refinements proceed
           iteratively until the assessment provides an adequate answer for
           the decision maker within the resources available. Both the
           revised cancer guidelines and EPA's 1995 Policy for Risk
           Characterization support an iterative approach to risk assessment.
           Some program offices have also adopted an iterative-or
           tiered-approach to risk assessment. For example, the air toxics
           program follows a tiered approach, beginning with an analysis that
           includes few data and many conservative assumptions. If this
           analysis indicates that the risk may be relatively high, assessors
           pursue more intensive analysis to determine if the risk is
           realistic or an artifact of the lower tier's conservative
           assumptions. Despite this move toward greater use of an iterative
           approach, EPA acknowledges it could be clearer about when it is
           taking such an approach. For example, EPA could be more
           transparent about when and why it makes a risk management decision
           based on a screening-level assessment rather than a more detailed
           assessment.

           EPA Has Refined Its Peer  Review and Quality-Assurance Practices
			  
			  In the years since the issuance of Science and Judgment in Risk
           Assessment, EPA has made strides to improve its peer review
           practices. EPA uses peer review to help ensure the quality of its
           risk assessments and keep them as objective and consistent as
           possible. EPA's Peer Review Policy states that scientifically and
           technically based work products related to agency decisions should
           be peer reviewed. In 2000, EPA issued its revised Peer Review
           Handbook, an update of the original 1998 edition. In the
           intervening 3 years, EPA's Science Advisory Board, EPA's Office of
           Inspector General, the National Research Council, and GAO
           scrutinized the peer review process. In response, in part, to
           recommendations made by many of these groups, EPA issued the
           current edition of the handbook. Among other things, it instructs
           EPA to balance peer review panels in terms of expertise and biases
           to help ensure a reasonable and scientific review.

           In addition to issuing the Peer Review Handbook, EPA has
           undertaken a number of actions to help ensure the quality of its
           data and information. In May 2000, EPA established an agencywide
           quality-assurance system and issued the EPA Quality Manual for
           Environmental Programs. Key components of this system include
           assigning a quality-assurance manager to conduct independent
           oversight of data quality, developing a management plan, and
           conducting an annual assessment of the quality system. In
           addition, the system calls for an assessment of the data EPA used
           to support agency decisions to verify that they were of sufficient
           quantity and quality for their intended use. In 2002, EPA
           developed its information-quality guidelines in response to those
           issued by OMB, which stated that agencies must ensure the quality,
           objectivity, utility, and integrity of information released to the
           public.25 EPA's guidelines outline its policy and procedures to
           ensure and maximize the quality of the information it disseminates
           and describe the mechanisms by which EPA reviews information prior
           to dissemination. EPA issued a complement to these guidelines in
           2003 to raise awareness among the public about EPA's ongoing
           interest in high-quality data and to serve as an additional
           resource for staff as they evaluate the quality and relevance of
           information.26

           EPA Has Enhanced Its Strategic Planning and Refocused Staff Resources
           to Initiate Review of Existing Data and Development of Needed Data

           Since 1994, EPA has initiated a number of actions to develop
           missing or incomplete scientific data on the potential adverse
           effects from exposure to contaminants, including refocusing and
           expanding the Office of Research and Development human health risk
           assessment program, improving strategic research planning,
           increasing collaboration with outside researchers, and enhancing
           databases.

           EPA Restructured and Augmented Its Research Office

           In 1995, EPA restructured ORD to more effectively generate and
           gather information needed for the risk assessment process,
           combining 12 existing laboratories into 3 national laboratories
           and 2 national centers-the National Health and Environmental
           Effects Research Laboratory (NHEERL), National Exposure Research
           Laboratory (NERL), National Risk Management Research Laboratory
           (NRMRL), National Center for Environmental Research (NCER), and
           National Center for Environmental Assessment (NCEA).27
           Specifically, one laboratory, NHEERL, conducts research on the
           effects of various exposure routes and rates, dose levels and
           duration, and cumulative doses on health outcomes. Its main
           research areas, all of which help improve risk assessments, target
           (1) the use of mechanistic information-how a substance causes an
           effect at a biological level-to reduce uncertainties in risk
           assessment; (2) the cumulative risks posed by exposure to
           contaminant mixtures; (3) the health risks to particularly
           susceptible populations; and (4) the evaluation of public health
           outcomes to determine the effectiveness of actions designed to
           reduce health risks. This research is developed in coordination
           with the program offices to target issues in risk assessment for
           air pollutants, safe foods and pesticides, water, and contaminated
           lands.

           A second laboratory, NERL, researches and develops improved
           methods and models to assess and predict human exposures to
           harmful contaminants in air, water, soil, and food. For example,
           it developed methods to better characterize pollution sources, and
           models to quantify the effects on exposure from various individual
           behaviors. NERL works closely with the program offices to set
           research priorities and help ensure that its results are useful to
           the program offices. For example, NERL coordinated with OAQPS to
           format data from several studies on airborne particulate matter in
           a comparable manner that allowed NERL and OAQPS staff to develop
           more powerful analytic results than would have been possible from
           the individual studies alone. NERL also developed tools to enhance
           the exposure assessment portion of the risk assessment, such as a
           database of human activities by age, sex, and location, to better
           characterize exposure risk based on personal activities. The third
           laboratory, NRMRL, focuses its research more on ways to minimize
           exposure to contaminants that cause health risks than on ways to
           improve the preparation of risk assessments. For example, NRMRL
           researches sources of chemicals that disrupt the endocrine system
           and strategies to minimize exposure to such chemicals.

           The two centers, NCER and NCEA, serve as focal points for external
           researchers and risk assessors, respectively. NCER funds
           innovative environmental research by academic scientists to reduce
           uncertainty in risk assessment. Specifically, NCER has sponsored
           research to develop data for use in models, thereby helping to
           reduce the model's uncertainties. NCER also coordinates with EPA's
           laboratories and program offices to develop its research topics.
           The Science to Achieve Results (STAR) program, one of NCER's grant
           programs, funds competitive research proposals and graduate
           fellowships in environmental science and engineering fields to
           complement ORD's research in its strategic and research plans,
           such as the health effects of contaminants on airborne particulate
           matter and in drinking water, and, more generally, on children's
           health. For example, the STAR program has funded research to
           generate data on human exposures that will improve risk estimates.
           NCER has also established academic research centers in such areas
           as children's health. NCEA, EPA's national resource center for
           human health risk assessment, supports EPA's work in three main
           ways. First, NCEA conducts risk assessments of national
           significance-for example, assessments of dioxin and diesel
           emissions-and prepares the air quality criteria documents that
           reflect the state of the science for six principal air pollutants.
           Second, it develops scientifically sound, defensible risk
           assessment methods to improve the use of science in risk
           assessment, such as software to estimate a benchmark dose-the dose
           that produces change in the risk of an adverse effect. Third, it
           provides guidance and support to risk assessors and risk managers
           through such means as its management of the Integrated Risk
           Information Systems (IRIS), a database of the potential human
           health effects of exposure to various chemicals in the
           environment.

           In addition to the six labs and centers, EPA in 2005 established
           the National Center for Computational Toxicology (NCCT) to
           coordinate and implement EPA's research on computational
           toxicology, a cutting-edge field that uses mathematical models to
           predict adverse effects and to better understand the mechanisms
           through which a given contaminant causes harm. Given advances in
           such newly emerging disciplines as the study of genes and their
           functions, computational toxicology offers the potential for
           scientists to develop a more detailed understanding of the risks
           posed by a much larger number of chemicals than is currently
           possible. NCCT research is designed to develop tools to conduct
           quantitative risk assessments more rapidly and to improve the
           identification of chemicals that may pose substantial health
           risks. NCCT staff have begun to collaborate with other ORD
           laboratories and centers to effectively target their research
           efforts.

           EPA Has Enhanced Strategic Research Planning
			  
			  Since 1994, EPA has undertaken a number of strategic activities to
           more closely link the data needs of program offices to research
           agendas of EPA and ORD.28 In connection with the goals presented
           in EPA's and ORD's strategic plans, ORD defined 16 high-priority
           research areas such as human health, endocrine disruption,
           airborne particulate matter and other air pollutants, and safe
           pesticides. Research strategies for each area are either in place
           or under development. For example, EPA's Human Health Research
           Strategy identifies four broad, overarching research areas to
           guide ORD's human health research over the next 5 to 10 years.29

           To carry out its 16 research strategies, ORD began in 2000 to
           develop a multiyear implementation plan for each. The first plans
           emerged in 2001, with most finalized by 2003 following widespread
           participation and input from many stakeholders as well as review
           by senior ORD managers. ORD invites input on the plans from ORD
           and program office staff, federal research partners, and outside
           peer groups, such as EPA's Science Advisory Board, the Board of
           Scientific Counselors, and the National Research Council. These
           plans establish the short- and long-term goals and timelines
           required for ORD's laboratories or centers to implement each of
           the strategies. The National Program Director, a newly established
           position for each major ORD research area including human health,
           helps ensure that ORD's time and staff resources are used
           strategically and that the overall planning effort links to the
           needs of the program offices.

           Program offices use various planning and review approaches to
           determine data needs. For example, the Office of Air Quality
           Planning and Standards develops a research-needs paper at the
           conclusion of each periodic review of the air quality standards it
           establishes to inform the research agenda for the next review,
           which occurs about every 5 years. Each paper helps ensure that
           research in key areas will be available for the next review. In
           contrast, the data needs of the Office of Pesticide Programs are
           defined by law.30 Applicants who wish to register a pesticide
           product must submit the data defined in the statute and
           regulations, and OPP staff determine whether the data are of
           sufficient quality and quantity to assess the risk from the
           pesticide product.31 This list of required data is currently
           undergoing revision and additional data requirements may be added.

           EPA Program Offices Have Established Collaborative 
           Relationships with External Researchers

           Since 1994, EPA has strengthened and formalized collaboration with
           a range of other federal researchers to better leverage its
           limited research dollars and foster the development of data to
           improve human health risk assessments. Specifically, EPA has
           developed relationships with agencies such as the National
           Institute for Environmental Health Sciences (NIEHS) and the Agency
           for Toxic Substances and Disease Registry (ATSDR). For example, in
           1998, EPA established a cooperative agreement with NIEHS to
           develop a body of research on the relationship between exposures
           and children's health. This collaboration jointly funded
           Children's Environmental Health Research Centers at seven U.S.
           universities and one medical center to research children's asthma
           and other respiratory diseases, as well as ways to reduce farm
           children's exposure to pesticides. In addition, EPA works closely
           with ATSDR to help fill research gaps and develop
           chemical-specific toxicological assessments used in risk
           assessments. In 2004, EPA and ATSDR entered into a formal
           agreement to ensure close collaboration to avoid duplicating
           efforts to fill data gaps. Under the agreement, the two agencies
           formed a work group to coordinate their efforts to develop
           toxicological assessments for ATSDR's work at specific highly
           contaminated locations and for EPA's Integrated Risk Information
           System (IRIS) database.

           EPA, NIEHS, and ATSDR also jointly develop and annually review a
           list of approximately 275 hazardous substances commonly found at
           the nation's highly contaminated sites and for which ATSDR will
           prepare toxicological assessments. At each annual review, agency
           staff may add chemicals to the list and identify priority research
           to fill gaps in knowledge. Of these 275 chemicals, approximately
           150 have been identified by EPA as high-priority needs. Each
           toxicological assessment contains almost everything that is known
           about the chemical, including its potential to harm human health
           or the environment. A key difference between these toxicological
           assessments and the ones in EPA's IRIS database is that ATSDR
           includes chronic cancer and noncancer effects, as well as acute
           effects, while IRIS generally includes only chronic cancer and
           noncancer effects.

           In addition, EPA has begun to establish collaborative
           relationships with scientific and industry-related researchers.
           For example, EPA has cooperative agreements with the International
           Life Sciences Institute's Risk Science Institute (ILSI-RSI), an
           organization that researches critical scientific issues in risk
           assessment, such as the development of risk assessment
           methodologies.32 These cooperative agreements were specifically
           designed to engage the scientific community and bring together
           scientists from different affiliations (including academia, other
           parts of government, and the private sector including industry) to
           address risk assessment issues. Under one agreement, ILSI-RSI is
           to research risk assessment approaches for cumulative and
           aggregate exposures. In addition, EPA has used research provided
           by CIIT Centers for Health Research, a chemical research
           laboratory funded by EPA, industry, and other federal agencies, to
           provide information for its formaldehyde IRIS assessment.
           Furthermore, EPA and industry jointly fund the Health Effects
           Institute (HEI)-an organization that researches the health effects
           of various air pollutants, including airborne particulate matter
           and ozone. HEI has provided data for risk assessments and convened
           panels of experts to review and issue reports related to risk
           assessment, recently on diesel exhaust. In its 2004 staff paper,
           EPA noted that it needs to continue to encourage development of
           the specific data necessary to more accurately assess potential
           risks, particularly with researchers responsible for generating
           appropriate data, such as those seeking approval to manufacture or
           use a chemical.

           The Office of Pollution Prevention and Toxics has two programs to
           work with industry to develop data on contaminants that can be
           used to better understand risks. The first is the High Production
           Volume (HPV) Challenge Program. This program was officially
           launched in late 1998 to ensure that a baseline set of data would
           be made available to the public on approximately 2,800 chemicals
           that are manufactured or imported in amounts greater than 1
           million pounds per year. Diverse stakeholders, including the
           American Chemistry Council, Environmental Defense, and the
           American Petroleum Institute participate in the program. The HPV
           Challenge Program provides an opportunity for all stakeholders,
           including the public, to comment on the tests and data summaries
           from the chemical sponsors-companies and consortia that
           volunteered to make publicly available screening-level data that
           allow EPA, industry, and other stakeholders to more effectively
           gauge the potential hazards of HPV chemicals. All comments are
           publicly available on the World Wide Web. As of January 2006, EPA
           had commitments from industry sponsors to provide data for 2,247
           of the chemicals. The second program, the Voluntary Children's
           Chemical Evaluation Program, is designed to provide data that will
           allow the public to better understand the potential health risks
           to children associated with certain chemical exposures. EPA asked
           companies that manufacture or import 23 chemicals that have been
           found in human tissues in various biological monitoring programs
           to voluntarily sponsor the evaluation of specific chemicals in a
           pilot program. Thirty-five companies and 10 consortia volunteered
           to sponsor 20 chemicals. This program was developed only after
           considering comments and concerns from stakeholders. Of the 23
           chemicals chosen for this pilot, data gathering has been completed
           for 9 and is under way for another 11. The remaining 3 chemicals
           in the pilot program have no sponsors.

           EPA Maintains Databases of Information Related to Risk Assessment
			  
			  EPA maintains both an agencywide and several program-specific
           databases of information to help in the development of risk
           assessments. The primary database used by risk assessors in the
           program offices is the Integrated Risk Information System (IRIS),
           an electronic database of descriptive and quantitative information
           on human health effects that may result from exposure to various
           chemicals in the environment. Program office staff combine the
           IRIS data with exposure information they develop to characterize
           the health risks of a given chemical in a given situation.

           Since 1994, EPA has changed the IRIS assessment process in several
           ways. For example, each IRIS file now contains a discussion of the
           key studies, as well as a description of the decisions and default
           assumptions used in the assessment. EPA has also expanded the
           review that IRIS assessments undergo. For example, internal peer
           reviewers, including EPA senior health scientists representing
           program offices and regions, review the IRIS summary and
           accompanying detailed technical information. After this review,
           ORD releases the document for external peer review. EPA makes
           draft assessments available to the public at this time and,
           following peer review, the IRIS assessment discusses the key
           issues reviewers raised and EPA's response. In addition, EPA has
           added a tracking system that allows IRIS users to readily
           determine where an individual assessment is in its development.

           In September 2003, EPA completed a congressionally requested
           review to assess the need to update information in IRIS, based on
           concerns that EPA and state regulators rely on potentially
           outdated scientific information. Input from EPA program and
           regional offices, the public, and other stakeholders indicated
           that EPA should, among other things, increase the number of new or
           updated assessments completed each year to 50. To date, EPA has
           fallen considerably short of this goal. According to a program
           official, EPA completed 8 IRIS assessments in 2005, plans to
           complete 16 in 2006, and has approximately 75 assessments under
           way. EPA officials said a number of factors, such as the
           complexity of the assessment process, resource limitations, and
           extensive peer review, had limited EPA's ability to complete more
           assessments last year. EPA has increased the number of staff
           working on IRIS assessments from 6 to 23 and may ultimately
           increase the number to 29. The review also indicated that EPA
           needs to assign staff to develop health assessments for IRIS, and
           provide funding for extramural research and contracts to develop
           IRIS files and subject them to external peer review.

           EPA also changed how it sets priorities for which chemicals need
           new or updated IRIS assessments.33 Annually, EPA asks its program
           offices, regions, and the public to identify contaminants for
           which it should develop or revise IRIS assessments. EPA publishes
           the list in the Federal Register and requests the public and
           scientific community to submit any relevant data on substances
           undergoing review. EPA is currently reviewing ways to increase
           coordination with other governmental agencies that develop
           chemical assessments, outreach to stakeholders earlier in the
           development of IRIS assessments, and consultation with independent
           external reviewers. In 2004, the IRIS program also initiated a
           review of available scientific literature for the 460 chemicals in
           the database that are not under active reassessment to determine
           whether a reassessment based on new literature could significantly
           change existing toxicity information. For 63 percent of the
           chemicals reviewed, no major new health effects studies were
           found. Such literature reviews will be conducted annually and the
           findings noted in the IRIS database.

           In addition, some program offices maintain databases to enhance
           their risk assessments. For example, the Office of Air Quality
           Planning and Standards (OAQPS) maintains a database of
           dose-response values developed by various sources, including IRIS,
           ATSDR, and the California Environmental Protection Agency, as an
           aide for its risk assessors. OAQPS staff update this database as
           better data become available. As part of its National Air Toxics
           Assessment-an ongoing comprehensive evaluation of hazardous air
           pollutants in the United States-EPA assessed 32 air pollutants
           plus particulate matter in diesel exhaust in 1996. The national
           assessment is designed to identify air pollutants with the
           greatest potential to harm human health, and the results will help
           set priorities for collecting additional data. As part of its
           assessment, EPA compiled a national emissions inventory of
           hazardous air pollutants from outdoor sources, estimated
           population exposures to the pollutants, and characterized the
           potential cancer and noncancer health risks from breathing the
           pollutants.

           ORD also maintains personal monitoring data on the chemicals in
           the air, foods and beverages, water, and dust in an individual's
           personal indoor and outdoor environments. For example, in its
           National Human Exposure Assessment Survey (NHEXAS) program, which
           was completed in 1998, ORD collected human exposure data from
           hundreds of subjects from several areas of the country. NHEXAS
           provided data on background levels of total exposure to
           environmental contaminants that can be used as a baseline in
           exposure and risk assessments to estimate whether specific
           populations are exposed to increased levels of environmental
           contaminants.

           EPA Has Improved Its Choice and Communication of Default Options
			  
			  EPA has explicitly stated the need for risk assessors to identify
           when they relied on a default option, why they chose it, and when
           they departed from using a standard default option, but the agency
           acknowledges more could be done. To carry out its mission to
           protect human health, EPA's risk assessment procedures, including
           its default options, are protective of human health. In three
           recent guidance documents-the 2004 staff paper, the 2005 cancer
           guidelines, and the risk characterization handbook-EPA advocated
           more transparency in the choice of default options. These
           documents summarize a significant change in EPA's approach.
           Specifically, EPA first critically examines all relevant and
           available data to assess health risks, then uses the default
           options only in the absence of adequate contaminant-specific data.
           EPA also states in its staff paper that it bases its default
           assumptions on peer reviewed published studies, empirical
           observations, extrapolation from related observations, and
           scientific theory. Moreover, the cancer guidelines include an
           appendix that defines the basis for each of the default options
           that may be used in a cancer risk assessment. The Risk
           Characterization Handbook notes that risk assessments should
           describe the full range of default options that were used,
           including ones to address uncertainty. Moreover, the handbook
           states that when defaults are used, the risk assessment should
           reference the relevant EPA guidance that explains them.

           EPA program offices also advocate greater transparency when using
           default options. Specifically, the majority of IRIS assessments
           completed since 1997 describe the defaults used in the analysis
           and any departures from those defaults. The Office of Air Quality
           Planning and Standard's Air Toxics Risk Assessment Reference
           Library contains a number of references to defaults that should be
           used in the course of preparing a human health risk assessment.34
           For example, to estimate an individual's exposure to an air
           pollutant, the guidance presents the default option to use for
           exposure in a screening-level assessment, namely that the
           individual remains at a single location and continuously breathes
           polluted air.

           Despite the increased focus on more transparency in the use of
           defaults, EPA acknowledges it could more consistently describe how
           the default was developed and explain why it is a reasonable
           assumption. In its staff paper, EPA acknowledges it needs to
           ensure that the defaults are supported by the best available data
           and should look for opportunities to increase certainty and
           confidence in the defaults and extrapolations used. EPA also
           acknowledges it may need to re-examine older risk assessments that
           relied on defaults that can now be replaced with relevant data. To
           a large degree, the use of defaults is intertwined with EPA's
           ability to get the data it needs. As was discussed previously, EPA
           has targeted research, both within EPA and through its grant
           programs, to understand variability and uncertainty in the data
           derived from studies of laboratory animals, and this research may
           further reduce EPA's need to rely on default options.

           EPA Has Taken Steps to Enhance Its Modeling Capabilities and Embrace
           New Methodologies

           Since 1994, EPA has taken several steps to enhance its modeling
           capabilities and embrace new methodologies for risk assessment
           through improved guidance and workshops. To help improve its
           models, EPA's Agency Task Force on Environmental Regulatory
           Modeling published a report that concluded a need existed for,
           among other things, training and technical support, agency
           guidance on external peer review of environmental regulatory
           modeling, and creation of a Committee on Regulatory Environmental
           Modeling.35 Also in 1994, EPA's Risk Assessment Forum developed a
           draft protocol to evaluate models for exposure assessments.36 In
           1997, ORD and program offices conducted an agencywide conference,
           called the Models 2000 Workshop, to facilitate adherence to
           existing guidance on modeling, to define and implement
           improvements in how the agency developed and used models, and to
           recommend an implementation plan for improving modeling within the
           agency.

           EPA followed up these activities in 2000 by creating the Committee
           on Regulatory Environmental Modeling (CREM) to promote consistency
           and consensus within the agency on modeling issues (including
           modeling guidance, development, and application) and to enhance
           internal and external communications on modeling activities. CREM
           supports and enhances the existing modeling activities in the
           program offices and provides EPA with tools to support
           environmental decision making. CREM also provides the public and
           EPA staff with a central point of inquiry about EPA's use of
           models. In 2000, CREM launched agencywide activities designed to
           enhance the development, use, and selection of regulatory
           environmental models at EPA. One such activity-a workshop to
           facilitate discussion of good modeling practices-resulted in the
           development of modeling guidance.

           In 2003, CREM developed guidance and created a database-called the
           Models Knowledge Base-of the models most frequently used in EPA.37
           The guidance recommends best practices to help determine when a
           model, despite its uncertainties, can be appropriately used to
           inform a decision. Specifically, it recommends that model
           developers and users subject their model to credible, objective
           peer review, assess the quality of the data they use as inputs,
           and perform sensitivity and uncertainty analysis to determine
           which of the model inputs has the greatest impact on the modeled
           results.

           EPA has also incorporated efforts to improve models in its
           research strategies and implementation plans. For example, in its
           plan for research on hazardous air pollutants, EPA established a
           long-term goal to reduce uncertainties in risk assessments through
           methods, data, and models of acute and chronic exposures and
           exposures through multiple pathways at both the national and
           regional levels. In addition, one of ORD's laboratories
           established an exposure modeling research branch and develops
           population exposure models, such as the Stochastic Human Exposure
           and Dose Simulation model for inhalation and exposures of general
           and sensitive subpopulations through multiple pathways. EPA has
           also begun to use geographic information systems (GIS) to present
           risk information spatially. For example, a GIS system is being
           developed that maps all of the drinking water intakes in the
           United States and their associated watersheds, so that the agency
           can better assess risks to drinking water supplies stemming from
           activities in the related watershed. For risk assessments of
           hazardous air pollutants, GIS can display and analyze data during
           planning, scoping, and problem formulation, during the exposure
           assessment, and during the characterization of risks. GIS can also
           help communicate information to risk managers and other
           stakeholders.

           In addition to models, EPA is beginning to embrace such new risk
           assessment methodologies as probabilistic risk assessment and mode
           of action analysis. Probabilistic risk assessment characterizes
           the variability or uncertainty in risk estimates as the range or
           distribution of the number of times each possible outcome will
           occur. In probabilistic risk assessment, one or more variables in
           the risk equation, such as the exposure rate, is defined as a
           distribution rather than as a single number. A primary advantage
           of probabilistic risk assessment is that it provides a
           quantitative description of the degree of variability or
           uncertainty. EPA's 1997 policy states that probabilistic
           techniques, such as Monte Carlo analysis, can be viable
           statistical tools to analyze variability in risk assessments, when
           they are based on adequate supporting data and credible
           assumptions.38 The guidance presents a general framework and broad
           set of principles to ensure the use of good scientific practices
           when conducting probabilistic analyses of variability and
           uncertainty. EPA currently uses a number of models that include
           probabilistic analyses and is developing a new modeling framework,
           known as the Multimedia Integrated Modeling System, that will
           further enhance the agency's ability to probabilistically model
           uncertainty.

           EPA's recently revised cancer guidelines advocate the use of a
           mode of action analysis-based on the sequence of biological events
           that must occur to produce a harmful effect-to improve the
           accuracy of risk assessments. As a general rule, EPA assumes that
           toxic responses observed in laboratory animals indicate that the
           same responses are likely to occur in people even though
           differences in such areas as metabolic rates can result in
           different sensitivities between laboratory animals and humans.
           Mode of action analysis will more clearly indicate whether a
           difference exists between animals and humans in their response to
           contaminants. In addition, the guidelines present a new cancer
           characterization system consisting of five summary descriptors, to
           be used in conjunction with narrative, to describe the extent to
           which available data support the conclusion that a contaminant
           causes cancer in humans and to justify the summary descriptor
           selected. For noncancer risk assessments, EPA has used, and
           continues to refine, the benchmark dose methodology, which
           identifies the dose or concentration of a contaminant that
           slightly increases the likelihood of an adverse effect.

           EPA Is Introducing More Quantitative Characterization of
           Uncertainty into Its Risk Assessments

           Uncertainty is inherent in all phases of risk assessment, from
           hazard identification through risk characterization. Over the
           years, EPA has relied more on qualitative, or descriptive,
           characterizations of uncertainty and less on quantitative, or
           numeric, characterizations. EPA's practice now is to use a tiered
           approach to analyze uncertainty. That is, EPA starts with a simple
           description of uncertainty and sequentially employs more
           sophisticated quantitative analysis, such as sensitivity analysis,
           provided the additional analysis reduces the uncertainty. To
           characterize risk quantitatively, EPA has typically used
           approaches that produce a single number to characterize the risk
           in terms of the level of a contaminant that does not cause harm,
           as opposed to presenting a range of possible values.

           Although EPA is beginning to use probabilistic approaches in
           exposure assessments, and has done so for six principal air
           pollutants, it does not typically do so to analyze uncertainty in
           its dose-response analyses, though its Science Advisory Board
           encouraged development of such approaches in 2000. EPA
           acknowledged in its 2004 staff paper that probabilistic risk
           assessment could be used more frequently and could provide useful
           information beyond screening-level assessments. However, the staff
           paper said probabilistic analysis may not be appropriate in all
           situations and the accuracy of the analysis will depend largely on
           the availability and quality of the data used in the analysis.

           Another quantitative approach to uncertainty is the use of
           uncertainty factors to account for such unknowns as variation in
           sensitivity among members of the human population or the
           appropriateness of extrapolating animal data to humans. EPA
           routinely uses uncertainty factors when it estimates the daily
           exposure to the human population that is likely to be without
           appreciable risk of adverse effects during a lifetime. This daily
           exposure estimate is called a reference dose for contaminants that
           are consumed and a reference concentration for inhaled
           contaminants. EPA states in its 2004 staff paper that it applies
           uncertainty factors in health assessments based on available data
           and the scientific judgment of EPA risk assessors and peer
           reviewers. According to EPA, most IRIS toxicological assessments,
           which contain a chemical's reference dose and reference
           concentration, provide justifications for the uncertainty factors
           applied to a particular chemical. Moreover, the factors undergo
           rigorous internal, and independent, external scientific peer
           review before being used in IRIS assessments.

           EPA has issued a number of documents that delineate the need to
           clearly and consistently characterize uncertainty in risk
           assessments. In 1995, EPA issued a risk characterization policy
           that stated that the risk assessment should fully, openly, and
           clearly characterize risks, and should disclose the scientific
           analyses, uncertainties, assumptions, and policies underlying the
           decisions. This policy was followed in December 2000 by the Risk
           Characterization Handbook, which includes guidance on how to
           address, among other things, uncertainty in risk assessment and
           describes the need and methods to present the sources and
           magnitude of uncertainty to the risk manager. More recently, the
           2005 revised cancer guidelines discuss each of the major
           uncertainties, such as model uncertainty or uncertainty related to
           human variation, and stress that assessments should discuss the
           significant uncertainties encountered in the analysis. For
           example, the guidance calls for the assessments of hazard,
           dose-response, and exposure to have accompanying technical
           characterizations covering the strengths and limitations of data
           and a discussion of uncertainty.

           EPA Better Analyzes and Communicates Variability in both Exposure and
           Susceptibility
			  
			  EPA has made progress in describing variability due to differences
           in both the exposure individuals receive and the susceptibility of
           exposed individuals to adverse effects. A key document EPA risk
           assessors use to account for variation in exposure is its Exposure
           Factors Handbook, originally issued in 1989 and revised in 1997.
           The handbook summarizes data on human behaviors and
           characteristics that influence exposure to environmental
           contaminants and recommends values to account for those factors in
           assessing risk. Specifically, the handbook contains a series of
           over 150 data tables that provide information on how much time
           individuals spend at various activities and in various
           environments. Assessors can use these data to develop exposure
           duration estimates for exposure scenarios. For example, the tables
           contain statistics-broken down by age, gender, race, education,
           and some medical conditions, such as asthma or emphysema-for time
           spent in various outdoor locations. The handbook also provides
           general guidance to risk assessors on the types of variability
           relevant to a risk assessment and ways variability can be analyzed
           and addressed.

           All program offices address exposure variability in their risk
           assessments, although they do so in different ways. For example,
           risk assessors in the Office of Air Quality Planning and Standards
           who set certain air quality standards for six principal pollutants
           said they consider individual activity patterns for sensitive
           populations like children or asthmatics in exposure modeling by
           including a distribution of breathing rates to reflect variability
           inherent in the population. Furthermore, by modeling to protect
           the most sensitive or at-risks groups, they are assured of
           protecting the rest of the population. Variability in exposure to
           the six principal pollutants is generally described qualitatively
           in scientific summaries for each pollutant. The Office of Water
           includes an analysis of risks to various subpopulations and a
           narrative discussion of the strengths and weaknesses of the
           studies it used to estimate exposure, but generally does not
           include a quantitative analysis. The Office of Pesticide Programs
           considers 24 different population subgroups in its exposure
           estimates, including differences in age, gender, ethnicity, and
           geographic dispersion. When data allow, the Office of Pesticide
           Programs develops a distribution of exposures and risks for its
           more refined risk assessments.

           To further its understanding of variability in exposure, EPA has
           undertaken a number of research projects. For example, one of
           ORD's laboratories conducted the National Human Activity Pattern
           Survey to provide detailed human exposure information for specific
           populations and allow EPA to better understand actual human
           exposure to pollutants in real-world situations. The survey
           results are stored in the Consolidated Human Activity Database to
           help risk assessors estimate the time that exposed people spend in
           various environments and their inhalation, ingestion, and dermal
           absorption rates while in those environments. This laboratory also
           conducts research to define, quantify, and reduce the uncertainty
           associated with the exposure and risk assessments, to develop
           improved methods to more accurately measure exposure and dose, and
           to develop technical information and quantitative tools to predict
           the nature and magnitude of human exposures to environmental
           contaminants. A recent EPA study was designed to identify
           chemicals commonly used in homes or day care centers, and whether
           children in these environments encountered the chemicals in the
           course of their daily activities.39 The research sought to
           identify the major routes (i.e., breathing and ingestion) and
           sources (i.e., dust, food, air, soil, and water) through which
           children come into contact with chemicals.

           Variability also exists with regard to susceptibility to adverse
           affects because of inherent differences among humans. EPA most
           recently addressed variability in susceptibility in the 2005
           revision of its cancer guidelines, which describe the importance
           of separate risk assessments for all potentially sensitive life
           stages, including adults and children. The supplementary cancer
           guidelines for children address issues pertaining to cancer risks
           associated with early-life exposures. Legislation can also require
           EPA to consider potentially susceptible populations and life
           stages. For example, the Safe Drinking Water Act Amendments
           mandate that EPA consider risks to groups within the general
           population that are at greater risk of adverse health effects,
           including children, the elderly, and people with serious
           illnesses. In addition, the Food Quality Protection Act contains
           special provisions for the consideration of risks to children from
           pesticides. In 1995, EPA's Science Policy Council called for EPA
           to consider the risks to infants and children consistently and
           explicitly as part of its risk assessments. In 1997, the White
           House issued an executive order that required EPA and other
           federal agencies to identify and assess environmental health and
           safety risks that may disproportionately affect children and to
           ensure that policies, programs, activities, and standards address
           such disproportionate risks.40

           In its 2004 staff paper, EPA acknowledges that characterizing
           variability for susceptible populations and life stages is an area
           where it could improve, although the absence of data limits its
           efforts. In 2002, EPA pointed out that data are limited to
           identify susceptible populations and life stages for many
           contaminants.41 In these situations, EPA typically relies on
           default options, such as the use of uncertainty factors to account
           for variations in susceptibility. Many of the exposure assumptions
           and default values used to assess plausible current and future
           exposure scenarios can be found in EPA's 1997 Exposure Factors
           Handbook, and recent updates to the handbook are available online.

           Another way EPA addresses variability is through research. One of
           ORD's four strategic research directions in its Human Health
           Research Strategy is designed to improve the understanding of why
           some people and groups are more susceptible and highly exposed
           than others. According to this strategy, ORD's research on
           subpopulations will focus on three factors-life stage, genetic
           factors, and pre-existing diseases-that have been identified by a
           program office and the scientific community as having a high
           priority for risk assessment. In 2000, ORD released its Strategy
           for Research on Environmental Risks to Children to strengthen the
           scientific foundation of risk assessment and management decisions
           that affect children and guide EPA's research needs and priorities
           over the following 5 to 10 years. Approximately 75 percent of the
           funding for this strategy will be dedicated to research grants
           under the STAR program, such as those designed to evaluate
           children's exposure to pesticides.

           EPA Has Begun to Consider the Combined Effects of Exposure to
           Multiple Contaminants through Multiple Pathways
			  
			  To help risk assessors analyze the health effects of exposure to
           multiple contaminants (cumulative exposure) and through multiple
           routes (aggregate exposure), EPA has issued guidance, developed
           methods and models, and supported research. In 1997, EPA's Science
           Policy Council issued guidance on cumulative risk assessment. This
           guidance directs each office to consider cumulative risk in
           planning major risk assessments and, where relevant data are
           available, to broaden the scope of the assessment to integrate
           multiple sources, effects, pathways, stressors, and populations
           for cumulative risk analyses. The guidance also highlights the
           need to ensure that the public and other stakeholders have an
           opportunity to help define the way EPA assesses an environmental
           or public health problem and calls for ongoing communication and
           coordination among EPA's risk assessors, risk managers,
           economists, engineers, and other technical experts.

           In 2000, EPA updated its 1986 guidance on chemical mixtures, to
           generate a consistent agencywide approach to assess health risks
           from exposures to multiple chemicals.42 The guidance is organized
           according to the type of data available to risk assessors, ranging
           from data-rich to data-poor situations, to help risk assessors
           select an appropriate methodology. For example, if data are of
           poor quality or quantitative data are very limited on chemical
           mixtures, the risk assessor may choose to perform a qualitative
           analysis of the potential human health impacts from exposure to
           the mixture. The guidance also contains procedures to develop
           toxicity equivalency factors, based on the toxicity of components
           of the mixture, to assess the risk from mixtures in the absence of
           data on the specific mixture.

           In 2003, EPA's Risk Assessment Forum developed a simple, flexible
           framework to help risk assessors consistently conduct and evaluate
           cumulative risk assessments.43 The framework is conceptually
           similar to the one used in human health assessments of a single
           contaminant in that it follows a three-stage approach of (1)
           planning, scoping, and problem formulation; (2) analysis; and (3)
           risk characterization. In addition, the document also highlights
           needed areas of research and methods development that may be
           important to the evaluation of cumulative risks, such as
           understanding how the sequence and timing of exposure may
           influence the ultimate risk for effects.

           EPA risk assessors use a number of models and methodologies to
           analyze multiple routes of exposure. For example, the branch of
           the Office of Air Quality Planning and Standards that regulates
           hazardous air pollutants employs the Multiple Pathways of Exposure
           model to assess and predict the movement and behavior of chemicals
           in the environment. For example, the Multiple Pathways of Exposure
           model includes procedures to estimate human exposures and health
           risks that result from the transfer of pollutants from the air to
           soil and surface water bodies and the subsequent uptake of the
           pollutant by plants, animals, and humans. The model specifically
           addresses exposures from breathing; consuming food, water, and
           soil; and contact with skin. More recently, EPA developed the
           Total Risk Integrated Methodology (TRIM) and created the TRIM
           Fate, Transport, and Ecological Exposure model that describes the
           movement of air pollutants emitted from any type of stationary
           source as well as their transformation over time in water, air and
           soil.

           Some program offices have also taken steps to explicitly consider
           the risks associated with more than one route of exposure or more
           than one chemical. Specifically, the Office of Pesticide Programs
           issued guidance in 2001 and 2002 in response to statutory
           requirements to assess the risk of aggregate exposure-exposure to
           a single chemical by multiple pathways and routes.44 The first set
           of guidance focuses on how to assess aggregate risk in those cases
           where more extensive data and more sophisticated exposure
           assessment methods and tools are available; this guidance also
           emphasizes, when data are available, the use of distributional
           data-aggregate exposures of many individuals in the population of
           interest-for all pathways of exposure.45 This approach allows the
           risk assessor to more fully evaluate exposure and resulting risk
           across the entire population, rather than the exposure of a single
           highly exposed individual. The second set of guidance describes a
           framework to assess potential human health risks from all pathways
           of exposure to multiple pesticides that share a common mechanism
           of toxicity-that is, the pesticides produce a similar toxic effect
           on the same organ or tissue.46

           In addition, the branch of the Office of Air Quality Planning and
           Standards that regulates hazardous air pollutants developed the
           Integrated Urban Air Toxics Strategy, which is used to consider
           cumulative risks presented by exposures to hazardous air
           pollutants emitted from various sources. Staff can assess risk at
           both a national and an urban or a neighborhood scale. Furthermore,
           the hazardous air pollutant office developed guidance on
           multipathway risk assessments that are particularly important for
           hazardous air pollutants, such as mercury and dioxins, because
           human exposure occurs both from breathing air containing the
           toxins and from consuming plants, water, and soil where the
           pollutants were deposited. Moreover, these pollutants persist in
           the environment for long periods of time and may also accumulate
           in the tissues of commonly consumed plants and animals to levels
           that are harmful to humans.47 The guidance, maintained in the Air
           Toxics Risk Assessment Reference Library, describes how to plan,
           scope, and formulate the problem, conduct the analysis, and
           characterize the risk for such cases.

           The extent to which program offices assess the effects of
           cumulative and aggregate exposures is related to the regulatory
           responsibilities of each office and by the availability of data.
           For example, the hazardous air pollutant office routinely analyzes
           a mix of chemicals from various emitting sources, such as
           petroleum refineries, to regulate hazardous air pollutants.48
           Similarly, as mentioned above, the Office of Pesticide Programs is
           required to consider exposure to pesticides from various pathways,
           such as food, drinking water, and residential uses, and various
           routes, such as eating, breathing, and contact with skin. In
           contrast, the Toxic Substances Control Act does not require the
           Office of Pollution Prevention and Toxics to assess the risks of a
           new chemical that may occur through its interaction with other
           chemicals. The office also assesses the risks of existing
           chemicals but cannot conduct cumulative risk assessment for
           classes of chemical that share a common mode of action because no
           data exist. Program managers hope such data will become available
           in the future.

           In its 2004 staff paper, EPA commented that while it has increased
           its emphasis on evaluating cumulative risks, it needs to expand on
           approaches to do so, and it needs to produce a rigorous scientific
           base to support such evaluations. To that end, one of ORD's four
           strategic research directions in its Human Health Research
           Strategy is to improve assessments of aggregate and cumulative
           risks. Specific research objectives are to develop exposure models
           and methods, provide a scientific basis to predict interactive
           effects of contaminants in mixtures, and determine the most
           appropriate approaches to combine effects and risks from mixtures.

           EPA Risk Assessors Responding to Our Survey Reported That
           Process Modifications Have Helped Them Prepare Better Risk
           Assessments but That Collaboration and Training Limitations
           Hamper Further Progress
			  
			  EPA risk assessors responding to our survey reported that some
           modifications to its risk assessment processes, such as new or
           updated EPA guidance issued over the last 10 years, have been
           helpful. They also said that although collaboration among internal
           and external researchers has improved, problems remain with
           communication and coordination. Finally, risk assessors said that
           the training they've taken in the last 5 years has been
           beneficial, but they need additional training on analytic tools,
           such as modeling, and on other scientific disciplines related to
           risk assessment.

           EPA Has Issued Numerous Guidance Documents That Have Been Generally
           Helpful to Risk Assessors
			  
			  At least two-thirds of risk assessors responding to our survey who
           reported using guidelines or reference documents indicated that
           these documents were moderately to very helpful in preparing risk
           assessments.49 In addition, between one-third and two-thirds of
           respondents who reported using policy documents said these
           documents were moderately to very helpful in preparing risk
           assessments. More specifically, many risk assessors said
           agencywide guidelines and reference documents provide a framework
           to assess risks to human health that help make risk assessments
           more consistent. For example, some risk assessors noted the
           usefulness of agency reviewed or approved procedures to support
           their assessments. In addition, some risk assessors said the
           guidelines and reference documents helped clarify issues, and
           several assessors said they were a good source for data needed to
           conduct assessments. Risk assessors responding to our survey cited
           the Guidelines for Carcinogen Risk Assessment as the document most
           frequently used when preparing human health risk assessments. More
           specifically, several risk assessors noted that the carcinogen
           guidelines provide a useful framework for preparing risk
           assessments. Many risk assessors commented that agencywide
           guidelines and reference documents are helpful or provide useful
           examples. For example, a few risk assessors stated that the
           Exposure Factors Handbook helps provide consistency among EPA
           offices that conduct exposure assessments because it defines
           standard values for exposure, and the rationale behind those
           values. Another assessor said that the Review of the Reference
           Dose and Reference Concentration Processes provides comprehensive
           guidance on setting reference values and contains a case study
           that serves as a model for concise and well-written hazard
           identification. Although risk assessors responding to our survey
           reported that guidance documents are generally helpful, many
           expressed concerns about them. For example, some risk assessors
           consider the documents too general or too difficult to decipher.

           In addition, 82 percent of the risk assessors whose offices have
           office-specific guidance said that the guidance is very or
           moderately helpful with regard to preparing risk assessments. (See
           fig. 2.) According to many risk assessors, office-specific
           guidance provides information in a format relevant to each
           office's specific needs. For example, the Office of Pesticide
           Programs periodically issues "hot sheets" that describe how to
           apply general guidance to pesticide product risk assessments. In
           addition, the Office of Air and Radiation created the Air Toxics
           Risk Assessment Reference Library that provides information on how
           to analyze the risks from hazardous air pollutants. Over 65
           percent of risk assessors reported that EPA and program offices
           were moderately to very effective at disseminating guidance.

           Figure 2: Helpfulness of Office-Specific Guidance

           Note: Percentages do not total to 100 because of some risk
           assessors did not respond to this question.

           Collaboration Has Improved EPAï¿½s Ability to Conduct Risk Assessment,
           but Problems Remain
			  
			  To prepare risk assessments, EPA relies on external peer review
           and scientific data from a variety of sources on the potential
           health risks from exposure to contaminants. Collectively, this
           collaboration has improved EPA's ability to conduct those
           assessments. For example, 35 percent of risk assessors responding
           to our survey reported that external peer review, which is often
           conducted by independent researchers, has definitely helped
           improve the overall quality of EPA's risk assessments, with an
           additional 48 percent noting that it has probably helped improve
           the overall quality. Furthermore, at least 32 percent noted that
           peer review has definitely helped ensure that the science used in
           risk assessments is appropriately characterized, helped advance
           the use of new scientific methods or models, and helped EPA
           produce risk assessments that are more defensible. Another 38
           percent said that peer review has probably helped with these
           improvements. In addition, responding EPA risk assessors rely
           primarily on other federal research entities, academia, and
           industry, and to a slightly lesser extent, ORD's research
           laboratories, to meet their offices' research needs. (See fig. 3.)

           Figure 3: Risk Assessors That Generally or Strongly Agree That
           Organizations Help Fill Data Needs

           More specifically, 63 percent of risk assessors generally or
           strongly agreed that they relied on other federal research
           entities, such as the National Toxicology Program; 50 the National
           Institute for Environmental Health Sciences (NIEHS) and the
           National Cancer Institute-both within the National Institutes of
           Health; and the Agency for Toxic Substances and Disease Registry
           (ATSDR) to help fill their offices' needs for scientific data. EPA
           has established formal collaborative agreements with both NIEHS
           and ATSDR to research children's health and to develop
           toxicological data useful to both agencies, respectively. In
           addition, EPA

           has also collaborated with the U.S. Geological Service to identify
           contaminants in ground and surface waters.

           In addition to federal research entities, over 57 percent of
           respondents generally or strongly agreed that their offices rely
           on research from academia and industry to meet their research
           needs. For example, EPA has formal agreements with the
           International Life Sciences Institute's Risk Science Institute and
           the Health Effects Institute to develop research on approaches to
           analyze cumulative and aggregate exposures and the health effects
           of various air pollutants, respectively. EPA also relies heavily
           on industry-generated research on specific chemical substances.
           For example, under the laws that govern registration of pesticide
           products and new chemicals, applicants must supply specific data
           for relevant EPA offices to review when deciding whether to
           approve the pesticide products or chemicals in question.

           Furthermore, 46 percent of risk assessors said they relied on
           ORD's laboratories to generate research that helps fill scientific
           data needs. For example, some risk assessors said scientists from
           ORD's laboratories provide useful technical guidance on scientific
           issues or the risk assessment process. Another risk assessor
           commented that ORD's expertise is very useful to help interpret
           unusual findings or to advise on emerging issues. One risk
           assessor stated that ORD helped develop a specific model to use in
           probabilistic risk assessments. Other types of collaboration with
           ORD's laboratories include help to develop models for assessing
           dose and response relationships, to interpret toxicity data, to
           conduct epidemiological studies, and to develop scientific
           summaries for risk assessments of priority air pollutants.

           Overall, of the risk assessors who said they often or always
           collaborate with other EPA offices, at least 46 percent said that
           the collaboration was very effective. A few risk assessors
           commented that collaboration has become more effective in recent
           years, in part because staff in the various offices have more
           contact with one other, established scientist-to-scientist
           relationships, or learned whom to contact to address a particular
           question. For example, one risk assessor pointed out that
           cross-agency workgroups help to facilitate agencywide
           collaboration. Furthermore, a few risk assessors commented that
           the program offices' knowledge of the regulatory context in which
           research will be used helps ORD's researchers structure their
           work. For example, NERL collaborated with the Office of Air to
           fund studies that coordinated the format of data produced by
           different researchers to enhance the consistency of research
           approaches, which created more powerful results and made the data
           more useful to program offices. Finally, because some chemicals
           may be assessed by more than one office, collaboration across
           program offices helps ensure the consistency of risk assessments
           across EPA. For example, the Office of Pesticide Programs and the
           Office of Water may both prepare risk assessments for certain
           contaminants that may be found on food and in drinking water.

           Despite the improvements to collaboration at EPA, some risk
           assessors pointed out two barriers that limit collaboration.
           Specifically, assessors noted that conflicting priorities or goals
           among EPA offices and poor communication between some offices
           hinder the effectiveness of collaboration. For example, although
           some chemicals are studied by more than one office within EPA, the
           approaches and timelines differ among offices because the laws and
           responsibilities for each program office can differ significantly.
           As a result, what may be a priority chemical in one program office
           may not be a priority in another, thereby hindering timely
           collaboration. Furthermore, a couple of risk assessors found
           collaboration challenging because they could not find the right
           person in another office to communicate with on a specific issue.

           Several risk assessors suggested ways to improve and increase
           communication among program offices, ORD, and non-EPA
           organizations. For example, some risk assessors suggested more
           interagency work groups or meetings as a way to address research
           needs and foster information exchange on the development of
           methods. A few risk assessors suggested that a central library of
           risk assessment information would facilitate collaboration and
           avoid duplicating work already done by others. Specifically, one
           risk assessor said EPA could provide centralized databases of work
           conducted by different agencies and organizations, such as
           chemical-specific toxicity data, specific exposure or other
           values, and points of contact at each office.

           EPAï¿½s Training Has Been Helpful, but Risk Assessors and Managers
           Lack Sufficient Training on Analytic Tools and Emerging Issues
			  
			  At least 80 percent of risk assessors responding to our survey
           said that the training they received, whether on the job,
           self-directed, office specific, or agencywide, was moderately to
           very useful. Moreover, over half of these risk assessors said that
           training improved their ability to prepare risk assessments.
           Nevertheless, risk assessors surveyed and agency officials
           interviewed reported that both risk assessors and managers would
           benefit from more in-depth training on subjects such as analytic
           tools and emerging scientific issues.

           Risk Assessors Have Enhanced Their Skills through Training in
           Various Areas 
			  
			  Over half of the risk assessors reported that training had
           moderately or greatly improved their abilities in at least seven
           different risk assessment skill and knowledge areas. More
           specifically, over 75 percent of risk assessors reported that
           training has helped them learn about a particular risk assessment
           method or model, enhance the quality of risk assessments or risk
           characterizations they prepare, and maintain or refresh their
           existing knowledge (see fig. 4). To a slightly lesser extent,
           training also helped risk assessors apply EPA guidance and
           policies, more effectively prepare risk assessments, and
           understand EPA's four-stage paradigm for risk assessment.

           Figure 4: Areas in Which Training Has Moderately or Greatly
           Improved Risk Assessor Knowledge and Skills

           Over three-quarters of risk assessors reported that they
           participated in on-the-job (82 percent) or self-directed (77
           percent) training to enhance their skills. (See fig. 5.) According
           to risk assessors in the office that reviews new chemical
           applications, training is primarily on the job, largely because
           the office makes risk assessment decisions under tight time
           frames, which limits the time available for formal training,
           according to program officials. Similarly, workload constraints
           affect opportunities for formal training at the Office of Water,
           which instead relies primarily on self-directed or on-the-job
           training. Some on-the-job training takes the form of mentoring,
           such as in the Office of Research and Development, where senior
           staff are expected to mentor newer staff.

           Figure 5: Types of Training Taken by Risk Assessors

           In addition, over 70 percent of risk assessors who responded to
           our survey reported that they participated in office-specific
           training to enhance their risk assessment skills. For example, the
           Office of Pesticide Programs (OPP) has a comprehensive risk
           assessment training program focusing mostly on scientific issues,
           but also on other issues to improve the overall quality of risk
           assessments. This training features speakers, including some from
           outside OPP, such as from other EPA offices, academia, and
           industry. These biweekly sessions are broadcast live over the
           office's internal computer system and taped for future use as well
           as broader distribution, so that staff can access them as needed.
           Moreover, OPP's training program includes having experienced risk
           assessors nearing retirement share their knowledge with other
           staff. Of the risk assessors responding to our survey that work in
           OPP, over 90 percent took advantage of its office-specific
           training. A similarly high percentage of staff in the hazardous
           air pollutant branch of the Office of Air and Radiation reported
           that they took that office's training on preparing risk
           assessments.

           In addition, 50 percent of risk assessors said they attended
           external training, such as professional society meetings sponsored
           by the Society for Risk Analysis, Health Effects Institute, and
           the Society of Toxicology and university courses related to
           scientific methods and disciplines. Finally, 46 percent of risk
           assessors said they participated in agencywide training. According
           to an EPA official, the agency offers a broad array of agencywide
           risk assessment training. For example, EPA's Risk Assessment Forum
           develops training to accompany the agency's risk assessment
           guidelines. More specifically, forum staff conducted briefings and
           orientation sessions on the 2005 cancer risk assessment
           guidelines. In addition, the forum sponsors colloquia
           approximately twice a year for staff to exchange information on
           risk assessment issues. Recent colloquia dealt with dermal
           exposure assessment and whether additional guidance is needed on
           the use of Monte Carlo analysis. Colloquia usually result in
           reports that summarize the findings and may inform future guidance
           documents.

           Additional Training Is Needed
			  
			  While risk assessors reported taking advantage of and benefiting
           from various forms of training, they also said additional training
           would improve their ability to prepare risk assessments, a belief
           echoed by agency officials we spoke with. Over 70 percent of risk
           assessors stated that more in-depth or relevant training would
           improve their ability to prepare risk assessments. More
           specifically, some risk assessors expressed the need for training
           on analytic tools, such as modeling the dose-response relationship
           and statistical analysis and software. Several risk assessors also
           expressed a strong interest in training on scientific topics,
           including toxicology and such emerging issues as genomics, as well
           as nonscientific topics, such as public communications and public
           relations. Some risk assessors suggested EPA provide formal,
           comprehensive training for new risk assessors to educate them on
           how to use the guidance documents and more advanced courses for
           experienced risk assessors. Furthermore, many risk assessors
           reported that more time (69 percent) and more funds (65 percent)
           for training would moderately or greatly improve their ability to
           prepare risk assessments. (See fig. 6.)

           Figure 6: Modifications in Training That Would Moderately or
           Greatly Improve the Preparation of Risk Assessments

           In addition to our survey respondents, some program officials we
           interviewed pointed out the need for additional training for both
           risk assessors and risk managers. For example, risk assessors need
           training in emerging issues, such as genomics, so that they can
           use these types of data in risk assessments, according to one
           agency official. In contrast, risk managers need training on the
           elements of risk assessment, so that they will be better able to
           interpret the information provided and apply it to risk management
           decisions. According to one program manager, risk managers often
           want the "bottom line" (numerical) results of a risk assessment
           without understanding the nature of the uncertainties in the
           assessment, or the potential value of obtaining more information
           to help clarify those uncertainties.

           Finally, the changing nature of the workforce may have
           implications for training at EPA. According to an agency official,
           many experienced risk assessors who possess years of institutional
           knowledge are retiring or nearing retirement age. Consequently,
           the agency needs to educate newer risk assessment staff as quickly
           and thoroughly as possible to help ensure that the agency's
           ability to accurately and effectively produce risk assessments
           does not decline. While recently hired risk assessors come to EPA
           with specialized knowledge in fields related to risk assessment,
           they may not understand the broader context of risk assessment.
           For example, a new employee with degrees in biology and toxicology
           may not know how to integrate that knowledge with other scientific
           information to prepare a risk assessment. Furthermore, one EPA
           official noted that the agency has no formal training course, or
           set of courses, to help develop staff's ability to prepare risk
           assessments.

           Enhanced Risk Assessment Planning, Improved Analysis and Review, and
           Added Training Could Further Improve EPAï¿½s Process, but
           Barriers Could Limit Progress
			  
			  The experts we spoke with, including representatives of federal
           and state agencies, regulated industry, environmental advocacy
           groups, and outside researchers and consultants, said the
           modifications EPA has made over the past 10 years have been
           beneficial overall. However, they identified additional actions
           EPA could take to improve its risk assessment process, recognizing
           that EPA would face barriers to doing so. Specifically, EPA could
           improve its planning process of what will be required to complete
           a risk assessment by better identifying the scientific data it has
           and data it needs on the potential adverse effects from exposure
           to a contaminant and by seeking stakeholder input early in this
           planning process. In addition, EPA could more thoroughly evaluate
           methods and models, transparently document its analytic choices,
           and enhance internal review. Finally, experts believe EPA could
           provide additional training for risk assessors, managers, and
           stakeholders. While these efforts would further improve the risk
           assessment process, EPA could face barriers in carrying them out,
           such as the scientific complexity of risk assessment, the
           difficulty of obtaining and applying data, and a cultural
           resistance to deviating from established methods.

           Enhance Planning by Increasing Focus on Data Needs and Involving a
           Broad Range of Stakeholders
			  
			  In order to ensure that EPA has the data needed for risk
           assessment, it needs to better identify data that are available,
           prioritize its data needs, and collaborate with the external
           research community during the planning phase. For example, several
           experts said EPA should generate a searchable database of studies
           conducted by different agencies and organizations related to the
           chemicals being evaluated, so that researchers and risk assessors
           could more easily identify what studies are available and what
           additional research is needed. Experts also suggested several ways
           for EPA to prioritize its data needs. For example, sensitivity
           analysis can be used within individual risk assessments to
           determine which data gaps are the most critical to the risk
           assessment result. Some experts also said EPA could better
           prioritize its data needs by increasing its use of data on the
           amounts of contaminants in people's bodies to help concentrate its
           research on the chemicals to which humans are actually exposed.
           Finally, several experts suggested that EPA increase its
           collaboration with external researchers, in part because the
           agency lacks the resources to independently generate all of the
           data that are needed. If EPA more effectively collaborated with
           other federal research organizations, such as the National
           Toxicology Program and the National Institute of Environmental
           Health Sciences, federal research dollars could be better
           harnessed to help EPA protect the public from exposure to
           contaminants. For example, the National Toxicology Program has the
           technology to assist the Office of Pesticide Programs with its
           screening of inert ingredients, which are all of the "other"
           chemicals in a pesticide product. In addition, experts said EPA
           should use all relevant data, including data from industry
           research laboratories, provided EPA takes steps to ensure the data
           are generated in an unbiased and scientifically defensible way.
           For example, experts suggested that EPA could subject studies to
           independent peer review and evaluate the sufficiency of data
           produced by these organizations to increase confidence about using
           these data in EPA risk assessments.

           Experts also said that EPA could improve the quality of risk
           assessments if the agency enhanced its planning by more
           consistently involving stakeholders, especially early in the
           process. Several experts said that increased involvement with a
           broad range of stakeholders early in the planning process would
           help identify alternate methods and models to use and obtain
           stakeholder concurrence with the agency's approach. Although all
           stakeholders might not agree on the methods chosen, some experts
           believe that by seeking stakeholder input on these issues early in
           the process, EPA may minimize arguments later. In addition to
           stakeholder involvement with early planning, several experts
           recommended that EPA increase coordination with stakeholders
           throughout the process. For example, one expert said EPA could
           more transparently acknowledge and address comments from the
           public and other stakeholders, regardless of whether the agency
           planned to implement their suggestions.

           More Thoroughly Evaluate and Transparently Document Analytic
           Choices, and Increase Internal Review
			  
			  EPA could more thoroughly evaluate its analytic choices and
           incorporate or develop a wider variety of analytic tools. Some
           experts said that, with regard to EPA's use of default options,
           risk assessors should more thoroughly document why available data
           are insufficient to allow EPA to use another analytic approach and
           commented that the revised cancer guidelines may provide a useful
           framework for making this decision. Furthermore, several experts
           said EPA could use a tool like sensitivity analysis to assess and
           clearly communicate the extent to which the choice of a method or
           default assumption affects the risk assessment outcome. For
           example, if sensitivity analysis demonstrated that the impacts
           were significant, EPA could use the analysis to identify the
           critical areas where additional studies might reduce the need to
           rely on a default assumption for that assessment. One expert
           pointed out that the Office of Pesticide Programs uses sensitivity
           analysis in its exposure assessments to estimate which uses of a
           pesticide present the greatest risk to workers and how to mitigate
           those risks. In addition, several experts recommended that EPA
           make better use of existing analytic tools and develop new ones,
           where needed. For example, some experts said EPA should more
           frequently employ probabilistic analyses in risk assessments and
           incorporate the latest scientific tools, such as genomics and
           computational toxicology, to better assess uncertainty and
           variability. In addition, several experts noted that EPA needs to
           develop tools and methodologies to better analyze certain aspects
           of risk assessment, such as the combined effects of exposure to
           multiple chemicals through multiple pathways.

           Experts also said EPA risk assessments should clearly describe the
           sufficiency of the data and the scientific basis for its choice of
           a default assumption, method, or model. Some experts pointed out
           that risk assessments should identify and clearly discuss any data
           that are not available for the analysis, including the form the
           data need to be in and the most appropriate study design or
           methodology to obtain the needed data. In addition, several
           experts said EPA needs to more explicitly communicate which
           default assumptions were used in a risk assessment and why the
           defaults were chosen. For example, one expert said that even
           though a risk assessment may be perfect, if the public does not
           understand the rationale behind the agency's choices, the risk
           assessment might be seen as flawed. Furthermore, in individual
           risk assessments, the agency could more transparently identify
           which critical studies would help the agency avoid relying on
           default assumptions. Some experts also suggested that EPA use as
           case studies completed assessments for which the agency had
           sufficient data to use models and other analytic tools rather than
           default assumptions to more accurately assess risks. Finally, some
           experts said that EPA should more transparently consider alternate
           methods and models in each risk assessment. For example, EPA
           should be more transparent about the judgments it makes when it
           employs certain methods, such as the benchmark dose method, which
           identifies the dose that produces a small increase in the risk of
           an adverse effect.

           Finally, experts suggested that EPA increase internal reviews of
           risk assessments by staff members with extensive risk assessment
           experience. Internal reviews could improve the risk assessment
           process in two ways: first, to assure the quality of risk
           assessments and second, to ensure that the design of its risk
           assessments match the needs of risk managers. For example, one
           expert suggested EPA reinstate a senior peer review group,
           composed of experienced risk assessors from throughout the agency.
           Others suggested that EPA could also internally peer review risk
           assessments prepared by less experienced staff to ensure that
           default assumptions are applied appropriately and transparently
           explained. In addition to increased review of individual risk
           assessments, some experts also felt the risk assessment process
           could benefit from additional examination of agencywide
           cross-cutting issues applicable to all program offices. For
           example, one expert said that some analytic tools, such as Monte
           Carlo analysis, were not developed specifically for use in risk
           assessments and suggested that EPA work with ORD's National Center
           for Computational Toxicology to define how these tools could be
           used in risk assessments across EPA. Moreover, according to
           several experts, agencywide discussions and activities promote
           consistency in risk assessment practices. For example, some
           experts thought EPA could benefit from a systematic agencywide
           discussion of the sources of uncertainty in risk assessment.

           More Training Could Improve Risk Assessor, Risk Manager, and
           Stakeholder Understanding of the Process
			  
			  Experts emphasized the importance of training for risk assessors,
           risk managers, and the stakeholder community on all elements of
           the risk assessment process. Several experts said risk assessors
           are not adequately trained in basic risk assessment principles,
           such as available default assumptions and when they should be used
           or replaced. Some experts also suggested risk assessors receive
           training in using and applying models and in how to interpret data
           from emerging scientific fields to improve their ability to use
           these data, as appropriate, in risk assessments. Several experts
           also believe that training for risk managers would help improve
           risk assessments because risk managers need to better understand
           the role risk assessment plays in risk management. According to
           some experts, risk managers who are more familiar with the process
           are better equipped to support risk assessors and ensure that the
           risk assessment considers all appropriate factors. Finally, a few
           experts also suggested that EPA hold training for stakeholders in
           the risk assessment process. For example, one expert suggested
           that EPA develop Web-based training for both the regulated
           community and regulators themselves to help ensure consistency in
           how they understand the process. As part of this training, EPA
           could explain how risk assessment fits into the overall risk
           management process.

           EPA Faces Barriers to Improving Its Risk Assessment Process, Such
           as the Complexity of Risk Assessment, Difficulty of Acquiring and Applying
           Data, and a Culture Resistant to Change
			  
			  While experts identified a number of actions EPA could take to
           improve the risk assessment process, they said EPA may face
           barriers such as the highly complex, technical, and time-intensive
           nature of preparing risk assessments, challenges in acquiring and
           applying data from all available sources, and a general reluctance
           to deviate from its established methods and assumptions. Several
           experts pointed out that EPA's risk assessments have grown more
           technically challenging and require risk assessors and managers to
           possess different skills than in the past. For example, some
           experts told us risk managers have different levels of expertise
           and background in risk assessment, and may not fully understand
           how risk assessment helps inform regulatory decisions. Moreover,
           some experts said that because risk assessment is just one piece
           of information used to make a regulatory decision, it is difficult
           to explain to stakeholders and the public the impact of risk
           assessments on risk management decisions. Experts also pointed out
           that scientific knowledge on subjects, such as uncertainty and
           variability, is limited and analytic tools are still being
           developed. For example, several experts said that while it would
           be useful for EPA to more fully consider the risks of exposure to
           a single chemical from all exposure pathways, at present it is an
           emerging science with few well developed analytic tools to use in
           risk assessments. In addition, using tools, such as probabilistic
           analysis, to assess variability requires large amounts of data
           that are seldom available. Finally, several experts said that
           improving the process by such steps as incorporating new analytic
           techniques and conducting thorough internal review requires more
           time and coordination. For example, one expert pointed out that
           EPA does not always have the staff and time to analyze all sets of
           data or to examine alternative methods or models that might
           provide a more robust risk estimate.

           In addition to barriers attributable to the complexity of
           preparing risk assessments, experts also said EPA may face
           barriers in acquiring and applying data from all available
           sources. Many experts commented that data are expensive to obtain,
           and EPA has limited financial resources to devote to such
           activities. For example, some experts pointed out that some of the
           more direct studies of human exposure, such as epidemiological or
           biomonitoring studies, are quite expensive to conduct.
           Furthermore, EPA may be reluctant to use available data from all
           sources. As several experts pointed out, data from
           industry-sponsored researchers might be perceived as biased,
           potentially subjecting EPA to criticism. Despite potential
           perceptions of bias, some experts thought EPA should have the
           ability to use all available data, regardless of its source, as
           long as the data in question have been appropriately peer
           reviewed. In addition, some experts said statutory requirements
           may limit EPA's ability to use certain data. For example, the
           Toxic Substances Control Act limits EPA's authority to require
           extensive data from industry before deciding whether to approve a
           new chemical. Some experts also pointed out that research does not
           always produce clear-cut results. For example, one expert
           commented that epidemiological studies of the general population
           may not account for confounding factors, such as exposure to other
           chemicals, which complicate efforts to draw conclusions about the
           effects of a single chemical. In addition, some experts said that
           variability, an important but scientifically complicated issue,
           often creates inconsistencies across studies because many factors
           such as geography, lifestyle, and food intake affect an
           individual's response.

           Finally, experts said that EPA has a general reluctance to deviate
           from using methods and assumptions it has used in the past. As a
           result, experts said EPA prefers to use techniques that have been
           generally accepted in the scientific community than to use methods
           that rely on recent scientific advances. For example, some experts
           told us EPA is often reluctant to deviate from its established
           default assumptions. Furthermore, some experts also commented that
           risk assessors may not have an incentive to deviate from methods
           and assumptions they have used in the past because it may make the
           risk assessment more easily challenged by those who disagree with
           it. In addition, some experts said the level of comfort in using
           new methods varies throughout the agency. For example, one expert
           believes that probabilistic models have been applied
           inconsistently because some risk assessors have been unwilling to
           deviate from the standard models.

           Conclusions
			  
			  While technical and difficult to understand by nature, risk
           assessment is a key element in EPA's efforts to protect human
           health from the potentially harmful effects of chemicals,
           pollutants, and toxic substances that people encounter in their
           everyday lives. Since 1994, EPA has taken a number of steps,
           including greater involvement by the public and other
           stakeholders, to strengthen and improve its process for preparing
           assessments of the risks posed by contaminants in the environment.
           Independent reviewers as well as the experts and EPA risk
           assessors we contacted said overall EPA's efforts have improved
           the agency's risk assessments. However, the agency itself and the
           individuals we contacted acknowledge that EPA needs to do more.
           While some barriers to further improvement depend on scientific
           advances that are largely beyond EPA's control, other actions to
           improve its risk assessment process are within its control.
           Specifically, when EPA engages the stakeholder and research
           communities after the risk assessment has largely been completed,
           it misses opportunities to benefit from their expertise. By
           working with stakeholders early and periodically throughout the
           process to identify key issues, studies, methods, and default
           assumptions that need to be considered in the analysis, EPA would
           help ensure consistent, transparent, and high-quality risk
           assessments. On the other hand, failure to take full advantage of
           stakeholders' knowledge and points of view is likely to contribute
           to the perception among stakeholders that their concerns are not
           adequately represented in the risk assessments and that EPA's
           decisions lack transparency. While EPA has issued a number of
           guidance and policy documents advocating the benefits of early
           planning, it acknowledges it could do more to ensure that such
           planning and consultation take place and involve relevant
           stakeholders. Furthermore, EPA does not always systematically
           communicate its data needs to the research community. While EPA
           has begun to better identify and prioritize its specific data
           needs, it has not been able to consistently develop data it needs
           in a timely manner. A more proactive approach to communicating its
           research needs to outside public and private researchers would
           help EPA more efficiently use the limited resources it has to
           obtain the data it needs. Furthermore, this approach would
           increase the likelihood that EPA would have data it needs to
           complete risk assessments now and into the future and that
           appropriately designed research projects would be conducted.
           Transparently communicating its research needs would also enhance
           EPA's ability to produce high-quality, scientifically defensible
           risk assessments and reduce the uncertainty associated with the
           effects of many contaminants on human health. Although experts we
           interviewed said EPA may hesitate to seek and use data from a wide
           range of sources because it could be seen as biased, EPA could
           take steps to ensure the quality of data generated by others. By
           doing so, EPA would expand its cache of available data and,
           potentially, reduce its reliance on default assumptions. Finally,
           current workforce models of high-performing organizations stress
           the need to formally and comprehensively assess the skill and
           competency requirements for staff and to identify related training
           and developmental needs to ensure that the workforce retains a
           high level of needed skills. In recent years, EPA's emphasis on
           training for its risk assessors and managers has declined in the
           areas risk assessors and experts say are needed to improve the
           quality of risk assessments and take advantage of recent
           scientific and analytic advances. Without an agencywide training
           program for its risk assessment and risk management workforce, the
           quality, consistency, and transparency of risk assessments and
           risk management decisions will likely continue to be challenged by
           stakeholders and the public.

           Recommendations for Executive Action

           To improve the overall quality, consistency, and transparency of
           its risk assessments, we recommend that the Administrator of EPA
           direct the appropriate agency entities to take the following three
           actions:

           o  Develop a strategy to ensure that offices engage in early
           planning to identify and seek the expertise needed, both within
           the EPA workforce and from external subject matter experts. The
           strategy should delineate such things as how EPA could use the
           available expertise to determine the needed data, the relevant
           default assumptions, the extent of internal and external review
           that needs to be included in the assessment, and the approach used
           to consistently involve a broad range of stakeholders-including
           the public, regulated industry, federal agencies, and advocacy
           groups-as appropriate to the risk assessment.

           o  More proactively identify the data most relevant to the current
           risk assessment needs, including the specific studies required and
           how those studies should be designed, and communicate those needs
           to the research community. Increased collaboration among program
           offices in identifying needed data would help ensure that the
           resulting data will meet the needs of multiple offices. In
           addition, EPA should better communicate these data needs and
           better coordinate research planning with the external public and
           private research community to help focus EPA's limited resources.

           o  Ensure that risk assessors and risk managers have the skills
           needed to produce quality risk assessments by developing and
           implementing in-depth training. This training should address the
           needs of risk assessors and managers with varying levels of
           expertise by including basic courses, such as an overview of risk
           assessment, as well as more advanced courses on topics such as
           modeling, toxicology, and other advanced scientific techniques.

           Agency Comments
			  
			  We provided EPA with a draft of this report for review and
           comment. EPA neither agreed nor disagreed with our findings and
           recommendations. However, the agency provided specific comments to
           improve the report's technical accuracy, which we incorporated as
           appropriate.

           As agreed with your office, unless you publicly announce the
           contents of this report earlier, we plan no further distribution
           until 30 days from the report date. At that time, we will send
           copies to the Administrator, EPA, as well as to appropriate
           congressional committees, and other interested Members of
           Congress. We also will make copies available to others upon
           request. In addition, the report will be available at no charge on
           the GAO Web site at http://www.gao/gov .

           If you or your staff have questions about this report, please
           contact me at (202) 512-6225 or [email protected] . Contact
           points for our Offices of Congressional Relations and Public
           Affairs may be found on the last page of this report. GAO staff
           who made key contributions to this report are listed in appendix
           II.

           Sincerely yours,

           John B. Stephenson Director, Natural Resources and Environment

           Appendix I: Objectives, Scope, and Methods
			  
			  Our objectives for this review were to (1) identify the
           significant recommendations to improve human health risk
           assessment that have been made since 1994; (2) describe what the
           Environmental Protection Agency (EPA) has done to modify its human
           health risk assessment process over the same period; (3) determine
           the effects these past modifications have had on the preparation
           of risk assessments; and (4) identify any additional actions
           experts believe EPA could take to improve its risk assessment
           process in the future, and the barriers EPA would face in doing
           so.

           To identify significant recommendations to improve human health
           risk assessment since 1994, we reviewed EPA documents, including
           those produced by EPA's Risk Assessment Forum (RAF), Science
           Policy Council (SPC), Council for Regulatory Environmental
           Modeling (CREM), and Science Advisory Board (SAB) as well as each
           of EPA's program offices. We also reviewed our own reports and
           documents produced by the National Academy of Sciences and the
           Presidential/Congressional Commission on Risk Assessment and Risk
           Management. To gain an external perspective, we spoke with experts
           in the risk assessment field, who identified many of these
           documents in the course of our discussions and provided insight
           into some of the documents' recommendations.

           To describe what EPA has done to modify its human health risk
           assessment process, we interviewed program office managers from
           the Office of Air and Radiation (OAR), Office of Pesticide
           Programs (OPP), Office of Pollution Prevention and Toxics (OPPT),
           Office of Water (OW), and Office of Research and Development
           (ORD). We did not include the site-specific risk assessment
           activities of the Office of Solid Waste and Emergency Response in
           our review. Within ORD, we interviewed managers in two of EPA's
           laboratories (the National Health and Environmental Effects
           Laboratory and the National Exposure Research Laboratory) and
           three of EPA's research centers (the National Center for
           Environmental Assessment, National Center for Environmental
           Research, and National Center for Computational Toxicology). We
           also interviewed officials from RAF, SPC, CREM, SAB, and the
           Office of the Science Advisor. Furthermore, we attended various
           EPA and stakeholder group training sessions and meetings. Since we
           limited our review to the human health aspects of risk assessment
           since 1994, our analysis does not highlight EPA's modifications
           prior to 1994, including publication of guidance documents that
           are highly relevant to risk assessment practices, and does not
           address issues specifically related to ecological risk assessment.

           To assess the effects these modifications have had on the
           preparation of risk assessments, we conducted a Web-based
           nonprobability survey of all human health risk assessors from ORD
           and four EPA program offices that conduct human health risk
           assessment (OAR, OPP, OPPT, and OW). We used the survey to obtain
           an internal perspective on the usefulness of many of the
           modifications EPA made since 1994, as well as on aspects of EPA's
           risk assessment process, including guidance documents, training,
           organizational structure, and collaboration. In developing the
           Web-based questionnaire, we met with EPA officials from the five
           offices surveyed to gain a thorough understanding of the risk
           assessment issues specific to each office and identify the
           sampling frame. In order to identify human health risk assessors-a
           label that is not an EPA job series-we obtained from EPA officials
           in each program office being reviewed the names of agency staff
           who worked on any part of the human health risk assessment process
           since January 2001. Our sampling frame consisted of 270 staff that
           met this criterion. This report does not contain all the results
           from the survey. The survey and results can be viewed at
           http://www.gao.gov/cgi-bin/getrpt?GAO-06-637SP .

           The practical difficulties of conducting any survey may introduce
           nonsampling error. For example, differences in how a particular
           question is interpreted, the sources of information available to
           respondents, or the types of people who do not respond can
           introduce unwanted variability into the survey results. In order
           to reduce nonsampling error, we pretested the questionnaire with
           five risk assessors, one from each of the offices surveyed. During
           these pretests, we asked agency officials to complete the survey
           as we observed the process. We then interviewed the respondents to
           ensure that (1) the questions were clear and unambiguous, (2) the
           terms used were precise, (3) the questionnaire did not place an
           undue burden on the agency officials completing it, and (4) the
           questionnaire was independent and unbiased. On the basis of the
           feedback from the pretests, we modified the questions, as
           appropriate. Information about accessing the questionnaire was
           provided via e-mail for all survey participants. The survey was
           activated, and staff informed of its availability on October 17,
           2005; it was available until January 13, 2006. To ensure security
           and data integrity, we provided all participants with a personal
           password that allowed them to access and complete a questionnaire.
           No one else could access that questionnaire or edit its data. We
           included steps in both the data collection and data analysis
           stages for the purpose of minimizing such nonsampling errors. To
           reduce survey nonresponse, we sent e-mail reminders and conducted
           follow-up telephone calls with nonrespondents. Overall, 82 percent
           of the 270 risk assessors in our sampling frame responded to our
           survey, and all offices had a response rate of at least 80
           percent.

           We used general modifiers (i.e., many, several, some, a few, and a
           couple) to characterize written responses to some open ended
           survey questions. We used the following method to assign these
           modifiers to our statements about risk assessor's survey
           responses: "many" represents 22 to 44 respondents (roughly 10 to
           20 percent), "several" represents 12 to 21 respondents (5 to 10
           percent), "some" represents 4 to 11 respondents, "a few"
           represents 3 respondents, and "a couple" represents 2 respondents.
           These divisions do not represent technically established
           categories; rather, we chose these divisions because they aligned
           with natural breaks in response "themes" highlighted in the
           report.

           To assess further actions EPA could take to improve its risk
           assessment process and to identify barriers it may face in doing
           so, we interviewed experts representing a range of stakeholders in
           the process. Specifically, we contacted risk assessment
           scientists; toxicologists; scientific advisers to EPA; state
           officials; and representatives from regulated industries,
           government agencies, and environmental advocacy groups who have an
           expertise in risk assessment. We used an iterative process (often
           referred to as the "snowball sampling" technique) to identify
           these knowledgeable experts and selected for interviews those who
           would provide us with a broad and balanced range of perspectives
           on EPA risk assessment practices.

           We first contacted the National Academy of Sciences' Board of
           Environmental Studies and Toxicology, which is the academy's
           principal study unit for environmental pollution problems
           affecting human health and the assessment and management of
           related risks to human health and the environment. We presented
           our engagement to the board and sought its input on the areas in
           which EPA has made the most progress improving its risk assessment
           practices and areas EPA will need to focus on in the future. We
           also asked members if they would be willing to participate in a
           future interview and solicited the names of other experts who
           would be appropriate for us to contact about this engagement. We
           selected for interviews experts who would provide us with a broad
           and balanced range of perspectives on EPA risk assessment
           practices. We continued interviewing and soliciting names until we
           determined we had appropriate coverage from all the relevant
           stakeholder groups. Our sampling identified 22 experts, listed
           alphabetically, as follows: Elizabeth L. Anderson, Ph.D.; Gail
           Charnley, Ph.D.; Harvey J. Clewell, M.A.; Shannon Cunniff; Kerry
           Dearfield, Ph.D.; Michael L. Dourson, Ph.D.; Elaine M. Faustman,
           Ph.D.; Paul Gilman, Ph.D.; Gary Ginsberg, Ph.D.; Sherri Goodman,
           Esq.; Judith A. Graham, Ph.D.; Dan Greenbaum; Leslie J. Hushka,
           Ph.D.; Annie M. Jarabek, B.S.; James H. Johnson, Ph.D.; Elizabeth
           Julien, Ph.D.; Dorothy Patton, Ph.D.; Jonathan M. Samet, Ph.D.;
           Jennifer Sass, Ph.D.; Chris Whipple, Ph.D.; Richard Wiles, M.A.;
           and Lauren Zeise, Ph.D.

           We used a standard set of questions to interview each of these
           experts to ensure we consistently discussed each aspect of EPA
           risk assessment policies and practices. To develop the questions,
           we reviewed documentation on EPA's risk assessment process and
           reports prepared by the National Academy of Sciences. We pretested
           our questions with two of the experts and refined the questions
           accordingly. We used content analysis to identify the main themes
           among their responses. In addition, we asked the experts for their
           opinions about the many risk assessment modifications EPA has made
           since 1994, and used content analysis to synthesize their
           comments.

           We conducted our work from February 2005 through March 2006 in
           accordance with generally accepted government auditing standards.

           Appendix II: GAO Contact and Staff Acknowledgments

           GAO Contact
			  
			  John B. Stephenson, (202) 512-6225 or [email protected]

           Staff Acknowledgments
			  
			  In addition to the contact named above, Cheryl Williams (Assistant
           Director), Jennifer Cook, Michelle Cooper, Elizabeth Erdmann, and
           Rebecca Shea made key contributions to this report. Also
           contributing to this report were Nancy Crothers, Richard Frankel,
           and Roderick Moore.

           GAOï¿½s Mission
			  
			  The Government Accountability Office, the audit, evaluation and
           investigative arm of Congress, exists to support Congress in
           meeting its constitutional responsibilities and to help improve
           the performance and accountability of the federal government for
           the American people. GAO examines the use of public funds;
           evaluates federal programs and policies; and provides analyses,
           recommendations, and other assistance to help Congress make
           informed oversight, policy, and funding decisions. GAO's
           commitment to good government is reflected in its core values of
           accountability, integrity, and reliability.

           Obtaining Copies of GAO Reports and Testimony

           The fastest and easiest way to obtain copies of GAO documents at
           no cost is through GAO's Web site ( www.gao.gov ). Each weekday,
           GAO posts newly released reports, testimony, and correspondence on
           its Web site. To have GAO e-mail you a list of newly posted
           products every afternoon, go to www.gao.gov and select "Subscribe
           to Updates."

           Order by Mail or Phone
			  
			  The first copy of each printed report is free. Additional copies
           are $2 each. A check or money order should be made out to the
           Superintendent of Documents. GAO also accepts VISA and Mastercard.
           Orders for 100 or more copies mailed to a single address are
           discounted 25 percent. Orders should be sent to:

           U.S. Government Accountability Office 441 G Street NW, Room LM
           Washington, D.C. 20548

           To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax:
           (202) 512-6061

           To Report Fraud, Waste, and Abuse in Federal Programs

           Contact:

           Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail:
           [email protected] Automated answering system: (800) 424-5454 or
           (202) 512-7470

			  Congressional Relations

           Gloria Jarmon, Managing Director, [email protected] (202) 512-4400
           U.S. Government Accountability Office, 441 G Street NW, Room 7125
           Washington, D.C. 20548

           Public Affairs
			  
			  Paul Anderson, Managing Director, [email protected] (202)
           512-4800 U.S. Government Accountability Office, 441 G Street NW,
           Room 7149 Washington, D.C. 20548

3Pollutants are generally categorized as hazardous air pollutants under
the Clean Air Act if they cause or may cause cancer or other serious
health effects, such as reproductive effects or birth defects, or adverse
environmental and ecological effects. Currently, the Clean Air Act
regulates 187 chemicals and chemical categories as hazardous air
pollutants.

4National Academy of Sciences, Risk Assessment in the Federal Government:
Managing the Process (Washington, D.C., 1983).

5The Presidential/Congressional Commission was created by Pub. L. No.
101-549 (1990).

6Peer review is a documented critical review of a specific scientific or
technical work product, conducted by qualified individuals who are
independent of those who performed the work, but who are collectively
equivalent in technical expertise-for example, EPA's Science Advisory
Board or the National Academy of Sciences. According to EPA's 2006 Peer
Review Policy, peer review can be internal, in which the reviewers are
independent experts from inside EPA, or external, in which the reviewers
are independent experts from outside EPA.

7National Academy of Sciences, Science and Judgment in Risk Assessment
(Washington, D.C., 1994).

8National Academy of Sciences, Understanding Risk: Informing Decision in a
Democratic Society (Washington, D.C., 1996).

9National Academy of Sciences, Strengthening Science at the U.S.
Environmental Protection Agency (Washington, D.C., 2000).

10EPA's Science Advisory Board provides independent advice and peer review
on scientific and technical aspects of environmental problems and issues.
Experts, including scientists, engineers, and economists, provide
independent, balanced, and scientifically sound advice to EPA.

11IRIS contains information on the human health effects that may result
from exposure to various chemicals in the environment and helps provide
consistent information on chemical substances for use in risk assessments.

12The Agency for Toxic Substances and Disease Registry is the primary
public health agency involved with hazardous waste issues and works to
prevent or reduce the harmful effects of exposure to hazardous substances
on human health by, for example, supporting research that assists with
risk assessment.

13Endocrine disrupting chemicals are thought to mimic natural human
hormones that influence important regulatory and development mechanisms
such as blood pressure, metabolism, and reproduction. Important endocrine
glands include the thyroid, pancreas, and male and female gonads (testes
and ovaries).

14The STAR program funds peer reviewed competitive grants on research
topics selected by ORD and postgraduate training for scientists in
environmental fields.

15EPA's defaults represent a choice that, although scientifically
plausible given the existing uncertainty, is more likely to result in
overestimating than underestimating human health risk.

16Contaminants can progressively accumulate in the tissues of an organism,
such as a human or a fish, as a result of uptake by the body from all
routes of exposure. This process, called bioaccumulation, occurs because
the rate of intake exceeds the organism's ability to eliminate the
substance from the body.

17Neurotoxicity is an adverse change in the structure or function of the
central or peripheral nervous system following exposure to a chemical,
physical, or biological agent.

18U.S. Environmental Protection Agency, Peer Review Handbook (Washington,
D.C., 2000).

19ORD's laboratories and centers are the National Center for Environmental
Assessment (NCEA), National Health and Environmental Effects Research
Laboratory (NHEERL), National Exposure Research Laboratory (NERL),
National Center for Environmental Research (NCER), National Risk
Management Research Laboratory (NRMRL), and National Homeland Security
Research Center.

20U.S. Environmental Protection Agency, Risk Assessment Principles and
Practice (Washington, D.C., 2004).

21U.S. Environmental Protection Agency, Guidance on Cumulative Risk
Assessment-Part 1: Planning and Scoping (Washington, D.C., 1997).

22U.S. Environmental Protection Agency, Lessons Learned on Planning and
Scoping for Environmental Risk Assessments (Washington, D.C., 2002).

23These six principal pollutants, known as "criteria pollutants," are
carbon monoxide, lead, nitrogen dioxide, ozone, particulate matter, and
sulfur dioxide.

24EPA is reviewing older pesticides (those initially registered prior to
November 1984) under the Federal Insecticide, Fungicide, and Rodenticide
Act to ensure they meet current scientific and regulatory standards. EPA
is also reassessing tolerances (pesticide residue limits in food) to
ensure they meet safety standards established under the Food Quality
Protection Act (FQPA) of 1996. Under FQPA, EPA must reassess all
tolerances established before August 3, 1996, within 10 years. The Office
of Pesticide Programs reviews the registrations of all pesticide products
every 15 years to determine whether they still meet safety standards as
part of its registration review process.

25Office of Management and Budget, Guidelines for Ensuring and Maximizing
the Quality, Objectivity, Utility, and Integrity of Information
Disseminated by Federal Agencies (Washington, D.C., 2002).

26U.S. Environmental Protection Agency, A Summary of General Assessment
Factors for Evaluating the Quality of Scientific and Technical Information
(Washington, D.C., 2003).

27In September 2002, EPA formed the National Homeland Security Research
Center to, among other things, provide appropriate, affordable, effective,
and validated technologies and methods for assessing risks posed by
chemical, biological, and radiological terror attacks.

28To comply with the Government Performance and Results Act (GPRA), EPA
every 3 years generates an agencywide 5-year strategic plan that
highlights high-level environmental issues. The most recent plan for
fiscal years 2003-2008 identified key research needs related to EPA's
mission.

29The four strategic research directions are harmonizing cancer and
noncancer risk assessments, assessing aggregate and cumulative risk,
determining risk to susceptible human subpopulations, and conducting
research to enable evaluation of public health outcomes from risk
management decisions.

30The Federal Insecticide, Fungicide, and Rodenticide Act (FIFRA), 7
U.S.C. S:S: 136-136y.

31Applicants must submit data on the acute and chronic toxicity of the
pesticide product under different conditions, such as how and at what rate
it can be applied. See 40 C.F.R. Part 158 (2005).

32ILSI-RSI is primarily funded through cooperative agreements with
regulatory agencies such as EPA and Health Canada, but also receives
financial support from the European Commission, the National Institute of
Environmental Health Sciences, and industry trade groups.

33EPA uses four criteria to prioritize chemicals for IRIS reviews: (1) EPA
statutory, regulatory, or program needs; (2) availability of new
scientific information that might significantly change the current
information; (3) interest from state and local government or the public;
and (4) the amount of additional information that would be needed to
complete the review. Ultimately, the decision to assess any given
substance hinges on available agency resources.

34The Air Toxics Risk Assessment Reference Library is a multivolume
technical resource manual that provides information on the fundamental
principles of risk-based assessment for hazardous air pollutants and how
to apply those principles in different settings.

35U.S. Environmental Protection Agency, Report of the Agency Task Force on
Environmental Regulatory Modeling-Guidance, Support Needs, Draft Criteria
and Charter (Washington, D.C., 1994).

36U.S. Environmental Protection Agency, Model Validation for Predictive
Exposure Assessments (Washington, D.C., 1994).

37U.S. Environmental Protection Agency, Draft Guidance on the Development,
Evaluation, and Application of Regulatory Environmental Models
(Washington, D.C., 2003).

38U.S. Environmental Protection Agency, Policy for Use of Probabilistic
Analysis in Risk Assessment (Washington, D.C., 2003).

39U.S. Environmental Protection Agency, Children's Total Exposure to
Persistent Pesticides and Other Persistent Organic Pollutants (Washington,
D.C., 1999).

40Exec. Order No. 13045, 62 Fed. Reg. 19885 (Apr. 21, 1997).

41U.S. Environmental Protection Agency, Review of the Reference Dose and
Reference Concentration Processes (Washington, D.C., 2002).

42U.S. Environmental Protection Agency, Supplementary Guidance for
Conducting Health Risk Assessment of Chemical Mixtures (Washington, D.C.,
2000).

43U.S. Environmental Protection Agency, Framework for Cumulative Risk
Assessment (Washington, D.C., 2003).

44See S: 405 of the Food Quality Protection Act of 1996, amending 21
U.S.C. S: 346a(b)(2)(D)(vi).

45U.S. Environmental Protection Agency, General Principles for Performing
Aggregate Exposure and Risk Assessments (Washington, D.C., 2001).

46U.S. Environmental Protection Agency, Guidance on Cumulative Risk
Assessment of Pesticide Chemicals That Have a Common Mechanism of Toxicity
(Washington, D.C., 2002).

47For example, metals released into the air may be deposited on the
ground, where they remain in surface soils for long periods of time. The
chemicals in the soil may be taken up into plants through the roots and
accumulate in the tissues of foraging animals.

48By statute, hazardous air pollutants are regulated not as individual
pollutants but by emission sources that consist of a group of similar
industrial processes or industries that release multiple pollutants.

49Guidelines refer to the Guidelines for Carcinogen Risk Assessment,
Guidelines for Neurotoxicity Risk Assessment, Guidelines for Reproductive
Toxicity Risk Assessment, and Supplemental Guidelines for Chemical
Mixtures. Reference documents refer to the Assessment Factors Handbook,
Exposure Factors Handbook, Framework for Cumulative Risk Assessment,
Guiding Principles for Monte Carlo Analysis, Peer Review Handbook, Review
of Reference Dose and Reference Concentration Processes, Risk Assessment
Principles and Practices, and Risk Characterization Handbook. Policy
documents refer to the Policy on Evaluating Health Risks to Children,
Policy for Use of Probabilistic Analysis in Risk Assessment, Interim
Genomics Policy, and Interim Position on Environmental Endocrine
Disruption.

50The National Toxicology Program is an interagency program established by
the Department of Health and Human Services that provides information
about potentially toxic chemicals to health, regulatory, and research
agencies, scientific and medical communities, and the public.

(360546)

www.gao.gov/cgi-bin/getrpt?GAO-06-595 .

To view the full product, including the scope

and methodology, click on the link above. To view the results of GAO's
survey of EPA's risk assessors, click www.gao.gov /cgi-bin/getrpt?
GAO-06-637SP . For more information, contact John Stephenson at (202)
512-6225 or [email protected].

Highlights of GAO-06-595 , a report to the Chairman, Committee on
Environment and Public Works, U.S. Senate

May 2006

HUMAN HEALTH RISK ASSESSMENT

EPA Has Taken Steps to Strengthen Its Process, but Improvements Needed in
Planning, Data Development, and Training

Over 100,000 chemicals, pollutants, and toxic substances are used in the
United States and regulated by the Environmental Protection Agency (EPA).
EPA uses risk assessment to determine the health risk from exposure to
these substances, collectively referred to as contaminants. In the last 12
years, independent reviewers have examined this process and made
recommendations for how it could be improved. GAO was asked to (1)
identify the significant recommendations that have been made to improve
human health risk assessment; (2) describe what EPA has done to modify its
human health risk assessment process; (3) determine the effects these past
modifications have had on the preparation of risk assessments; and (4)
identify any additional actions experts believe EPA could take to improve
its process, and the barriers it would face in doing so.

What GAO Recommends

GAO recommends that EPA enhance early planning of each risk assessment,
identify and communicate data needs to the public and private research
community, and support development and implementation of in-depth training
for risk assessors and managers. EPA neither agreed nor disagreed with our
findings and recommendations. However, the agency provided specific
technical comments, which we incorporated as appropriate.

Since 1994, independent reviewers recommended that EPA better plan its
risk assessments. In doing so, they said EPA should better utilize
scientific data it has and identify other data it needs on the potential
adverse effects from exposure to contaminants, and prioritize and support
research to meet those needs. Furthermore, reviewers recommended that EPA
better evaluate the analytic tools it uses and employ more powerful tools
when appropriate. Reviewers also recommended that EPA better analyze and
characterize the sources of uncertainty in its risk assessments. Finally,
they recommended that EPA enhance its analysis of variability in exposure
to contaminants and in susceptibility to harm from exposure, and improve
how it considers the effects of exposure to multiple contaminants and
through many sources.

EPA has strengthened its risk assessment process since 1994 and
improvement efforts are ongoing. For example, EPA has increased planning
for assessments and has initiated actions to develop missing or incomplete
scientific data. EPA has also begun to embrace new methodologies, such as
ones to predict how the body will react to a contaminant. Furthermore, EPA
now uses a tiered approach to conducting uncertainty analysis, employing
more sophisticated analysis as warranted. Finally, EPA has made progress
in characterizing variability due to differences in both exposure and
susceptibility of exposed individuals and has begun to take steps to
address exposure to multiple contaminants and through multiple sources.

According to EPA's risk assessors, the modifications EPA has made have
generally helped improve risk assessments. Many EPA risk assessors believe
that agencywide guidance has helped them prepare risk assessments and have
resulted in greater consistency across program offices. Furthermore, while
most assessors report collaboration with internal and external entities is
effective and has improved the quality of risk assessments, some said
conflicting priorities and poor communication hindered collaboration among
some EPA offices. Finally, while risk assessors said training has helped
them gain skills and knowledge, over 70 percent said that more in-depth or
relevant training would improve their risk assessment abilities.

Experts identified additional actions EPA could take to further improve
its risk assessment process, recognizing that it may face barriers in
doing so. Experts said EPA could improve its planning process by better
focusing on scientific data needs and involving stakeholders early to
obtain their concurrence with EPA's approach. Experts also said EPA could
more thoroughly evaluate methods and models, transparently document its
analytic choices, and enhance internal review. Finally, experts said EPA
could provide additional training for risk assessors, managers, and
stakeholders on the risk assessment process. Experts, however, said that
the scientific complexity of risk assessment, the difficulty of obtaining
and applying data, and a cultural resistance to deviating from established
methods could act as obstacles to successfully making such changes.
*** End of document. ***