[Federal Register Volume 70, Number 10 (Friday, January 14, 2005)]
[Notices]
[Pages 2664-2677]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 05-769]
=======================================================================
-----------------------------------------------------------------------
OFFICE OF MANAGEMENT AND BUDGET
Final Information Quality Bulletin for Peer Review
AGENCY: Office of Management and Budget, Executive Office of the
President.
ACTION: Final bulletin.
-----------------------------------------------------------------------
SUMMARY: On December 16, 2004, the Office of Management and Budget
(OMB), in consultation with the Office of Science and Technology Policy
(OSTP), issued its Final Information Quality Bulletin for Peer Review
to the heads of departments and agencies (available at http://www.whitehouse.gov/omb/memoranda/fy2005/m05-03.html). This new guidance
is designed to realize the benefits of meaningful peer review of the
most important science disseminated by the Federal Government. It is
part of an ongoing effort to improve the quality, objectivity, utility,
and integrity of information disseminated by the Federal Government to
the public. This final bulletin has benefited from an extensive
stakeholder process. OMB originally requested comment on its ``Proposed
Bulletin on Peer Review and Information Quality,'' published in the
Federal Register on September 15, 2003. OMB received 187 public
comments during the comment period (available at http://www.whitehouse.gov/omb/inforeg/2003iq/iq_list.html). In addition, to
improve the draft Bulletin, OMB encouraged federal agencies to sponsor
a public workshop at the National Academy of Sciences (NAS). The NAS
workshop (November 18, 2003, at the National Academies in Washington,
DC) attracted several hundred participants, including leaders in the
scientific community (available at http://www7.nationalacademies.org/stl/STL_Peer_Review_Agenda.html). OMB also participated in outreach
activities with major scientific organizations and societies that had
expressed specific interest in the draft Bulletin. A formal interagency
review of the draft Bulletin, resulting in detailed comments from
numerous Federal departments and agencies, was undertaken in
collaboration with the White House Office of Science and Technology
Policy. In light of the substantial interest in the Bulletin, including
a wide range of constructive criticisms of the initial draft, OMB
decided to issue a revised draft for further comment. This revised
draft was published in the Federal Register on April 28, 2004, and
solicited a second round of public comment. The revised draft
stimulated a much smaller number of comments (57) (available at: http://www.whitehouse.gov/omb/inforeg/peer2004/list_peer2004.html). OMB's
response to the additional criticisms, suggestions, and refinements
offered for consideration is available at: http://www.whitehouse.gov/omb/inforeg/peer2004/peer_response.pdf. The final Bulletin includes
refinements that strike a balance among the diverse perspectives
expressed during the comment period. Part I of the SUPPLEMENTARY
INFORMATION below provides background. Part II provides the text of the
final Bulletin.
DATES: The requirements of this Bulletin, with the exception of those
in Section V (Peer Review Planning), apply to information disseminated
on or after June 16, 2005. However, they do not apply to information
for which an agency has already provided a draft report and an
associated charge to peer reviewers. The requirements in Section V
regarding ``highly influential scientific assessments'' are effective
June 16, 2005. The requirements in Section V regarding ``influential
scientific information'' are effective December 16, 2005.
[[Page 2665]]
FOR FURTHER INFORMATION CONTACT: Dr. Margo Schwab, Office of
Information and Regulatory Affairs, Office of Management and Budget,
725 17th Street, NW., New Executive Office Building, Room 10201,
Washington, DC 20503. Telephone (202) 395-5647 or email: [email protected].
SUPPLEMENTARY INFORMATION:
Introduction
This Bulletin establishes that important scientific information
shall be peer reviewed by qualified specialists before it is
disseminated by the Federal government. We published a proposed
Bulletin on September 15, 2003. Based on public comments, we published
a revised proposal for additional comment on April 28, 2004. We are now
finalizing the April version, with minor revisions responsive to the
public's comments.
The purpose of the Bulletin is to enhance the quality and
credibility of the government's scientific information. We recognize
that different types of peer review are appropriate for different types
of information. Under this Bulletin, agencies are granted broad
discretion to weigh the benefits and costs of using a particular peer
review mechanism for a specific information product. The selection of
an appropriate peer review mechanism for scientific information is left
to the agency's discretion. Various types of information are exempted
from the requirements of this Bulletin, including time-sensitive health
and safety determinations, in order to ensure that peer review does not
unduly delay the release of urgent findings.
This Bulletin also applies stricter minimum requirements for the
peer review of highly influential scientific assessments, which are a
subset of influential scientific information. A scientific assessment
is an evaluation of a body of scientific or technical knowledge that
typically synthesizes multiple factual inputs, data, models,
assumptions, and/or applies best professional judgment to bridge
uncertainties in the available information. To ensure that the Bulletin
is not too costly or rigid, these requirements for more intensive peer
review apply only to the more important scientific assessments
disseminated by the Federal government.
Even for these highly influential scientific assessments, the
Bulletin leaves significant discretion to the agency formulating the
peer review plan. In general, an agency conducting a peer review of a
highly influential scientific assessment must ensure that the peer
review process is transparent by making available to the public the
written charge to the peer reviewers, the peer reviewers' names, the
peer reviewers' report(s), and the agency's response to the peer
reviewers' report(s). The agency selecting peer reviewers must ensure
that the reviewers possess the necessary expertise. In addition, the
agency must address reviewers' potential conflicts of interest
(including those stemming from ties to regulated businesses and other
stakeholders) and independence from the agency. This Bulletin requires
agencies to adopt or adapt the committee selection policies employed by
the National Academy of Sciences (NAS) \1\ when selecting peer
reviewers who are not government employees. Those that are government
employees are subject to federal ethics requirements. The use of a
transparent process, coupled with the selection of qualified and
independent peer reviewers, should improve the quality of government
science while promoting public confidence in the integrity of the
government's scientific products.
---------------------------------------------------------------------------
\1\ National Academy of Sciences, ``Policy and Procedures on
Committee Composition and Balance and Conflicts of Interest for
Committees Used in the Development of Reports,'' May 2003: Available
at: http://www.nationalacademies.org/coi/index.html.
---------------------------------------------------------------------------
Peer Review
Peer review is one of the important procedures used to ensure that
the quality of published information meets the standards of the
scientific and technical community. It is a form of deliberation
involving an exchange of judgments about the appropriateness of methods
and the strength of the author's inferences.\2\ Peer review involves
the review of a draft product for quality by specialists in the field
who were not involved in producing the draft.
---------------------------------------------------------------------------
\2\ Carnegie Commission on Science, Technology, and Government,
Risk and the Environment: Improving Regulatory Decision Making,
Carnegie Commission, New York, 1993: 75.
---------------------------------------------------------------------------
The peer reviewer's report is an evaluation or critique that is
used by the authors of the draft to improve the product. Peer review
typically evaluates the clarity of hypotheses, the validity of the
research design, the quality of data collection procedures, the
robustness of the methods employed, the appropriateness of the methods
for the hypotheses being tested, the extent to which the conclusions
follow from the analysis, and the strengths and limitations of the
overall product.
Peer review has diverse purposes. Editors of scientific journals
use reviewer comments to help determine whether a draft scientific
article is of sufficient quality, importance, and interest to a field
of study to justify publication. Research funding organizations often
use peer review to evaluate research proposals. In addition, some
Federal agencies make use of peer review to obtain evaluations of draft
information that contains important scientific determinations.
Peer review should not be confused with public comment and other
stakeholder processes. The selection of participants in a peer review
is based on expertise, with due consideration of independence and
conflict of interest. Furthermore, notice-and-comment procedures for
agency rulemaking do not provide an adequate substitute for peer
review, as some experts--especially those most knowledgeable in a
field--may not file public comments with Federal agencies.
The critique provided by a peer review often suggests ways to
clarify assumptions, findings, and conclusions. For instance, peer
reviews can filter out biases and identify oversights, omissions, and
inconsistencies.\3\ Peer review also may encourage authors to more
fully acknowledge limitations and uncertainties. In some cases,
reviewers might recommend major changes to the draft, such as
refinement of hypotheses, reconsideration of research design,
modifications of data collection or analysis methods, or alternative
conclusions. However, peer review does not always lead to specific
modifications in the draft product. In some cases, a draft is in
excellent shape prior to being submitted for review. In others, the
authors do not concur with changes suggested by one or more reviewers.
---------------------------------------------------------------------------
\3\ William W. Lowrance, Modern Science and Human Values, Oxford
University Press, New York, NY 1985: 85.
---------------------------------------------------------------------------
Peer review may take a variety of forms, depending upon the nature
and importance of the product. For example, the reviewers may represent
one scientific discipline or a variety of disciplines; the number of
reviewers may range from a few to more than a dozen; the names of each
reviewer may be disclosed publicly or may remain anonymous (e.g., to
encourage candor); the reviewers may be blinded to the authors of the
report or the names of the authors may be disclosed to the reviewers;
the reviewers may prepare individual reports or a panel of reviewers
may be constituted to produce a collaborative report; panels may do
their work electronically or they may meet together in person to
discuss and prepare their evaluations; and reviewers may be compensated
for their work or they may donate their time as a
[[Page 2666]]
contribution to science or public service.
For large, complex reports, different reviewers may be assigned to
different chapters or topics. Such reports may be reviewed in stages,
sometimes with confidential reviews that precede a public process of
panel review. As part of government-sponsored peer review, there may be
opportunity for written and/or oral public comments on the draft
product.
The results of peer review are often only one of the criteria used
to make decisions about journal publication, grant funding, and
information dissemination. For instance, the editors of scientific
journals (rather than the peer reviewers) make final decisions about a
manuscript's appropriateness for publication based on a variety of
considerations. In research-funding decisions, the reports of peer
reviewers often play an important role, but the final decisions about
funding are often made by accountable officials based on a variety of
considerations. Similarly, when a government agency sponsors peer
review of its own draft documents, the peer review reports are an
important factor in information dissemination decisions but rarely are
the sole consideration. Agencies are not expected to cede their
discretion with regard to dissemination or use of information to peer
reviewers; accountable agency officials must make the final decisions.
The Need for Stronger Peer Review Policies
There are a multiplicity of science advisory procedures used at
Federal agencies and across the wide variety of scientific products
prepared by agencies.\4\ In response to congressional inquiry, the U.S.
General Accounting Office (now the Government Accountability Office)
documented the variability in both the definition and implementation of
peer review across agencies.\5\ The Carnegie Commission on Science,
Technology and Government \6\ has highlighted the importance of
``internal'' scientific advice (within the agency) and ``external''
advice (through scientific advisory boards and other mechanisms).
---------------------------------------------------------------------------
\4\ Sheila Jasanoff, The Fifth Branch: Science Advisors as
Policy Makers, Harvard University Press, Boston, 1990.
\5\ U.S. General Accounting Office, Federal Research: Peer
Review Practices at Federal Agencies Vary, GAO/RCED-99-99,
Washington, DC, 1999.
\6\ Carnegie Commission on Science, Technology, and Government,
Risk and the Environment: Improving Regulatory Decision Making,
Carnegie Commission, New York, 1993: 90.
---------------------------------------------------------------------------
A wide variety of authorities have argued that peer review
practices at federal agencies need to be strengthened.\7\ Some
arguments focus on specific types of scientific products (e.g.,
assessments of health, safety and environmental hazards).\8\ The
Congressional/Presidential Commission on Risk Assessment and Risk
Management suggests that ``peer review of economic and social science
information should have as high a priority as peer review of health,
ecological, and engineering information.'' \9\
---------------------------------------------------------------------------
\7\ National Academy of Sciences, Peer Review in the Department
of Energy--Office of Science and Technology, Interim Report,
National Academy Press, Washington, DC, 1997; National Academy of
Sciences, Peer Review in Environmental Technology Development: The
Department of Energy--Office of Science and Technology, National
Academy Press, Washington, DC, 1998; National Academy of Sciences,
Strengthening Science at the U.S. Environmental Protection Agency:
Research-Management and Peer-Review Practices, National Academy
Press, Washington, DC, 2000; U.S. General Accounting Office, EPA's
Science Advisory Board Panels: Improved Policies and Procedures
Needed to Ensure Independence and Balance, GAO-01-536, Washington,
DC, 2001; U.S. Environmental Protection Agency, Office of Inspector
General, Pilot Study: Science in Support of Rulemaking 2003-P-00003,
Washington, DC, 2002; Carnegie Commission on Science, Technology,
and Government, In the National Interest: The Federal Government in
the Reform of K-12 Math and Science Education, Carnegie Commission,
New York, 1991; U.S. General Accounting Office, Endangered Species
Program: Information on How Funds Are Allocated and What Activities
are Emphasized, GAO-02-581, Washington, DC, 2002.
\8\ National Research Council, Science and Judgment in Risk
Assessment, National Academy Press, Washington, DC, 1994.
\9\ Presidential/Congressional Commission on Risk Assessment and
Risk Management, Risk Commission Report, Volume 2, Risk Assessment
and Risk Management in Regulatory Decision-Making, 1997:103.
---------------------------------------------------------------------------
Some agencies have formal peer review policies, while others do
not. Even agencies that have such policies do not always follow them
prior to the release of important scientific products.
Prior to the development of this Bulletin, there were no
government-wide standards concerning when peer review is required and,
if required, what type of peer review processes are appropriate. No
formal interagency mechanism existed to foster cross-agency sharing of
experiences with peer review practices and policies. Despite the
importance of peer review for the credibility of agency scientific
products, the public lacked a consistent way to determine when an
important scientific information product is being developed by an
agency, the type of peer review planned for that product, or whether
there would be an opportunity to provide comments and data to the
reviewers.
This Bulletin establishes minimum standards for when peer review is
required for scientific information and the types of peer review that
should be considered by agencies in different circumstances. It also
establishes a transparent process for public disclosure of peer review
planning, including a Web-accessible description of the peer review
plan that the agency has developed for each of its forthcoming
influential scientific disseminations.
Legal Authority for the Bulletin
This Bulletin is issued under the Information Quality Act and OMB's
general authorities to oversee the quality of agency information,
analyses, and regulatory actions. In the Information Quality Act,
Congress directed OMB to issue guidelines to ``provide policy and
procedural guidance to Federal agencies for ensuring and maximizing the
quality, objectivity, utility and integrity of information''
disseminated by Federal agencies. Public Law No. 106-554, Sec. 515(a).
The Information Quality Act was developed as a supplement to the
Paperwork Reduction Act, 44 U.S.C. 3501 et seq., which requires OMB,
among other things, to ``develop and oversee the implementation of
policies, principles, standards, and guidelines to * * * apply to
Federal agency dissemination of public information.'' In addition,
Executive Order 12866, 58 FR 51,735 (Oct. 4, 1993), establishes that
OIRA is ``the repository of expertise concerning regulatory issues,''
and it directs OMB to provide guidance to the agencies on regulatory
planning. E.O. 12866, Sec. 2(b). The Order also requires that ``[e]ach
agency shall base its decisions on the best reasonably obtainable
scientific, technical, economic, or other information.'' E.O. 12866,
Sec. 1(b)(7). Finally, OMB has authority in certain circumstances to
manage the agencies under the purview of the President's Constitutional
authority to supervise the unitary Executive Branch. All of these
authorities support this Bulletin.
The Requirements of This Bulletin
This Bulletin addresses peer review of scientific information
disseminations that contain findings or conclusions that represent the
official position of one or more agencies of the Federal government.
Section I: Definitions
Section I provides definitions that are central to this Bulletin.
Several terms are identical to or based on those used in OMB's
government-wide information quality guidelines, 67 FR 8452 (Feb. 22,
2002), and the Paperwork Reduction Act, 44 U.S.C. 3501 et seq.
[[Page 2667]]
The term ``Administrator'' means the Administrator of the Office of
Information and Regulatory Affairs in the Office of Management and
Budget (OIRA).
The term ``agency'' has the same meaning as in the Paperwork
Reduction Act, 44 U.S.C. 3502(1).
The term ``Information Quality Act'' means Section 515 of Public
Law 106-554 (Pub. L. No. 106-554, Sec. 515, 114 Stat. 2763, 2763A-153-
154 (2000)).
The term ``dissemination'' means agency initiated or sponsored
distribution of information to the public. Dissemination does not
include distribution limited to government employees or agency
contractors or grantees; intra-or inter-agency use or sharing of
government information; or responses to requests for agency records
under the Freedom of Information Act, the Privacy Act, the Federal
Advisory Committee Act, the Government Performance and Results Act, or
similar laws. This definition also excludes distribution limited to
correspondence with individuals or persons, press releases, archival
records, public filings, subpoenas and adjudicative processes. In the
context of this Bulletin, the definition of ``dissemination'' modifies
the definition in OMB's government-wide information quality guidelines
to address the need for peer review prior to official dissemination of
the information product. Accordingly, under this Bulletin,
``dissemination'' also excludes information distributed for peer review
in compliance with this Bulletin or shared confidentially with
scientific colleagues, provided that the distributing agency includes
an appropriate and clear disclaimer on the information, as explained
more fully below. Finally, the Bulletin does not directly cover
information supplied to the government by third parties (e.g., studies
by private consultants, companies and private, non-profit
organizations, or research institutions such as universities). However,
if an agency plans to disseminate information supplied by a third party
(e.g., using this information as the basis for an agency's factual
determination that a particular behavior causes a disease), the
requirements of the Bulletin apply, if the dissemination is
``influential''.
In cases where a draft report or other information is released by
an agency solely for purposes of peer review, a question may arise as
to whether the draft report constitutes an official ``dissemination''
under information-quality guidelines. Section I instructs agencies to
make this clear by presenting the following disclaimer in the report:
This information is distributed solely for the purpose of pre-
dissemination peer review under applicable information quality
guidelines. It has not been formally disseminated by [the agency].
It does not represent and should not be construed to represent any
agency determination or policy.
In cases where the information is highly relevant to specific
policy or regulatory deliberations, this disclaimer shall appear on
each page of a draft report. Agencies also shall discourage state,
local, international and private organizations from using information
in draft reports that are undergoing peer review. Draft influential
scientific information presented at scientific meetings or shared
confidentially with colleagues for scientific input prior to peer
review shall include the disclaimer: ``The Findings and Conclusions in
This Report (Presentation) Have Not Been Formally Disseminated by [The
Agency] and Should Not Be Construed to Represent Any Agency
Determination or Policy.''
An information product is not covered by the Bulletin unless it
represents an official view of one or more departments or agencies of
the Federal government. Accordingly, for the purposes of this Bulletin,
``dissemination'' excludes research produced by government-funded
scientists (e.g., those supported extramurally or intramurally by
Federal agencies or those working in state or local governments with
Federal support) if that information is not represented as the views of
a department or agency (i.e., they are not official government
disseminations). For influential scientific information that does not
have the imprimatur of the Federal government, scientists employed by
the Federal government are required to include in their information
product a clear disclaimer that ``the findings and conclusions in this
report are those of the author(s) and do not necessarily represent the
views of the funding agency.'' A similar disclaimer is advised for non-
government employees who publish government-funded research.
For the purposes of the peer review Bulletin, the term ``scientific
information'' means factual inputs, data, models, analyses, technical
information, or scientific assessments related to such disciplines as
the behavioral and social sciences, public health and medical sciences,
life and earth sciences, engineering, or physical sciences. This
includes any communication or representation of knowledge such as facts
or data, in any medium or form, including textual, numerical, graphic,
cartographic, narrative, or audiovisual forms. This definition includes
information that an agency disseminates from a Web page, but does not
include the provision of hyperlinks on a Web page to information that
others disseminate. This definition excludes opinions, where the
agency's presentation makes clear that an individual's opinion, rather
than a statement of fact or of the agency's findings and conclusions,
is being offered.
The term ``influential scientific information'' means scientific
information the agency reasonably can determine will have or does have
a clear and substantial impact on important public policies or private
sector decisions. In the term ``influential scientific information,''
the term ``influential'' should be interpreted consistently with OMB's
government-wide information quality guidelines and the information
quality guidelines of the agency. Information dissemination can have a
significant economic impact even if it is not part of a rulemaking. For
instance, the economic viability of a technology can be influenced by
the government's characterization of its attributes. Alternatively, the
Federal government's assessment of risk can directly or indirectly
influence the response actions of state and local agencies or
international bodies.
One type of scientific information is a scientific assessment. For
the purposes of this Bulletin, the term ``scientific assessment'' means
an evaluation of a body of scientific or technical knowledge, which
typically synthesizes multiple factual inputs, data, models,
assumptions, and/or applies best professional judgment to bridge
uncertainties in the available information. These assessments include,
but are not limited to, state-of-science reports; technology
assessments; weight-of-evidence analyses; meta-analyses; health,
safety, or ecological risk assessments; toxicological characterizations
of substances; integrated assessment models; hazard determinations; or
exposure assessments. Such assessments often draw upon knowledge from
multiple disciplines. Typically, the data and models used in scientific
assessments have already been subject to some form of peer review
(e.g., refereed journal peer review or peer review under Section II of
this Bulletin).
Section II: Peer Review of Influential Scientific Information
Section II requires each agency to subject ``influential''
scientific information to peer review prior to dissemination. For
dissemination of
[[Page 2668]]
influential scientific information, Section II provides agencies broad
discretion in determining what type of peer review is appropriate and
what procedures should be employed to select appropriate reviewers.
Agencies are directed to chose a peer review mechanism that is
adequate, giving due consideration to the novelty and complexity of the
science to be reviewed, the relevance of the information to decision
making, the extent of prior peer reviews, and the expected benefits and
costs of additional review.
The National Academy of Public Administration suggests that the
intensity of peer review should be commensurate with the significance
of the information being disseminated and the likely implications for
policy decisions.\10\ Furthermore, agencies need to consider tradeoffs
between depth of peer review and timeliness.\11\ More rigorous peer
review is necessary for information that is based on novel methods or
presents complex challenges for interpretation. Furthermore, the need
for rigorous peer review is greater when the information contains
precedent-setting methods or models, presents conclusions that are
likely to change prevailing practices, or is likely to affect policy
decisions that have a significant impact.
---------------------------------------------------------------------------
\10\ National Academy of Public Administration, Setting
Priorities, Getting Results: A New Direction for EPA, National
Academy Press, Washington, DC, 1995:23.
\11\ Presidential/Congressional Commission on Risk Assessment
and Risk Management, Risk Commission Report, 1997.
---------------------------------------------------------------------------
This tradeoff can be considered in a benefit-cost framework. The
costs of peer review include both the direct costs of the peer review
activity and those stemming from potential delay in government and
private actions that can result from peer review. The benefits of peer
review are equally clear: the insights offered by peer reviewers may
lead to policy with more benefits and/or fewer costs. In addition to
contributing to strong science, peer review, if performed fairly and
rigorously, can build consensus among stakeholders and reduce the
temptation for courts and legislators to second-guess or overturn
agency actions.\12\ While it will not always be easy for agencies to
quantify the benefits and costs of peer review, agencies are encouraged
to approach peer review from a benefit-cost perspective.
---------------------------------------------------------------------------
\12\ Mark R. Powell, Science at EPA: Information in the
Regulatory Process, Resources for the Future, Washington, DC, 1999:
148, 176; Sheila Jasanoff, The Fifth Branch: Science Advisors as
Policy Makers, Harvard University Press, Boston, 1990: 242.
---------------------------------------------------------------------------
Regardless of the peer review mechanism chosen, agencies should
strive to ensure that their peer review practices are characterized by
both scientific integrity and process integrity. ``Scientific
integrity,'' in the context of peer review, refers to such issues as
``expertise and balance of the panel members; the identification of the
scientific issues and clarity of the charge to the panel; the quality,
focus and depth of the discussion of the issues by the panel; the
rationale and supportability of the panel's findings; and the accuracy
and clarity of the panel report.'' ``Process integrity'' includes such
issues as ``transparency and openness, avoidance of real or perceived
conflicts of interest, a workable process for public comment and
involvement,'' and adherence to defined procedures.\13\
---------------------------------------------------------------------------
\13\ ILSI Risk Sciences Institute, ``Policies and Procedures:
Model Peer Review Center of Excellence,'' 2002: 4. Available at
http://rsi.ilsi.org/file/Policies&Procedures.pdf.
---------------------------------------------------------------------------
When deciding what type of peer review mechanism is appropriate for
a specific information product, agencies will need to consider at least
the following issues: Individual versus panel review; timing; scope of
the review; selection of reviewers; disclosure and attribution; public
participation; disposition of reviewer comments; and adequacy of prior
peer review.
Individual Versus Panel Review
Letter reviews by several experts generally will be more
expeditious than convening a panel of experts. Individual letter
reviews are more appropriate when a draft document covers only one
discipline or when premature disclosure of a sensitive report to a
public panel could cause harm to government or private interests. When
time and resources warrant, panels are preferable, as they tend to be
more deliberative than individual letter reviews and the reviewers can
learn from each other. There are also multi-stage processes in which
confidential letter reviews are conducted prior to release of a draft
document for public notice and comment, followed by a formal panel
review. These more rigorous and expensive processes are particularly
valuable for highly complex, multidisciplinary, and more important
documents, especially those that are novel or precedent-setting.
Timing of Peer Review
As a general rule, it is most useful to consult with peers early in
the process of producing information. For example, in the context of
risk assessments, it is valuable to have the choice of input data and
the specification of the model reviewed by peers before the agency
invests time and resources in implementing the model and interpreting
the results. ``Early'' peer review occurs in time to ``focus attention
on data inadequacies in time for corrections.
When an information product is a critical component of rule-making,
it is important to obtain peer review before the agency announces its
regulatory options so that any technical corrections can be made before
the agency becomes invested in a specific approach or the positions of
interest groups have hardened. If review occurs too late, it is
unlikely to contribute to the course of a rulemaking. Furthermore,
investing in a more rigorous peer review early in the process ``may
provide net benefit by reducing the prospect of challenges to a
regulation that later may trigger time consuming and resource-draining
litigation.'' \14\
---------------------------------------------------------------------------
\14\ Fred Anderson, Mary Ann Chirba Martin, E Donald Elliott,
Cynthia Farina, Ernest Gellhorn, John D. Graham, C. Boyden Gray,
Jeffrey Holmstead, Ronald M. Levin, Lars Noah, Katherine Rhyne,
Jonathan Baert Wiener, ``Regulatory Improvement Legislation: Risk
Assessment, Cost-Benefit Analysis, and Judicial Review,'' Duke
Environmental Law and Policy Forum, Fall 2000, vol. XI (1): 132.
---------------------------------------------------------------------------
Scope of the Review
The ``charge'' contains the instructions to the peer reviewers
regarding the objective of the peer review and the specific advice
sought. The importance of the information, which shapes the goal of the
peer review, influences the charge. For instance, the goal of the
review might be to determine the utility of a body of literature for
drawing certain conclusions about the feasibility of a technology or
the safety of a product. In this context, an agency might ask reviewers
to determine the relevance of conclusions drawn in one context for
other contexts (e.g., different exposure conditions or patient
populations).
The charge to the reviewers should be determined in advance of the
selection of the reviewers. In drafting the charge, it is important to
remember the strengths and limitations of peer review. Peer review is
most powerful when the charge is specific and steers the reviewers to
specific technical questions while also directing reviewers to offer a
broad evaluation of the overall product.
Uncertainty is inherent in science, and in many cases individual
studies do not produce conclusive evidence. Thus, when an agency
generates a scientific
[[Page 2669]]
assessment, it is presenting its scientific judgment about the
accumulated evidence rather than scientific fact.\15\ Specialists
attempt to reach a consensus by weighing the accumulated evidence. Peer
reviewers can make an important contribution by distinguishing
scientific facts from professional judgments. Furthermore, where
appropriate, reviewers should be asked to provide advice on the
reasonableness of judgments made from the scientific evidence. However,
the charge should make clear that the reviewers are not to provide
advice on the policy (e.g., the amount of uncertainty that is
acceptable or the amount of precaution that should be embedded in an
analysis). Such considerations are the purview of the government.\16\
---------------------------------------------------------------------------
\15\ Mark R. Powell, Science at EPA: Information in the
Regulatory Process, Resources for the Future, Washington, DC, 1999:
139.
\16\ Ibid.
---------------------------------------------------------------------------
The charge should ask that peer reviewers ensure that scientific
uncertainties are clearly identified and characterized. Since not all
uncertainties have an equal effect on the conclusions drawn, reviewers
should be asked to ensure that the potential implications of the
uncertainties for the technical conclusions drawn are clear. In
addition, peer reviewers might be asked to consider value-of-
information analyses that identify whether more research is likely to
decrease key uncertainties.\17\ Value-of-information analysis was
suggested for this purpose in the report of the Presidential/
Congressional Commission on Risk Assessment and Risk Management.\18\ A
description of additional research that would appreciably influence the
conclusions of the assessment can help an agency assess and target
subsequent efforts.
---------------------------------------------------------------------------
\17\ Granger Morgan and Max Henrion, ``The Value of Knowing How
Little You Know,'' Uncertainty: A Guide to Dealing with Uncertainty
in Quantitative Risk and Policy Analysis, Cambridge University
Press, 1990: 307.
\18\ Presidential/Congressional Commission on Risk Assessment
and Risk Management, Risk Commission Report, 1997, Volume 1: 39,
Volume 2: 91.
---------------------------------------------------------------------------
Selection of Reviewers
Expertise. The most important factor in selecting reviewers is
expertise: ensuring that the selected reviewer has the knowledge,
experience, and skills necessary to perform the review. Agencies shall
ensure that, in cases where the document being reviewed spans a variety
of scientific disciplines or areas of technical expertise, reviewers
who represent the necessary spectrum of knowledge are chosen. For
instance, expertise in applied mathematics and statistics is essential
in the review of models, thereby allowing an audit of calculations and
claims of significance and robustness based on the numeric data.\19\
For some reviews, evaluation of biological plausibility is as important
as statistical modeling. Agencies shall consider requesting that the
public, including scientific and professional societies, nominate
potential reviewers.
---------------------------------------------------------------------------
\19\ William W. Lowrance, Modern Science and Human Values,
Oxford University Press, New York, NY 1985: 86.
---------------------------------------------------------------------------
Balance. While expertise is the primary consideration, reviewers
should also be selected to represent a diversity of scientific
perspectives relevant to the subject. On most controversial issues,
there exists a range of respected scientific viewpoints regarding
interpretation of the available literature. Inviting reviewers with
competing views on the science may lead to a sharper, more focused peer
review. Indeed, as a final layer of review, some organizations (e.g.,
the National Academy of Sciences) specifically recruit reviewers with
strong opinions to test the scientific strength and balance of their
reports. The NAS policy on committee composition and balance \20\
highlights important considerations associated with perspective, bias,
and objectivity.
---------------------------------------------------------------------------
\20\ National Academy of Sciences, ``Policy and Procedures on
Committee Composition and Balance and Conflicts of Interest for
Committees Used in the Development of Reports,'' May 2003: Available
at: http://www.nationalacademies.org/coi/index.html.
---------------------------------------------------------------------------
Independence. In its narrowest sense, independence in a reviewer
means that the reviewer was not involved in producing the draft
document to be reviewed. However, for peer review of some documents, a
broader view of independence is necessary to assure credibility of the
process. Reviewers are generally not employed by the agency or office
producing the document. As the National Academy of Sciences has stated,
``external experts often can be more open, frank, and challenging to
the status quo than internal reviewers, who may feel constrained by
organizational concerns.'' \21\ The Carnegie Commission on Science,
Technology, and Government notes that ``external science advisory
boards serve a critically important function in providing regulatory
agencies with expert advice on a range of issues.'' \22\ However, the
choice of reviewers requires a case-by-case analysis. Reviewers
employed by other Federal and state agencies may possess unique or
indispensable expertise.
---------------------------------------------------------------------------
\21\ National Research Council, Peer Review in Environmental
Technology Development Programs: The Department of Energy's Office
of Science and Technology, National Academy Press, Washington, DC,
1998: 3.
\22\ Carnegie Commission on Science, Technology, and Government,
Risk and the Environment: Improving Regulatory Decision Making,
Carnegie Commission, New York, 1993: 90.
---------------------------------------------------------------------------
A related issue is whether government-funded scientists in
universities and consulting firms have sufficient independence from the
federal agencies that support their work to be appropriate peer
reviewers for those agencies.\23\ This concern can be mitigated in
situations where the scientist initiates the hypothesis to be tested or
the method to be developed, which effectively creates a buffer between
the scientist and the agency. When an agency awards grants through a
competitive process that includes peer review, the agency's potential
to influence the scientist's research is limited. As such, when a
scientist is awarded a government research grant through an
investigator-initiated, peer-reviewed competition, there generally
should be no question as to that scientist's ability to offer
independent scientific advice to the agency on other projects. This
contrasts, for example, to a situation in which a scientist has a
consulting or contractual arrangement with the agency or office
sponsoring a peer review. Likewise, when the agency and a researcher
work together (e.g., through a cooperative agreement) to design or
implement a study, there is less independence from the agency.
Furthermore, if a scientist has repeatedly served as a reviewer for the
same agency, some may question whether that scientist is sufficiently
independent from the agency to be employed as a peer reviewer on
agency-sponsored projects.
---------------------------------------------------------------------------
\23\ Lars Noah, ``Scientific `Republicanism': Expert Peer Review
and the Quest for Regulatory Deliberation, Emory Law Journal,
Atlanta, Fall 2000:1066.
---------------------------------------------------------------------------
As the foregoing suggests, independence poses a complex set of
questions that must be considered by agencies when peer reviewers are
selected. In general, agencies shall make an effort to rotate peer
review responsibilities across the available pool of qualified
reviewers, recognizing that in some cases repeated service by the same
reviewer is needed because of essential expertise.
Some agencies have built entire organizations to provide
independent scientific advice while other agencies tend to employ ad
hoc scientific panels on specific issues. Respect for the independence
of reviewers may be enhanced if an agency collects names of potential
reviewers (based on considerations of expertise and reputation for
objectivity) from the
[[Page 2670]]
public, including scientific or professional societies. The Department
of Energy's use of the American Society of Mechanical Engineers to
identify potential peer reviewers from a variety of different
scientific societies provides an example of how professional societies
can assist in the development of an independent peer review panel.\24\
---------------------------------------------------------------------------
\24\ American Society for Mechanical Engineers, Assessment of
Technologies Supported by the Office of Science and Technology,
Department of Energy: Results of the Peer Review for Fiscal Year
2002, ASME Technical Publishing, Danvers, MA, 2003.
---------------------------------------------------------------------------
Conflict of Interest. The National Academy of Sciences defines
``conflict of interest'' as any financial or other interest that
conflicts with the service of an individual on the review panel because
it could impair the individual's objectivity or could create an unfair
competitive advantage for a person or organization.\25\ This standard
provides a useful benchmark for agencies to consider in selecting peer
reviewers. Agencies shall make a special effort to examine prospective
reviewers' potential financial conflicts, including significant
investments, consulting arrangements, employer affiliations and grants/
contracts. Financial ties of potential reviewers to regulated entities
(e.g., businesses), other stakeholders, and regulatory agencies shall
be scrutinized when the information being reviewed is likely to be
relevant to regulatory policy. The inquiry into potential conflicts
goes beyond financial investments and business relationships and
includes work as an expert witness, consulting arrangements, honoraria
and sources of grants and contracts. To evaluate any real or perceived
conflicts of interest with potential reviewers and questions regarding
the independence of reviewers, agencies are referred to federal ethics
requirements, applicable standards issued by the Office of Government
Ethics, and the prevailing practices of the National Academy of
Sciences. Specifically, peer reviewers who are Federal employees
(including special government employees) are subject to Federal
requirements governing conflicts of interest. See, e.g., 18 U.S.C. 208;
5 CFR part 2635 (2004). With respect to reviewers who are not Federal
employees, agencies shall adopt or adapt the NAS policy for committee
selection with respect to evaluating conflicts of interest.\26\ Both
the NAS and the Federal government recognize that under certain
circumstances some conflict may be unavoidable in order to obtain the
necessary expertise. See, e.g., 18 U.S.C. 208(b)(3); 5 U.S.C. App. 15
(governing NAS committees). To improve the transparency of the process,
when an agency determines that it is necessary to use a reviewer with a
real or perceived conflict of interest, the agency should consider
publicly disclosing those conflicts. In such situations, the agency
shall inform potential reviewers of such disclosure at the time they
are recruited.
---------------------------------------------------------------------------
\25\ National Academy of Sciences, ``Policy and Procedures on
Committee Composition and Balance and Conflicts of Interest for
Committees Used in the Development of Reports,'' May 2003: Available
at: http://www.nationalacademies.org/coi/index.html.
\26\ Ibid.
---------------------------------------------------------------------------
Disclosure and Attribution: Anonymous Versus Identified
Peer reviewers must have a clear understanding of how their
comments will be conveyed to the authors of the document and to the
public. When peer review of government reports is considered, the case
for transparency is stronger, particularly when the report addresses an
issue with significant ramifications for the public and private
sectors. The public may not have confidence in the peer review process
when the names and affiliations of the peer reviewers are unknown.
Without access to the comments of reviewers, the public is incapable of
determining whether the government has seriously considered the
comments of reviewers and made appropriate revisions. Disclosure of the
slate of reviewers and the substance of their comments can strengthen
public confidence in the peer review process. It is common at many
journals and research funding agencies to disclose annually the slate
of reviewers. Moreover, the National Academy of Sciences now discloses
the names of its peer reviewers, without disclosing the substance of
their comments. The science advisory committees to regulatory agencies
typically disclose at least a summary of the comments of reviewers as
well as their names and affiliations.
For agency-sponsored peer review conducted under Sections II and
III, this Bulletin strikes a compromise by requiring disclosure of the
identity of the reviewers, but not public attribution of specific
comments to specific reviewers. The agency has considerable discretion
in the implementation of this compromise (e.g., summarizing the views
of reviewers as a group or disclosing individual reviewer comments
without attribution). Whatever approach is employed, the agency must
inform reviewers in advance of how it intends to address this issue.
Information about a reviewer retrieved from a record filed by the
reviewer's name or other identifier may be disclosed only as permitted
by the conditions of disclosure enumerated in the Privacy Act, 5 U.S.C.
552a as amended, and as interpreted in OMB implementing guidance, 40 FR
28,948 (July 9, 1975).
Public Participation
Public comments can be important in shaping expert deliberations.
Agencies may decide that peer review should precede an opportunity for
public comment to ensure that the public receives the most
scientifically strong product (rather than one that may change
substantially as a result of peer reviewer suggestions). However, there
are situations in which public participation in peer review is an
important aspect of obtaining a high-quality product through a credible
process. Agencies, however, should avoid open-ended comment periods,
which may delay completion of peer reviews and complicate the
completion of the final work product.
Public participation can take a variety of forms, including
opportunities to provide oral comments before a peer review panel or
requests to provide written comments to the peer reviewers. Another
option is for agencies to publish a ``request for comment'' or other
notice in which they solicit public comment before a panel of peer
reviewers performs its work.
Disposition of Reviewer Comments
A peer review is considered completed once the agency considers and
addresses the reviewers' comments. All reviewer comments should be
given consideration and be incorporated where relevant and valid. For
instance, in the context of risk assessments, the National Academy of
Sciences recommends that peer review include a written evaluation made
available for public inspection.\27\ In cases where there is a public
panel, the agency should plan publication of the peer review report(s)
and the agency's response to peer reviewer comments.
---------------------------------------------------------------------------
\27\ National Research Council, Risk Assessment in the Federal
Government: Managing the Process, National Academy Press,
Washington, DC, 1983.
---------------------------------------------------------------------------
In addition, the credibility of the final scientific report is
likely to be enhanced if the public understands how the agency
addressed the specific concerns raised by the peer reviewers.
Accordingly, agencies should consider preparing a written response to
the peer review report explaining: The agency's agreement or
disagreement, the actions the agency has undertaken or will undertake
in response to the report, and (if applicable) the reasons the agency
believes those actions satisfy any key
[[Page 2671]]
concerns or recommendations in the report.
Adequacy of Prior Peer Review
In light of the broad range of information covered by Section II,
agencies are directed to choose a peer review mechanism that is
adequate, giving due consideration to the novelty and complexity of the
science to be reviewed, the relevance of the information to decision
making, the extent of prior peer reviews, and the expected benefits and
costs of additional review.
Publication in a refereed scientific journal may mean that adequate
peer review has been performed. However, the intensity of peer review
is highly variable across journals. There will be cases in which an
agency determines that a more rigorous or transparent review process is
necessary. For instance, an agency may determine a particular journal
review process did not address questions (e.g., the extent of
uncertainty inherent in a finding) that the agency determines should be
addressed before disseminating that information. As such, prior peer
review and publication is not by itself sufficient grounds for
determining that no further review is necessary.
Section III: Peer Review of Highly Influential Scientific Assessments
Whereas Section II leaves most of the considerations regarding the
form of the peer review to the agency's discretion, Section III
requires a more rigorous form of peer review for highly influential
scientific assessments. The requirements of Section II of this Bulletin
apply to Section III, but Section III has some additional requirements,
which are discussed below. In planning a peer review under Section III,
agencies typically will have to devote greater resources and attention
to the issues discussed in Section II, i.e., individual versus panel
review; timing; scope of the review; selection of reviewers; disclosure
and attribution; public participation; and disposition of reviewer
comments.
A scientific assessment is considered ``highly influential'' if the
agency or the OIRA Administrator determines that the dissemination
could have a potential impact of more than $500 million in any one year
on either the public or private sector or that the dissemination is
novel, controversial, or precedent-setting, or has significant
interagency interest. One of the ways information can exert economic
impact is through the costs or benefits of a regulation based on the
disseminated information. The qualitative aspect of this definition may
be most useful in cases where it is difficult for an agency to predict
the potential economic effect of dissemination. In the context of this
Bulletin, it may be either the approach used in the assessment or the
interpretation of the information itself that is novel or precedent-
setting. Peer review can be valuable in establishing the bounds of the
scientific debate when methods or interpretations are a source of
controversy among interested parties. If information is covered by
Section III, an agency is required to adhere to the peer review
procedures specified in Section III.
Section III(2) clarifies that the principal findings, conclusions
and recommendations in official reports of the National Academy of
Sciences that fall under this Section are generally presumed not to
require additional peer review. All other highly influential scientific
assessments require a review that meets the requirements of Section III
of this Bulletin.
With regard to the selection of reviewers, Section III(3)(a)
emphasizes consideration of expertise and balance. As discussed in
Section II, expertise refers to the required knowledge, experience and
skills required to perform the review whereas balance refers to the
need for diversity in scientific perspective and disciplines. We
emphasize that the term ``balance'' here refers not to balancing of
stakeholder or political interests but rather to a broad and diverse
representation of respected perspectives and intellectual traditions
within the scientific community, as discussed in the NAS policy on
committee composition and balance.\28\
---------------------------------------------------------------------------
\28\ National Academy of Sciences, ``Policy and Procedures on
Committee Composition and Balance and Conflicts of Interest for
Committees Used in the Development of Reports,'' May 2003: Available
at: http://www.nationalacademies.org/coi/index.html.
---------------------------------------------------------------------------
Section III(3)(b) instructs agencies to consider barring
participation by scientists with a conflict of interest. The conflict
of interest standards for Sections II and III of the Bulletin are
identical. As discussed under Section II, those peer reviewers who are
Federal employees, including Special Government Employees, are subject
to applicable statutory and regulatory standards for Federal employees.
For non-government employees, agencies shall adopt or adapt the NAS
policy for committee member selection with respect to evaluating
conflicts of interest.
Section III(3)(c) instructs agencies to ensure that reviewers are
independent of the agency sponsoring the review. Scientists employed by
the sponsoring agency are not permitted to serve as reviewers for
highly influential scientific assessments. This does not preclude
Special Government Employees, such as academics appointed to advisory
committees, from serving as peer reviewers. The only exception to this
ban would be the rare situation in which a scientist from a different
agency of a Cabinet-level department than the agency that is
disseminating the scientific assessment has expertise, experience and
skills that are essential but cannot be obtained elsewhere. In
evaluating the need for this exception, agencies shall use the NAS
criteria for assessing the appropriateness of using employees of
sponsors (e.g., the government scientist must not have had any part in
the development or prior review of the scientific information and must
not hold a position of managerial or policy responsibility).
We also considered whether a reviewer can be independent of the
agency if that reviewer receives a substantial amount of research
funding from the agency sponsoring the review. Research grants that
were awarded to the scientist based on investigator-initiated,
competitive, peer-reviewed proposals do not generally raise issues of
independence. However, significant consulting and contractual
relationships with the agency may raise issues of independence or
conflict, depending upon the situation.
Section III(3)(d) addresses concerns regarding repeated use of the
same reviewer in multiple assessments. Such repeated use should be
avoided unless a particular reviewer's expertise is essential. Agencies
should rotate membership across the available pool of qualified
reviewers. Similarly, when using standing panels of scientific
advisors, it is suggested that the agency rotate membership among
qualified scientists in order to obtain fresh perspectives and
reinforce the reality and perception of independence from the agency.
Section III(4) requires agencies to provide reviewers with
sufficient background information, including access to key studies,
data and models, to perform their role as peer reviewers. In this
respect, the peer review envisioned in Section III is more rigorous
than some forms of journal peer review, where the reviewer is often not
provided access to underlying data or models. Reviewers shall be
informed of applicable access, objectivity, reproducibility and other
quality standards under Federal information quality laws.
[[Page 2672]]
Section III(5) addresses opportunity for public participation in
peer review, and provides that the agency shall, wherever possible,
provide for public participation. In some cases, an assessment may be
so sensitive that it is critical that the agency's assessment achieve a
high level of quality before it is publicized. In those situations, a
rigorous yet confidential peer review process may be appropriate, prior
to public release of the assessment. If an agency decides to make a
draft assessment publicly available at the onset of a peer review
process, the agency shall, whenever possible, provide a vehicle for the
public to provide written comments, make an oral presentation before
the peer reviewers, or both. When written public comments are received,
the agency shall ensure that peer reviewers receive copies of comments
that address significant scientific issues with ample time to consider
them in their review. To avoid undue delay of agency activities, the
agency shall specify time limits for public participation throughout
the peer review process.
Section III(6) requires that agencies instruct reviewers to prepare
a peer review report that describes the nature and scope of their
review and their findings and conclusions. The report shall disclose
the name of each peer reviewer and a brief description of his or her
organizational affiliation, credentials and relevant experiences. The
peer review report should either summarize the views of the group as a
whole (including any dissenting views) or include a verbatim copy of
the comments of the individual reviewers (with or without attribution
of specific views to specific names). The agency shall also prepare a
written response to the peer review report, indicating whether the
agency agrees with the reviewers and what actions the agency has taken
or plans to take to address the points made by reviewers. The agency is
required to disseminate the peer review report and the agency's
response to the report on the agency's Web site, including all the
materials related to the peer review such as the charge statement, peer
review report, and agency response to the review. If the scientific
information is used to support a final rule then, where practicable,
the peer review report shall be made available to the public with
enough time for the public to consider the implications of the peer
review report for the rule being considered.
Section III(7) authorizes but does not require an agency to
commission an entity independent of the agency to select peer reviewers
and/or manage the peer review process in accordance with this Bulletin.
The entity may be a scientific or professional society, a firm
specializing in peer review, or a non-profit organization with
experience in peer review.
Section IV: Alternative Procedures
Peer review as described in this Bulletin is only one of many
procedures that agencies can employ to ensure an appropriate degree of
pre-dissemination quality of influential scientific information. For
example, Congress has assigned the NAS a special role in advising the
Federal government on scientific and technical issues. The procedures
of the NAS are generally quite rigorous, and thus agencies should
presume that major findings, conclusions, and recommendations of NAS
reports meet the performance standards of this Bulletin.
As an alternative to complying with Sections II and III of this
Bulletin, an agency may instead (1) rely on scientific information
produced by the National Academy of Sciences, (2) commission the
National Academy of Sciences to peer review an agency draft scientific
information product, or (3) employ an alternative procedure or set of
procedures, specifically approved by the OIRA Administrator in
consultation with the Office of Science and Technology Policy (OSTP),
that ensures that the scientific information product meets applicable
information-quality standards.
An example of an alternative procedure is to commission a respected
third party other than the NAS (e.g., the Health Effects Institute or
the National Commission on Radiation Protection and Measurement) to
conduct an assessment or series of related assessments. Another example
of an alternative set of procedures is the three-part process used by
the National Institutes of Health (NIH) to generate scientific
guidance. Under that process, a scientific proposal or white paper is
generated by a working group composed of external, independent
scientific experts; that paper is then forwarded to a separate external
scientific council, which then makes recommendations to the agency. The
agency, in turn, decides whether to adopt and/or modify the proposal.
For large science agencies that have diverse research portfolios and do
not have significant regulatory responsibilities, such as NIH, an
acceptable alternative would be to allow scientists from one part of
the agency (for example, an NIH institute) to participate in the review
of documents prepared by another part of the agency, as long as the
head of the agency confirms in writing that each of the reviewers meets
the NAS criteria relating to the appropriateness of using employees of
sponsors (e.g., the government scientist must not have had any part in
the development or prior review of the scientific information and must
not hold a position of managerial or policy responsibility). The
purpose of Section IV is to encourage these types of innovation in the
methods used to ensure pre-dissemination quality control of influential
scientific information.
The mere existence of a public comment process (e.g., notice-and-
comment procedures under the Administrative Procedure Act) does not
constitute adequate peer review or an ``alternative process,'' because
it does not assure that qualified, impartial specialists in relevant
fields have performed a critical evaluation of the agency's draft
product.\29\
---------------------------------------------------------------------------
\29\ William W. Lowrance, Modern Science and Human Values,
Oxford University Press, New York, NY 1985: 86.
---------------------------------------------------------------------------
Section V: Peer Review Planning
Section V requires agencies to begin a systematic process of peer
review planning for influential scientific information (including
highly influential scientific assessments) that the agency plans to
disseminate in the foreseeable future. A key feature of this planning
process is a Web-accessible listing of forthcoming influential
scientific disseminations (i.e., an agenda) that is regularly updated
by the agency. By making these plans publicly available, agencies will
be able to gauge the extent of public interest in the peer review
process for influential scientific information, including highly
influential scientific assessments. These Web-accessible agendas can
also be used by the public to monitor agency compliance with this
Bulletin.
Each entry on the agenda shall include a preliminary title of the
planned report, a short paragraph describing the subject and purpose of
the planned report, and an agency contact person. The agency shall
provide its prediction regarding whether the dissemination will be
``influential scientific information'' or a ``highly influential
scientific assessment,'' as the designation can influence the type of
peer review to be undertaken. The agency shall discuss the timing of
the peer review, as well as the use of any deferrals. Agencies shall
include entries in the agenda for influential scientific information,
including highly influential scientific assessments, for which the
Bulletin's requirements have
[[Page 2673]]
been deferred or waived. If the agency, in consultation with the OIRA
Administrator, has determined that it is appropriate to use a Section
IV ``alternative procedure'' for a specific dissemination, a
description of that alternative procedure shall be included in the
agenda.
Furthermore, for each entry on the agenda, the agency shall
describe the peer review plan. Each peer review plan shall include: (i)
A paragraph including the title, subject and purpose of the planned
report, as well as an agency contact to whom inquiries may be directed
to learn the specifics of the plan; (ii) whether the dissemination is
likely to be influential scientific information or a highly influential
scientific assessment; (iii) the timing of the review (including
deferrals); (iv) whether the review will be conducted through a panel
or individual letters (or whether an alternative procedure will be
exercised); (v) whether there will be opportunities for the public to
comment on the work product to be peer reviewed, and if so, how and
when these opportunities will be provided; (vi) whether the agency will
provide significant and relevant public comments to the peer reviewers
before they conduct their review; (vii) the anticipated number of
reviewers (3 or fewer; 4-10; or more than 10); (viii) a succinct
description of the primary disciplines or expertise needed in the
review; (ix) whether reviewers will be selected by the agency or by a
designated outside organization; and (x) whether the public, including
scientific or professional societies, will be asked to nominate
potential peer reviewers. The agency shall provide a link from the
agenda to each document made public pursuant to this Bulletin. Agencies
shall link their peer review agendas to the U.S. Government's official
Web portal: firstgov at http://www.FirstGov.gov.
Agencies should update their peer review agendas at least every six
months. However, in some cases--particularly for highly influential
scientific assessments and other particularly important information--
more frequent updates of existing entries on the agenda, or the
addition of new entries to the agenda, may be warranted. When new
entries are added to the agenda of forthcoming reports and other
information, the public should be provided with sufficient time to
comment on the agency's peer review plan for that report or product.
Agencies shall consider public comments on the peer review plan.
Agencies are encouraged to offer a listserve or similar mechanism for
members of the public who would like to be notified by email each time
an agency's peer review agenda has been updated.
The peer review planning requirements of this Bulletin are designed
to be implemented in phases. Specifically, the planning requirements of
the Bulletin will go into effect for documents subject to Section III
of the Bulletin (highly influential scientific assessments) six months
after publication. However, the planning requirements for documents
subject to Section II of the Bulletin do not go into effect until one
year after publication. It is expected that agency experience with the
planning requirements of the Bulletin for the smaller scope of
documents encompassed in Section III will be used to inform
implementation of these planning requirements for the larger scope of
documents covered under Section II.
Section VI: Annual Report
Each agency shall prepare an annual report that summarizes key
decisions made pursuant to this Bulletin. In particular, each agency
should provide to OIRA the following: (1) The number of peer reviews
conducted subject to the Bulletin (i.e., for influential scientific
information and highly influential scientific assessments); (2) the
number of times alternative procedures were invoked; (3) the number of
times waivers or deferrals were invoked (and in the case of deferrals,
the length of time elapsed between the deferral and the peer review);
(4) any decision to appoint a reviewer pursuant to any exception to the
applicable independence or conflict of interest standards of the
Bulletin, including determinations by the Secretary or Deputy Secretary
pursuant to Section III(3)(c); (5) the number of peer review panels
that were conducted in public and the number that allowed public
comment; (6) the number of public comments provided on the agency's
peer review plans; and (7) the number of peer reviewers that the agency
used that were recommended by professional societies.
Section VII: Certification in the Administrative Record
If an agency relies on influential scientific information or a
highly influential scientific assessment subject to the requirements of
this Bulletin in support of a regulatory action, the agency shall
include in the administrative record for that action a certification
that explains how the agency has complied with the requirements of this
Bulletin and the Information Quality Act. Relevant materials are to be
placed in the administrative record.
Section VIII: Safeguards, Deferrals, and Waivers
Section VIII recognizes that individuals serving as peer reviewers
have a privacy interest in information about themselves that the
government maintains and retrieves by name or identifier from a system
of records. To the extent information about a reviewer (name,
credential, affiliation) will be disclosed along with his/her comments
or analysis, the agency must comply with the requirements of the
Privacy Act, 5 U.S.C. 552a, as amended, and OMB Circular A-130,
Appendix I, 61 FR 6428 (February 20, 1996) to establish appropriate
routine uses in a published System of Records Notice. Furthermore, the
peer review must be conducted in a manner that respects confidential
business information as well as intellectual property.
Section VIII also allows for a deferral or waiver of the
requirements of the Bulletin where necessary. Specifically, the agency
head may waive or defer some or all of the peer review requirements of
Sections II or III of this Bulletin if there is a compelling rationale
for waiver or deferral. Waivers will seldom be warranted under this
provision because the Bulletin already provides significant safety
valves, such as: The exemptions provided in Section IX, including the
exemption for time-sensitive health and safety information; the
authorization for alternative procedures in Section IV; and the overall
flexibility provided for peer reviews of influential scientific
information under Section II. Nonetheless, we have included this waiver
and deferral provision to ensure needed flexibility in unusual and
compelling situations not otherwise covered by the exemptions to the
Bulletin, such as situations where unavoidable legal deadlines prevent
full compliance with the Bulletin before information is disseminated.
Deadlines found in consent decrees agreed to by agencies after the
Bulletin is issued will not ordinarily warrant waiver of the Bulletin's
requirements because those deadlines should be negotiated to permit
time for all required procedures, including peer review. In addition,
when an agency is unavoidably up against a deadline, deferral of some
or all requirements of the Bulletin (as opposed to outright waiver of
all of them) is the most appropriate accommodation between the need to
satisfy immovable deadlines and the need to undertake proper peer
review. If the agency head defers any of the peer
[[Page 2674]]
review requirements prior to dissemination, peer review should be
conducted as soon as practicable thereafter.
Section IX: Exemptions
There are a variety of situations where agencies need not conduct
peer review under this Bulletin. These include, for example,
disseminations of sensitive information related to certain national
security, foreign affairs, or negotiations involving international
treaties and trade where compliance with this Bulletin would interfere
with the need for secrecy or promptness.
This Bulletin does not cover official disseminations that arise in
adjudications and permit proceedings, unless the agency determines that
peer review is practical and appropriate and that the influential
dissemination is scientifically or technically novel (i.e., a major
change in accepted practice) or likely to have precedent-setting
influence on future adjudications or permit proceedings. This exclusion
is intended to cover, among other things, licensing, approval and
registration processes for specific product development activities as
well as site-specific activities. The determination as to whether peer
review is practical and appropriate is left to the discretion of the
agency. While this Bulletin is not broadly applicable to adjudications,
agencies are encouraged to hold peer reviews of scientific assessments
supporting adjudications to the same technical standards as peer
reviews covered by the Bulletin, including transparency and disclosure
of the data and models underlying the assessments. Protections apply to
confidential business information.
The Bulletin does not cover time-sensitive health and safety
disseminations, for example, a dissemination based primarily on data
from a recent clinical trial that was adequately peer reviewed before
the trial began. For this purpose, ``health'' includes public health,
or plant or animal infectious diseases.
This Bulletin covers original data and formal analytic models used
by agencies in Regulatory Impact Analyses (RIAs). However, the RIA
documents themselves are already reviewed through an interagency review
process under E.O. 12866 that involves application of the principles
and methods defined in OMB Circular A-4. In that respect, RIAs are
excluded from coverage by this Bulletin, although agencies are
encouraged to have RIAs reviewed by peers within the government for
adequacy and completeness.
The Bulletin does not cover accounting, budget, actuarial, and
financial information including that which is generated or used by
agencies that focus on interest rates, banking, currency, securities,
commodities, futures, or taxes.
Routine statistical information released by Federal statistical
agencies (e.g., periodic demographic and economic statistics) and
analyses of these data to compute standard indicators and trends (e.g.,
unemployment and poverty rates) is excluded from this Bulletin.
The Bulletin does not cover information disseminated in connection
with routine rules that materially alter entitlements, grants, user
fees, or loan programs, or the rights and obligations of recipients
thereof.
If information is disseminated pursuant to an exemption to this
Bulletin, subsequent disseminations are not automatically exempted. For
example, if influential scientific information is first disseminated in
the course of an exempt agency adjudication, but is later disseminated
in the context of a non-exempt rulemaking, the subsequent dissemination
will be subject to the requirements of this Bulletin even though the
first dissemination was not.
Section X: OIRA and OSTP Responsibilities
OIRA, in consultation with OSTP, is responsible for overseeing
agency implementation of this Bulletin. In order to foster learning
about peer review practices across agencies, OIRA and OSTP shall form
an interagency workgroup on peer review that meets regularly, discusses
progress and challenges, and recommends improvements to peer review
practices.
Section XI: Effective Date and Existing Law
The requirements of this Bulletin, with the exception of Section V,
apply to information disseminated on or after six months after
publication of this Bulletin. However, the Bulletin does not apply to
information that is already being addressed by an agency-initiated peer
review process (e.g., a draft is already being reviewed by a formal
scientific advisory committee established by the agency). An existing
peer review mechanism mandated by law should be implemented by the
agency in a manner as consistent as possible with the practices and
procedures outlined in this Bulletin. The requirements of Section V
apply to ``highly influential scientific assessments,'' as designated
in Section III of the Bulletin, within six months of publication of the
final Bulletin. The requirements in Section V apply to documents
subject to Section II of the Bulletin one year after publication of the
final Bulletin.
Section XII: Judicial Review
This Bulletin is intended to improve the internal management of the
Executive Branch and is not intended to, and does not, create any right
or benefit, substantive or procedural, enforceable at law or in equity,
against the United States, its agencies or other entities, its officers
or employees, or any other person.
Bulletin for Peer Review
I. Definitions
For purposes of this Bulletin--
1. The term ``Administrator'' means the Administrator of the Office
of Information and Regulatory Affairs in the Office of Management and
Budget (OIRA);
2. The term ``agency'' has the same meaning as in the Paperwork
Reduction Act, 44 U.S.C. 3502(1);
3. The term ``dissemination'' means agency initiated or sponsored
distribution of information to the public (see 5 CFR 1320.3(d)
(definition of ``Conduct or Sponsor'')). Dissemination does not include
distribution limited to government employees or agency contractors or
grantees; intra- or inter-agency use or sharing of government
information; or responses to requests for agency records under the
Freedom of Information Act, the Privacy Act, the Federal Advisory
Committee Act, the Government Performance and Results Act or similar
law. This definition also excludes distribution limited to
correspondence with individuals or persons, press releases, archival
records, public filings, subpoenas and adjudicative processes. The term
``dissemination'' also excludes information distributed for peer review
in compliance with this Bulletin, provided that the distributing agency
includes a clear disclaimer on the information as follows: ``This
information is distributed solely for the purpose of pre-dissemination
peer review under applicable information quality guidelines. It has not
been formally disseminated by [the agency]. It does not represent and
should not be construed to represent any agency determination or
policy.'' For the purposes of this Bulletin, ``dissemination'' excludes
research produced by government-funded scientists (e.g., those
supported extramurally or intramurally by Federal
[[Page 2675]]
agencies or those working in state or local governments with Federal
support) if that information does not represent the views of an agency.
To qualify for this exemption, the information should display a clear
disclaimer that ``the findings and conclusions in this report are those
of the author(s) and do not necessarily represent the views of the
funding agency'';
4. The term ``Information Quality Act'' means Section 515 of Public
Law 106-554 (Pub. L. No. 106-554, Sec. 515, 114 Stat. 2763, 2763A-153-
154 (2000));
5. The term ``scientific information'' means factual inputs, data,
models, analyses, technical information, or scientific assessments
based on the behavioral and social sciences, public health and medical
sciences, life and earth sciences, engineering, or physical sciences.
This includes any communication or representation of knowledge such as
facts or data, in any medium or form, including textual, numerical,
graphic, cartographic, narrative, or audiovisual forms. This definition
includes information that an agency disseminates from a Web page, but
does not include the provision of hyperlinks to information that others
disseminate. This definition does not include opinions, where the
agency's presentation makes clear that what is being offered is
someone's opinion rather than fact or the agency's views;
6. The term ``influential scientific information'' means scientific
information the agency reasonably can determine will have or does have
a clear and substantial impact on important public policies or private
sector decisions; and
7. The term ``scientific assessment'' means an evaluation of a body
of scientific or technical knowledge, which typically synthesizes
multiple factual inputs, data, models, assumptions, and/or applies best
professional judgment to bridge uncertainties in the available
information. These assessments include, but are not limited to, state-
of-science reports; technology assessments; weight-of-evidence
analyses; meta-analyses; health, safety, or ecological risk
assessments; toxicological characterizations of substances; integrated
assessment models; hazard determinations; or exposure assessments.
II. Peer Review of Influential Scientific Information
1. In General: To the extent permitted by law, each agency shall
conduct a peer review on all influential scientific information that
the agency intends to disseminate. Peer reviewers shall be charged with
reviewing scientific and technical matters, leaving policy
determinations for the agency. Reviewers shall be informed of
applicable access, objectivity, reproducibility and other quality
standards under the Federal laws governing information access and
quality.
2. Adequacy of Prior Peer Review: For information subject to this
section of the Bulletin, agencies need not have further peer review
conducted on information that has already been subjected to adequate
peer review. In determining whether prior peer review is adequate,
agencies shall give due consideration to the novelty and complexity of
the science to be reviewed, the importance of the information to
decision making, the extent of prior peer reviews, and the expected
benefits and costs of additional review. Principal findings,
conclusions and recommendations in official reports of the National
Academy of Sciences are generally presumed to have been adequately peer
reviewed.
3. Selection of Reviewers: a. Expertise and Balance: Peer reviewers
shall be selected based on expertise, experience and skills, including
specialists from multiple disciplines, as necessary. The group of
reviewers shall be sufficiently broad and diverse to fairly represent
the relevant scientific and technical perspectives and fields of
knowledge. Agencies shall consider requesting that the public,
including scientific and professional societies, nominate potential
reviewers.
b. Conflicts: The agency--or the entity selecting the peer
reviewers--shall (i) ensure that those reviewers serving as federal
employees (including special government employees) comply with
applicable Federal ethics requirements; (ii) in selecting peer
reviewers who are not government employees, adopt or adapt the National
Academy of Sciences policy for committee selection with respect to
evaluating the potential for conflicts (e.g., those arising from
investments; agency, employer, and business affiliations; grants,
contracts and consulting income). For scientific information relevant
to specific regulations, the agency shall examine a reviewer's
financial ties to regulated entities (e.g., businesses), other
stakeholders, and the agency.
c. Independence: Peer reviewers shall not have participated in
development of the work product. Agencies are encouraged to rotate
membership on standing panels across the pool of qualified reviewers.
Research grants that were awarded to scientists based on investigator-
initiated, competitive, peer-reviewed proposals generally do not raise
issues as to independence or conflicts.
4. Choice of Peer Review Mechanism: The choice of a peer review
mechanism (for example, letter reviews or ad hoc panels) for
influential scientific information shall be based on the novelty and
complexity of the information to be reviewed, the importance of the
information to decision making, the extent of prior peer review, and
the expected benefits and costs of review, as well as the factors
regarding transparency described in II(5).
5. Transparency: The agency--or entity managing the peer review--
shall instruct peer reviewers to prepare a report that describes the
nature of their review and their findings and conclusions. The peer
review report shall either (a) include a verbatim copy of each
reviewer's comments (either with or without specific attributions) or
(b) represent the views of the group as a whole, including any
disparate and dissenting views. The agency shall disclose the names of
the reviewers and their organizational affiliations in the report.
Reviewers shall be notified in advance regarding the extent of
disclosure and attribution planned by the agency. The agency shall
disseminate the final peer review report on the agency's Web site along
with all materials related to the peer review (any charge statement,
the peer review report, and any agency response). The peer review
report shall be discussed in the preamble to any related rulemaking and
included in the administrative record for any related agency action.
6. Management of Peer Review Process and Reviewer Selection: The
agency may commission independent entities to manage the peer review
process, including the selection of peer reviewers, in accordance with
this Bulletin.
III. Additional Peer Review Requirements for Highly Influential
Scientific Assessments
1. Applicability: This section applies to influential scientific
information that the agency or the Administrator determines to be a
scientific assessment that:
(i) Could have a potential impact of more than $500 million in any
year, or
(ii) Is novel, controversial, or precedent-setting or has
significant interagency interest.
2. In General: To the extent permitted by law, each agency shall
conduct peer reviews on all information subject to this Section. The
peer reviews shall satisfy the requirements of Section II of this
Bulletin, as well as the additional
[[Page 2676]]
requirements found in this Section. Principal findings, conclusions and
recommendations in official reports of the National Academy of Sciences
that fall under this Section are generally presumed not to require
additional peer review.
3. Selection of Reviewers: a. Expertise and Balance: Peer reviewers
shall be selected based on expertise, experience and skills, including
specialists from multiple disciplines, as necessary. The group of
reviewers shall be sufficiently broad and diverse to fairly represent
the relevant scientific and technical perspectives and fields of
knowledge. Agencies shall consider requesting that the public,
including scientific and professional societies, nominate potential
reviewers.
b. Conflicts: The agency--or the entity selecting the peer
reviewers--shall (i) ensure that those reviewers serving as Federal
employees (including special government employees) comply with
applicable Federal ethics requirements; (ii) in selecting peer
reviewers who are not government employees, adopt or adapt the National
Academy of Sciences' policy for committee selection with respect to
evaluating the potential for conflicts (e.g., those arising from
investments; agency, employer, and business affiliations; grants,
contracts and consulting income). For scientific assessments relevant
to specific regulations, a reviewer's financial ties to regulated
entities (e.g., businesses), other stakeholders, and the agency shall
be examined.
c. Independence: In addition to the requirements of Section II
(3)(c), which shall apply to all reviews conducted under Section III,
the agency--or entity selecting the reviewers--shall bar participation
of scientists employed by the sponsoring agency unless the reviewer is
employed only for the purpose of conducting the peer review (i.e.,
special government employees). The only exception to this bar would be
the rare case where the agency determines, using the criteria developed
by NAS for evaluating use of ``employees of sponsors,'' that a premier
government scientist is (a) not in a position of management or policy
responsibility and (b) possesses essential expertise that cannot be
obtained elsewhere. Furthermore, to be eligible for this exception, the
scientist must be employed by a different agency of the Cabinet-level
department than the agency that is disseminating the scientific
information. The agency's determination shall be documented in writing
and approved, on a non-delegable basis, by the Secretary or Deputy
Secretary of the department prior to the scientist's appointment.
d. Rotation: Agencies shall avoid repeated use of the same reviewer
on multiple assessments unless his or her participation is essential
and cannot be obtained elsewhere.
4. Information Access: The agency--or entity managing the peer
review--shall provide the reviewers with sufficient information--
including background information about key studies or models--to enable
them to understand the data, analytic procedures, and assumptions used
to support the key findings or conclusions of the draft assessment.
5. Opportunity for Public Participation: Whenever feasible and
appropriate, the agency shall make the draft scientific assessment
available to the public for comment at the same time it is submitted
for peer review (or during the peer review process) and sponsor a
public meeting where oral presentations on scientific issues can be
made to the peer reviewers by interested members of the public. When
employing a public comment process as part of the peer review, the
agency shall, whenever practical, provide peer reviewers with access to
public comments that address significant scientific or technical
issues. To ensure that public participation does not unduly delay
agency activities, the agency shall clearly specify time limits for
public participation throughout the peer review process.
6. Transparency: In addition to the requirements specified in
II(5), which shall apply to all reviews conducted under Section III,
the peer review report shall include the charge to the reviewers and a
short paragraph on both the credentials and relevant experiences of
each peer reviewer. The agency shall prepare a written response to the
peer review report explaining (a) the agency's agreement or
disagreement with the views expressed in the report, (b) the actions
the agency has undertaken or will undertake in response to the report,
and (c) the reasons the agency believes those actions satisfy the key
concerns stated in the report (if applicable). The agency shall
disseminate its response to the peer review report on the agency's Web
site with the related material specified in Section II(5).
7. Management of Peer Review Process and Reviewer Selection: The
agency may commission independent entities to manage the peer review
process, including the selection of peer reviewers, in accordance with
this Bulletin.
IV. Alternative Procedures
As an alternative to complying with Sections II and III of this
Bulletin, an agency may instead: (i) Rely on the principal findings,
conclusions and recommendations of a report produced by the National
Academy of Sciences; (ii) commission the National Academy of Sciences
to peer review an agency's draft scientific information; or (iii)
employ an alternative scientific procedure or process, specifically
approved by the Administrator in consultation with the Office of
Science and Technology Policy (OSTP), that ensures the agency's
scientific information satisfies applicable information quality
standards. The alternative procedure(s) may be applied to a designated
report or group of reports.
V. Peer Review Planning
1. Peer Review Agenda: Each agency shall post on its Web site, and
update at least every six months, an agenda of peer review plans. The
agenda shall describe all planned and ongoing influential scientific
information subject to this Bulletin. The agency shall provide a link
from the agenda to each document that has been made public pursuant to
this Bulletin. Agencies are encouraged to offer a listserve or similar
mechanism to alert interested members of the public when entries are
added or updated.
2. Peer Review Plans: For each entry on the agenda the agency shall
describe the peer review plan. Each peer review plan shall include: (i)
A paragraph including the title, subject and purpose of the planned
report, as well as an agency contact to whom inquiries may be directed
to learn the specifics of the plan; (ii) whether the dissemination is
likely to be influential scientific information or a highly influential
scientific assessment; (iii) the timing of the review (including
deferrals); (iv) whether the review will be conducted through a panel
or individual letters (or whether an alternative procedure will be
employed); (v) whether there will be opportunities for the public to
comment on the work product to be peer reviewed, and if so, how and
when these opportunities will be provided; (vi) whether the agency will
provide significant and relevant public comments to the peer reviewers
before they conduct their review; (vii) the anticipated number of
reviewers (3 or fewer; 4-10; or more than 10); (viii) a succinct
description of the primary disciplines or expertise needed in the
review; (ix) whether reviewers will be selected by the agency or by a
[[Page 2677]]
designated outside organization; and (x) whether the public, including
scientific or professional societies, will be asked to nominate
potential peer reviewers.
3. Public Comment: Agencies shall establish a mechanism for
allowing the public to comment on the adequacy of the peer review
plans. Agencies shall consider public comments on peer review plans.
VI. Annual Reports
Each agency shall provide to OIRA, by December 15 of each year, a
summary of the peer reviews conducted by the agency during the fiscal
year. The report should include the following: (1) The number of peer
reviews conducted subject to the Bulletin (i.e., for influential
scientific information and highly influential scientific assessments);
(2) the number of times alternative procedures were invoked; (3) the
number of times waivers or deferrals were invoked (and in the case of
deferrals, the length of time elapsed between the deferral and the peer
review); (4) any decision to appoint a reviewer pursuant to any
exception to the applicable independence or conflict of interest
standards of the Bulletin, including determinations by the Secretary
pursuant to Section III(3)(c); (5) the number of peer review panels
that were conducted in public and the number that allowed public
comment; (6) the number of public comments provided on the agency's
peer review plans; and (7) the number of peer reviewers that the agency
used that were recommended by professional societies.
VII. Certification in the Administrative Record
If an agency relies on influential scientific information or a
highly influential scientific assessment subject to this Bulletin to
support a regulatory action, it shall include in the administrative
record for that action a certification explaining how the agency has
complied with the requirements of this Bulletin and the applicable
information quality guidelines. Relevant materials shall be placed in
the administrative record.
VIII. Safeguards, Deferrals, and Waivers
1. Privacy: To the extent information about a reviewer (name,
credentials, affiliation) will be disclosed along with his/her comments
or analysis, the agency shall comply with the requirements of the
Privacy Act, 5 U.S.C. 522a as amended, and OMB Circular A-130, Appendix
I, 61 FR 6428 (February 20, 1996) to establish appropriate routine uses
in a published System of Records Notice.
2. Confidentiality: Peer review shall be conducted in a manner that
respects (i) confidential business information and (ii) intellectual
property.
3. Deferral and Waiver: The agency head may waive or defer some or
all of the peer review requirements of Sections II and III of this
Bulletin where warranted by a compelling rationale. If the agency head
defers the peer review requirements prior to dissemination, peer review
shall be conducted as soon as practicable.
IX. Exemptions
Agencies need not have peer review conducted on information that
is:
1. Related to certain national security, foreign affairs, or
negotiations involving international trade or treaties where compliance
with this Bulletin would interfere with the need for secrecy or
promptness;
2. Disseminated in the course of an individual agency adjudication
or permit proceeding (including a registration, approval, licensing,
site-specific determination), unless the agency determines that peer
review is practical and appropriate and that the influential
dissemination is scientifically or technically novel or likely to have
precedent-setting influence on future adjudications and/or permit
proceedings;
3. A health or safety dissemination where the agency determines
that the dissemination is time-sensitive (e.g., findings based
primarily on data from a recent clinical trial that was adequately peer
reviewed before the trial began);
4. An agency regulatory impact analysis or regulatory flexibility
analysis subject to interagency review under Executive Order 12866,
except for underlying data and analytical models used;
5. Routine statistical information released by federal statistical
agencies (e.g., periodic demographic and economic statistics) and
analyses of these data to compute standard indicators and trends (e.g.,
unemployment and poverty rates);
6. Accounting, budget, actuarial, and financial information,
including that which is generated or used by agencies that focus on
interest rates, banking, currency, securities, commodities, futures, or
taxes; or
7. Information disseminated in connection with routine rules that
materially alter entitlements, grants, user fees, or loan programs, or
the rights and obligations of recipients thereof.
X. Responsibilities of OIRA and OSTP
OIRA, in consultation with OSTP, shall be responsible for
overseeing implementation of this Bulletin. An interagency group,
chaired by OSTP and OIRA, shall meet periodically to foster better
understanding about peer review practices and to assess progress in
implementing this Bulletin.
XI. Effective Date and Existing Law
The requirements of this Bulletin, with the exception of those in
Section V (Peer Review Planning), apply to information disseminated on
or after six months following publication of this Bulletin, except that
they do not apply to information for which an agency has already
provided a draft report and an associated charge to peer reviewers. Any
existing peer review mechanisms mandated by law shall be employed in a
manner as consistent as possible with the practices and procedures laid
out herein. The requirements in Section V apply to ``highly influential
scientific assessments,'' as designated in Section III of this
Bulletin, within six months of publication of this Bulletin. The
requirements in Section V apply to documents subject to Section II of
this Bulletin one year after publication of this Bulletin.
XII. Judicial Review
This Bulletin is intended to improve the internal management of the
executive branch, and is not intended to, and does not, create any
right or benefit, substantive or procedural, enforceable at law or in
equity, against the United States, its agencies or other entities, its
officers or employees, or any other person.
John D. Graham,
Administrator, Office of Information and Regulatory Affairs.
[FR Doc. 05-769 Filed 1-13-05; 8:45 am]
BILLING CODE 3110-01-P