[Federal Register Volume 66, Number 189 (Friday, September 28, 2001)]
[Notices]
[Pages 49718-49725]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 01-24172]
[[Page 49718]]
=======================================================================
-----------------------------------------------------------------------
OFFICE OF MANAGEMENT AND BUDGET
Guidelines for Ensuring and Maximizing the Quality, Objectivity,
Utility, and Integrity of Information Disseminated by Federal Agencies
AGENCY: Executive Office of the President, Office of Management and
Budget.
ACTION: Final guidelines, with request for comments.
-----------------------------------------------------------------------
SUMMARY: These guidelines implement section 515 of the Treasury and
General Government Appropriations Act for Fiscal Year 2001 (Public Law
106-554). Section 515 directs the Office of Management and Budget (OMB)
to issue government-wide guidelines that ``provide policy and
procedural guidance to Federal agencies for ensuring and maximizing the
quality, objectivity, utility, and integrity of information (including
statistical information) disseminated by Federal agencies.'' Within one
year after OMB issues these guidelines, agencies must issue their own
implementing guidelines that include ``administrative mechanisms
allowing affected persons to seek and obtain correction of information
maintained and disseminated by the agency'' that does not comply with
the OMB guidelines. OMB is also requesting additional comment for 30
days on the ``capable of being substantially reproduced'' standard
(paragraphs V.3.B, V.9, and V.10) which is issued on an interim final
basis.
DATES: Effective Date: October 1, 2001.
Comment Date: Comments on the ``capable of being substantially
reproduced'' standard in paragraphs V.3.B, V.9, and V.10 must be
submitted by October 29, 2001.
ADDRESSES: Please submit comments to Brooke J. Dickson of the Office of
Information and Regulatory Affairs, Office of Management and Budget,
Washington, DC 20503. Comments can also be e-mailed to
[email protected].
FOR FURTHER INFORMATION CONTACT: Brooke J. Dickson, Office of
Information and Regulatory Affairs, Office of Management and Budget,
Washington, DC 20503. Telephone (202) 395-3785.
SUPPLEMENTARY INFORMATION: In section 515(a) of the Treasury and
General Government Appropriations Act for Fiscal Year 2001 (Public Law
106-554; H.R. 5658), Congress directed the Office of Management and
Budget (OMB) to issue, by September 30, 2001, government-wide
guidelines that ``provide policy and procedural guidance to Federal
agencies for ensuring and maximizing the quality, objectivity, utility,
and integrity of information (including statistical information)
disseminated by Federal agencies * * *.'' Section 515(b) goes on to
state that the OMB guidelines shall:
``(1) apply to the sharing by Federal agencies of, and access to,
information disseminated by Federal agencies; and
``(2) require that each Federal agency to which the guidelines
apply--
``(A) issue guidelines ensuring and maximizing the quality,
objectivity, utility, and integrity of information (including
statistical information) disseminated by the agency, by not later than
1 year after the date of issuance of the guidelines under subsection
(a);
``(B) establish administrative mechanisms allowing affected persons
to seek and obtain correction of information maintained and
disseminated by the agency that does not comply with the guidelines
issued under subsection (a); and
``(C) report periodically to the Director--
``(i) the number and nature of complaints received by the agency
regarding the accuracy of information disseminated by the agency; and
``(ii) how such complaints were handled by the agency.''
These guidelines are to be issued ``under sections 3504(d)(1) and
3516'' of the Paperwork Reduction Act of 1995; pursuant to section 3503
of that Act, the authorities of the OMB Director are carried out by the
Administrator of the Office of Information and Regulatory Affairs.
Background
The focus of section 515 is on the Federal Government's information
dissemination activities. Indeed, Federal agencies have disseminated
information to the public for decades. Until recently, agencies have
disseminated information principally by making paper copies of
documents available to the public. In recent years, however, Federal
information dissemination has grown due to the advent of the Internet,
which has ushered in a revolution in communications. The Internet has
enabled Federal agencies to disseminate an ever-increasing amount of
information. Congress has strongly encouraged the Executive Branch's
dissemination efforts in statutes that include particular dissemination
activities and in the government-wide dissemination provisions of the
Paperwork Reduction Act of 1995 (44 U.S.C. chapter 35) (the PRA). In
addition, the Executive Branch's strong support for information
dissemination is reflected in the dissemination provisions of OMB
Circular A-130, ``Management of Federal Information Resources,'' as
well as in the provisions in OMB Circular A-110, ``Uniform
Administrative Requirements for Grants and Agreements With Institutions
of Higher Education, Hospitals, and Other Non-Profit Organizations,''
related to a Freedom of Information Act request for research data
relating to published research findings produced under an award that
were used by the Federal Government in developing an agency action that
has the force and effect of law (64 FR 54926; October 8, 1999).
Section 515 builds upon the existing agency responsibility to
ensure information quality. According to the PRA, agency Chief
Information Officers (CIOs) must manage information resources to
``improve the integrity, quality, and utility of information to all
users within and outside the agency, including capabilities for
ensuring dissemination of public information, public access to
government information, and protections for privacy and security.''
Before an agency collects information from 10 or more persons, the
agency must seek public comment ``to enhance the quality, utility, and
clarity of the information to be collected.'' The agency then must
obtain OMB approval that is based upon an evaluation of the agency's
need for the information, the ``practical utility'' of the information
to be collected, and the minimization of burden that would be imposed
on the public in responding to the collection. The CIO must certify to
OMB that the agency, ``to the maximum extent practicable, uses
information technology to reduce burden and improve data quality.''
In developing these guidelines to implement section 515, OMB
recognized that Federal agencies disseminate many types of information
in many different ways. A few examples can only begin to describe the
breadth of information disseminated by the Federal government. Agencies
disseminate statistical information, such as the aggregated information
from the 2000 Census and the monthly and quarterly economic reports
issued by the Bureau of Economic Analysis and the Bureau of Labor
Statistics. Agencies disseminate information that aids members of the
public in their daily activities, such as the National Weather
Service's weather reports and the FAA's air travel advisories. Agencies
disseminate information about health, safety, and environmental risks
and information that they collect from regulated entities, such as
EPA's
[[Page 49719]]
dissemination of Toxic Release Inventory information. Agencies also
disseminate technical information that they create or obtain in the
course of developing regulations, often involving scientific,
engineering, and economic analysis. Agencies disseminate information
when they issue reports and studies. Moreover, agencies provide the
public with basic descriptions of agency authorities, activities and
programs, along with the contact information for the public to interact
with and access that information or those services.
Underlying Principles
In accordance with section 515, OMB has designed the guidelines to
help agencies ensure and maximize the quality, utility, objectivity and
integrity of the information that they disseminate (meaning to share
with, or give access to, the public). It is crucial that information
Federal agencies disseminate meets these guidelines. In this respect,
the fact that the Internet enables agencies to communicate information
quickly and easily to a wide audience not only offers great benefits to
society, but also increases the potential harm that can result from the
dissemination of information that does not meet basic information
quality guidelines. Recognizing the wide variety of information Federal
agencies disseminate and the wide variety of dissemination practices
that agencies have, OMB developed the guidelines with several
principles in mind.
First, OMB designed the guidelines to apply to a wide variety of
government information dissemination activities that may range in
importance and scope. OMB also designed the guidelines to be generic
enough to fit all media, be they printed, electronic, or in other form.
OMB sought to avoid the problems that would be inherent in developing
detailed, prescriptive, ``one-size-fits-all'' government-wide
guidelines that would artificially require different types of
dissemination activities to be treated in the same manner. Through this
flexibility, each agency will be able to incorporate the requirements
of these OMB guidelines into the agency's own information resource
management and administrative practices.
Second, OMB designed the guidelines so that agencies will meet
basic information quality standards. Given the administrative
mechanisms required by section 515 as well as the standards set forth
in the PRA, it is clear that agencies should not disseminate
substantive information that does not meet a basic level of quality. We
recognize that some government information may need to meet higher or
more specific information quality standards than those that would apply
to other types of government information. The more important the
information, the higher the quality standards to which it should be
held, for example, in those situations involving ``influential
scientific or statistical information'' (a phrased defined in these
guidelines). The guidelines recognize, however, that information
quality comes at a cost. Accordingly, the agencies should weigh the
costs (for example, including costs attributable to agency processing
effort, respondent burden, maintenance of needed privacy, and
assurances of suitable confidentiality) and the benefits of higher
information quality in the development of information, and the level of
quality to which the information disseminated will be held.
More specifically, the OMB guidelines state that ``agencies shall
have a basic standard of quality (including objectivity, utility, and
integrity) as a performance goal * * *''. We note, in the scientific
context, that in 1996 the Congress, for health decisions under the Safe
Drinking Water Act, has already adopted a basic standard of quality for
the use of science in agency decisionmaking. Under 42 U.S.C. 300g-
1(b)(3)(A), an agency is directed, ``to the degree that an Agency
action is based on science,'' to use ``(i) the best available, peer-
reviewed science and supporting studies conducted in accordance with
sound and objective scientific practices; and (ii) data collected by
accepted methods or best available methods (if the reliability of the
method and the nature of the decision justifies use of the data).'' We
also note that the OMB guidelines call for an additional level of
quality ``in those situations involving influential scientific or
statistical information.'' The additional level of quality concerns a
standard of care for scientific or statistical analytical results, a
``capable of being substantially reproduced'' standard that is
discussed below.
We further note that in the 1996 amendments to the Safe Drinking
Water Act the Congress adopted a basic quality standard for the
dissemination of public information about risks of adverse health
effects. Under 42 U.S.C. 300g-1(b)(3)(B), the agency is directed, ``to
ensure that the presentation of information [risk] effects is
comprehensive, informative, and understandable.'' The agency is further
directed, ``in a document made available to the public in support of a
regulation [to] specify, to the extent practicable--(i) each population
addressed by any estimate [of applicable risk effects]; (ii) the
expected risk or central estimate of risk for the specific populations
[affected]; (iii) each appropriate upper-bound or lower-bound estimate
of risk; (iv) each significant uncertainty identified in the process of
the assessment of [risk] effects and the studies that would assist in
resolving the uncertainty; and (v) peer-reviewed studies known to the
[agency] that support, are directly relevant to, or fail to support any
estimate of [risk] effects and the methodology used to reconcile
inconsistencies in the scientific data.'' We urge each agency in
developing its guidelines to evaluate whether adopting or adapting
these basic Congressional standards would be appropriate for judging
the quality of disseminated scientific or statistical information.
Third, OMB designed the proposed guidelines so that agencies can
apply them in a common-sense and workable manner. It is important that
these guidelines do not impose unnecessary administrative burdens that
would inhibit agencies from continuing to take advantage of the
Internet and other technologies to disseminate information that can be
of great benefit and value to the public. In this regard, OMB
encourages agencies to incorporate the standards and procedures
required by these guidelines into their existing information resources
management and administrative practices rather than create new and
potentially duplicative or contradictory processes. The primary example
of this is that the guidelines recognize that, in accordance with OMB
Circular A-130, agencies already have in place well-established
information quality standards and administrative mechanisms that allow
persons to seek and obtain correction of information that is maintained
and disseminated by the agency. Under the OMB guidelines, agencies need
only ensure that their own guidelines are consistent with these OMB
guidelines, and then ensure that their administrative mechanisms
satisfy the standards and procedural requirements in the new agency
guidelines. Similarly, agencies may rely on their implementation of the
Federal Government's computer security laws (formerly, the Computer
Security Act, and now the computer security provisions of the PRA) to
establish appropriate security safeguards for ensuring the
``integrity'' of the information that the agencies disseminate.
Summary of OMB Guidelines
These guidelines apply to Federal agencies subject to the Paperwork
Reduction Act (44 U.S.C. chapter 35). Agencies are directed to develop
information resources management
[[Page 49720]]
procedures for reviewing and substantiating (by documentation or other
means selected by the agency) the quality (including the objectivity,
utility, and integrity) of information before it is disseminated. In
addition, agencies are to establish administrative mechanisms allowing
affected persons to seek and obtain, where appropriate, correction of
information disseminated by the agency that does not comply with the
OMB or agency guidelines. Consistent with the underlying principles
described above, these guidelines stress the importance of having
agencies apply these standards and develop their administrative
mechanisms so they can be implemented in a common sense and workable
manner. Moreover, agencies must apply these standards flexibly, and in
a manner appropriate to the nature and timeliness of the information to
be disseminated, and incorporate them into existing agency information
resources management and administrative practices.
Section 515 denotes four substantive terms regarding information
disseminated by Federal agencies: quality, utility, objectivity, and
integrity. It is not always clear how each substantive term relates--or
how the four terms in aggregate relate--to the widely divergent types
of information that agencies disseminate. The guidelines provide
definitions that attempt to establish a clear meaning so that both the
agency and the public can readily judge whether a particular type of
information to be disseminated does or does not meet these attributes.
In the guidelines, OMB defines ``quality'' as the encompassing
term, of which ``utility,'' ``objectivity,'' and ``integrity'' are the
constituents. ``Utility'' refers to the usefulness of the information
to the intended users. ``Objectivity'' focuses on whether the
disseminated information is being presented in an accurate, clear,
complete, and unbiased manner, and as a matter of substance, is
accurate, reliable, and unbiased. ``Integrity'' refers to security--the
protection of information from unauthorized access or revision, to
ensure that the information is not compromised through corruption or
falsification. OMB modeled the definitions of ``information,''
``government information,'' ``information dissemination product,'' and
``dissemination'' on the longstanding definitions of those terms in OMB
Circular A-130, but tailored them to fit into the context of these
guidelines.
In addition, agencies have two reporting requirements. The first
report, implemented no later than one year after the issuance of these
OMB guidelines (no later than October 1, 2002), must provide the
agency's information quality guidelines that describe administrative
mechanisms allowing affected persons to seek and obtain, where
appropriate, correction of disseminated information that does not
comply with the OMB and agency guidelines. The second report is an
annual fiscal year report to OMB (to be first submitted on January 1,
2004) providing information (both quantitative and qualitative, where
appropriate) on the number, nature, and resolution of complaints
received by the agency regarding its perceived or confirmed failure to
comply with these OMB and agency guidelines.
Public Comments and OMB Response
Section 515(a) required OMB to provide the public and the Federal
agencies the opportunity to comment on these guidelines. OMB worked
with Federal agencies, through a working group and through an inter-
agency comment process, in the development of the proposed guidelines.
The proposed guidelines were published in the Federal Register on June
28, 2001 (66 FR 34489) providing a public comment period of 45 days.
OMB received a total of 100 comments from academic institutions (36),
Federal agencies (26), individual members of the public (7),
associations affiliated with academia (5), associations affiliated with
medical, social science or science interests (15), associations
affiliated with Federal Government interests (4), and associations
affiliated with industry interests (7).
General Concerns. Many comments expressed support for the idea of
government-wide quality standards for information disseminated by
Federal agencies. Comments also expressed support for OMB's commitment
to creating flexible general guidelines and to minimizing the
administrative costs and burdens that these guidelines will impose. The
majority of comments focused on two aspects of the proposed guidelines:
suggestions for placing limitations on the administrative correction
mechanisms requirements of the statute; and the need to clarify
specific definitions and other terms found in the guidelines.
Many comments raised questions and concerns about how these
guidelines interact with existing statutes and policies, including the
Paperwork Reduction Act and the Government Performance and Results Act.
We have attempted to draft these guidelines in a way that addresses the
requirements of section 515, but does not impose a completely new and
untried set of standards upon Federal agencies. We encourage agencies
to consider the effect of relevant existing statutes and policies in
the development of their own guidelines.
Administrative Mechanisms. These guidelines require agencies to
establish administrative mechanisms allowing affected persons to seek
and obtain, where appropriate, correction of information maintained and
disseminated by the agency that does not comply with the OMB
guidelines. Many comments suggested that limits be imposed on the types
of information that should be subject to these guidelines, in
particular, information that is disseminated by agency libraries. OMB
agrees that archival information disseminated by Federal agency
libraries (for example, Internet distribution of published articles)
should not be covered by these guidelines, given that libraries do not
endorse the information that they disseminate. Moreover, an agency's
dissemination of public filings (for example, corporate filings with
the Securities and Exchange Commission) is not covered by these
guidelines. In each of these situations, the agencies have not authored
these documents and have not adopted them as representing the agencies'
views. By disseminating these materials, the agencies are simply
ensuring that the public can have quicker and easier access to
materials that are publicly available. In developing its implementing
guidelines, and in accordance with the criteria set forth in these
guidelines, each agency should evaluate and identify the types of
information that it disseminates that will be subject to its
guidelines.
In addition, comments also raised the concern that the guidelines
would apply to ``preliminary'' information, and they recommended that
the guidelines exclude such information. OMB appreciates the concerns
that these comments have raised. However, OMB does not believe that an
exclusion for ``preliminary'' information is necessary or appropriate.
It is still important that the quality of preliminary information be
ensured and that preliminary information be subject to the
administrative complaint-and-correction process.
A few comments stated that affected information should be limited
to information used in agency rulemaking. While this has been the
position of previous policies which these guidelines are not intended
to modify or replace (see, e.g., section __.36(d) in OMB Circular A-
110), we believe the
[[Page 49721]]
plain meaning and intent of section 515 covers the larger government
information universe.
Based on the public comments received, these guidelines allow
agencies to determine the appropriate level of correction for a
complaint received. Several comments suggested that agencies use
disclaimers to distinguish the status of information, a practice that
agencies should consider adopting as they consider their information
holdings.
OMB received detailed discussion on the requirement that agencies
develop administrative mechanisms allowing for affected persons to
``seek and obtain correction of information that does not comply with
OMB's guidelines.'' Members of the scientific community expressed
strong concerns about the possibility of a Federal agency that would
``correct'' scientific information without carrying out the scientific
analysis to support the correction. Comments from all fields suggested
in various ways that challenging individuals should be ``required to
openly state his/her relationship with the data/information
(familiarity/expertise) and provide information [as] to his/her
interest in it.''
Comments also pointed out great potential for abuse of this
process. As one association summarized, ``This could be seen to provide
grounds for interested parties to demand access to underlying data, to
compel the government to replicate research findings (at great expense
and with unnecessary delay), or in other ways impede, discredit, harass
or stymie research.'' For example, the National Oceanic and Atmospheric
Administration (NOAA) explained that they receive numerous complaints
from the public when they miss a weather forecast. ``Does this mean
that the NWS [National Weather Service] could be requested to change a
forecast after the fact? Or could someone with an economic interest
challenge official observational data which could affect the value of
an insurance payment?'' asks NOAA.
Overall, OMB does not envision administrative mechanisms that would
burden agencies with frivolous claims. Instead, the correction process
should serve to address the genuine and valid needs of the agency and
its constituents without disrupting agency processes. Agencies, in
making their determination of whether or not to correct information,
may reject claims made in bad faith or without justification, and are
required to undertake only the degree of correction that they conclude
is appropriate for the nature and timeliness of the information
involved, and explain such practices in their annual fiscal year
reports to OMB.
Numerous comments provided language to clarify or limit the term,
``affected persons.'' One academic institution suggested that the term,
``affected persons,'' reflects a criterion of ``direct measurable
impact with significant personal consequence.'' Other academic
institutions suggested that ``affected persons should not be permitted
to challenge the substance of information without showing that a
qualified scientist has found fault with its quality or integrity.''
Similarly, some comments argued that the ability to correct scientific
information should be limited only to other scientists. Several
associations suggested that OMB identify the types of information that
could be challenged rather than to focus on the characteristics of a
``legitimate'' challenger. OMB considered these comments at length. Our
conclusion is that ``affected persons'' are people who may benefit or
be harmed by the disseminated information. This includes persons who
are seeking to address information about themselves as well as persons
who use information. However, each agency should consider how persons
(which includes groups, organizations and corporations, as defined by
the Paperwork Reduction Act) will be affected by the agency's
information. Agencies should address the issue of ``affected persons''
in consultation with their constituents through the public comment
process that agencies will provide after drafting their proposed
guidelines and before submitting them for OMB review.
These guidelines require that an agency official be designated to
receive and resolve complaints regarding information that does not
comply with either the OMB guidelines or the agency's guidelines. In
the proposed guidelines, we required, with a limited exception, that
the Chief Information Officer (CIO) of the agency have this
responsibility. Of the government agencies that commented on this
provision, many pointed to their specific agency practices on
information quality and their designation of a ``quality official'' who
was not necessarily working under the agency CIO. Recognizing that some
agencies may have specific officials in place to address quality
issues, the final guidelines allow agencies to designate an appropriate
official. Agencies may also designate multiple officials, i.e., based
on the needs of individual agency components, as long as there is a
single official with these overall responsibilities designated at the
agency level. The authorized official also needs to consult with the
CIO on quality matters pertaining to information disseminated by the
agency.
Agencies need to respond to complaints in a manner appropriate to
the nature and extent of the complaint. Examples of appropriate
responses, as suggested by comments, include personal contacts via
letter or telephone, form letters, press releases or mass mailings that
correct a widely disseminated error or address a frequently raised
complaint. Agencies may want to utilize other methods of response under
existing agency practices. For example, for agencies with a high volume
of complaints, it is acceptable for the agency to describe a sample of
those complaints in the annual fiscal year report to OMB. For
categories of inconsequential or trivial complaints identified in the
agency guidelines, an agency may decide that no response is necessary.
Agencies should describe to OMB as part of the annual fiscal year
report the chosen response mechanisms and how they are working.
Definitions and Other Terms. Section 515 denotes four substantive
terms regarding information disseminated by Federal agencies: quality,
utility, objectivity, and integrity. We have defined ``quality'' as an
encompassing term. The organizations and individuals that submitted
comments did not object to having ``quality'' defined as an
encompassing term, but suggested that we should discuss each term
separately. The principles laid out in the proposed guidelines, stated
one comment, create ``subjective definitions'' of the four terms. This
comment warned OMB that ``subjective definitions of quality,
objectivity, utility, and integrity could cause agencies to delay data
release or disregard data for fear of challenge.'' Other comments
expressed similar views, or as one association observed, ``Science does
not recognize a sliding scale of quality.''
These guidelines reflect OMB's determination that ``quality,''
``utility,'' ``objectivity,'' and ``integrity'' are closely
interrelated concepts in the context of these guidelines. Collectively,
these terms address the following three aspects of the information that
is to be disseminated: whether the information is useful to the
intended users of the information; whether the disseminated information
is being presented in an accurate, clear, complete, and unbiased manner
in both presentation and as a characteristic that should be inherent to
quality information; and whether the information has been protected
from unauthorized access or revision.
[[Page 49722]]
Some comments stated that OMB was ``exceeding the statutory
mandate'' and going beyond ``Congressional intent'' in specifying
scientific and statistical information in these guidelines. Others felt
that we should simply acknowledge that the scientific and statistical
communities already have practices and standards for their information,
rather than create another set of standards for these information
types. OMB does not agree with those comments that said the proposed
guidelines went beyond the statute in covering statistical and
scientific information. Section 515 expressly states that its scope
includes statistical information. Moreover, section 515 has no
exclusion for scientific information, and in many respects it is very
similar to (and overlaps with) statistical information. OMB, however,
does appreciate the concerns that the comments raised about the
guidelines not creating another set of standards for statistical and
scientific information. Our guidelines do not seek to impose new
standards on these communities, but to reiterate the standards that are
already held in those communities.
Recognizing public interest in medical and public health
information, we have specifically added a provision stating, ``Agencies
shall adopt specific standards of quality that are appropriate for the
various categories of information they disseminate.'' For example, OMB
encourages agencies, in crafting their agency-specific guidelines, to
promote objectivity in information quality in ways that protect the
confidentiality of research subjects and encourage public participation
in research. These concerns are particularly salient in medical and
public health research.
A number of comments regarded our discussion of ensuring that
scientific information be ``substantially reproducible'' as requiring
agencies to replicate original data and to perform independent analysis
upon all scientific information disseminated by the agency. We have
responded to these concerns in a number of ways. First, we make it
clear that what we now refer to as the ``capable of being substantially
reproduced'' standard applies to analytical results that are
disseminated, and does not apply to the original or supporting data.
Thus, replication of original data is not required. Second, the
``capable of being substantially reproduced'' standard is applicable
only to ``influential'' scientific and statistical information as
defined in the guidelines. Third, the guidelines call for the agency to
determine that ``influential'' analytical results be capable of being
substantially reproducible by independent analysis. We intend this
standard to say that, if appropriately qualified persons used the same
or a similar methodology, they would be expected to achieve similar
findings and results.
Based on the concerns expressed in the comments, we expanded upon
our discussion of ``capable of being substantially reproduced'' in our
definition of ``objective,'' and added two explanatory definitions. We
state, in paragraphs V.3.B, V.9, and V.10:
In addition, ``objectivity'' involves a focus on ensuring accurate,
reliable, and unbiased information. In a scientific or statistical
context, the original or supporting data shall be generated, and the
analytical results shall be developed, using sound statistical and
research methods.
i. If the results have been subject to formal, independent,
external peer review, the information can generally be considered of
acceptable objectivity.
ii. In those situations involving influential scientific or
statistical information, the results must be capable of being
substantially reproduced, if the original or supporting data are
independently analyzed using the same models. Reproducibility does not
mean that the original or supporting data have to be capable of being
replicated through new experiments, samples or tests.
iii. Making the data and models publicly available will assist in
determining whether analytical results are capable of being
substantially reproduced. However, these guidelines do not alter the
otherwise applicable standards and procedures for determining when and
how information is disclosed. Thus, the objectivity standard does not
override other compelling interests, such as privacy, trade secret, and
other confidentiality protections.
``Influential'' when used in the phrase ``influential scientific or
statistical information'' means the agency expects that information in
the form of analytical results will likely have an important effect on
the development of domestic or international government or private
sector policies or will likely have important consequences for specific
technologies, substances, products or firms.
``Capable of being substantially reproduced'' means that
independent reanalysis of the original or supporting data using the
same methods would generate similar analytical results, subject to an
acceptable degree of imprecision.
As a general matter, in the scientific and research context, we
regard technical information that has been subjected to formal,
independent, external peer review as presumptively objective. An
example of a formal independent external peer review is the review
process used by scientific journals. However, depending on the nature
and timeliness of the information involved, an agency may decide that
peer review is not necessary or appropriate. On the other hand, in
those situations involving influential scientific or statistical
information, the substantial reproducibility standard is added as a
quality standard above and beyond some peer review quality standards.
In the definition of ``influential,'' when used in the phrase
``influential scientific or statistical information,'' we note that the
manner in which people perceive the scientific or statistical
information can have important consequences for specific policies,
technologies, substances, products, and firms.
Based on concerns with the ``substantially reproducible'' standard,
a number of comments suggested that OMB should repropose this standard
for additional public comment, rather than going final at this time.
While, in deference to the statutory deadline, OMB is issuing the
``capable of being substantially reproduced'' standard (paragraphs
V.3.B, V.9, and V.10), OMB is doing so on an interim final basis. We
specifically request public comments on this standard by October 29,
2001. In addition, OMB wants to stress that the guidelines published
today should be understood as a beginning of an evolutionary process
that will include draft agency guidelines, public comment, final agency
guidelines, development of experience with OMB and agency guidelines,
and continued refinement of both OMB and agency guidelines.
OMB modeled the draft definitions of ``information,'' ``government
information,'' ``information dissemination product,'' and
``dissemination'' on the longstanding definitions of those terms in OMB
Circular A-130, but tailored them to fit into the context of these
guidelines. Information that is disseminated on behalf of an agency
(through a contract or a grant) is considered to be sponsored by the
agency and is subject to these guidelines. Consistent with the PRA
concept of agency ``sponsorship'' of a collection of information,
information is considered to be disseminated on behalf of an agency by
a contractor or grantee if the dissemination is done at the agency's
specific request or with the agency's specific approval. See 5 CFR
1320.3(d). Finally, it should be noted
[[Page 49723]]
that these guidelines focus primarily on the dissemination of
substantive information (i.e. reports, studies, summaries) rather than
information pertaining to basic agency operations.
We have clarified two terms for the final guidelines. The proposed
guidelines included ``opinions'' in the definition of ``information.''
We agree with comments that indicated agencies should not be
accountable for correcting someone's opinion, but in the agency's
presentation of the information, it should be clear that what is being
offered is someone's opinion rather than facts or the agency's views.
``Opinion'' has therefore been removed from the definition of
``information'' in the final guidelines. The definition for
``dissemination'' was also revised after discussions with two Federal
agencies that correspond frequently with individual members of the
public regarding their participation in the agency's programs. In
addition, in the definition of ``dissemination,'' we changed the
exclusion for ``judicial process'' to ``adjudicative process'' to make
it clear that these guidelines do not apply to the issuance of agency
adjudicative decisions.
Reporting Requirements. Agencies have two reporting requirements.
The first report, taking effect no later than one year after the
issuance of these OMB guidelines, must provide the agency's information
quality guidelines that describe administrative mechanisms allowing
affected persons to seek and obtain, where appropriate, correction of
disseminated information that does not comply with these OMB
guidelines. During the year that agencies have to complete their agency
guidelines, agencies must publish the draft reports in the Federal
Register for a period of public comment, and no later than nine months
after the issuance of OMB's guidelines, submit their draft reports to
OMB for review. Upon completion of OMB's review, final agency
guidelines must be published in the Federal Register and made available
through the agency website. The entire process must be completed by no
later than one year after the issuance of the OMB guidance (no later
than October 1, 2002).
The second report is an annual fiscal year report to OMB (to be
first submitted on January 1, 2004) providing information on the
number, nature, and resolution of complaints received by the agency
regarding its perceived or confirmed failure to comply with these OMB
and agency guidelines. Regarding the proposed guidelines, we received
detailed comments on the required report to OMB describing the number
and nature of complaints received by the agency and how such complaints
were resolved. Two Federal agencies stated that it would be burdensome
to report to OMB on every single complaint they received and responded
to, particularly because many of the complaints may be received in
phone calls and given informal responses that address the callers'
concerns. Recognizing that agencies may deal with large volumes of
complaints on particular types of information disseminated by the
agency, OMB's guidelines allow the agency to provide qualitative and/or
quantitative descriptions of complaints received and how they were
resolved (or not). OMB also recognizes that a large number of comments
about a specific document may only demonstrate that the information is
controversial, not that its quality is flawed.
In conclusion, issuance of these final guidelines meets the
statutory requirement that section 515 imposed on OMB. As we stated
earlier in this preamble, and in connection with the proposed
guidelines, OMB has sought in developing these guidelines to make them
flexible enough so that Federal agencies can apply them in a common
sense, workable, and appropriately tailored manner to the wide variety
of dissemination activities that the Federal Government undertakes. In
addition, in drafting guidelines that will apply on a government-wide
basis, OMB has been sensitive to the problem of unintended consequences
and has tried to anticipate and address issues that could arise during
the implementation of these guidelines. In this respect, the public and
agency comments that we received on the proposed guidelines were very
helpful and are greatly appreciated. As we explained above, we made a
number of revisions to the guidelines to address the concerns raised in
the comments, and we also believe that these and other concerns can be
addressed as well in the implementing guidelines that each agency will
develop in the coming months. In addition, OMB is issuing the ``capable
of being substantially reproduced'' standard (paragraphs V.3.B, V.9,
and V.10) on an interim final basis. We specifically request public
comments on this standard over the next 30 days.
Moreover, over time as the agencies and the public gain further
experience with the OMB guidelines, we would appreciate receiving any
suggestions for how OMB could improve them. Just as OMB requested
public comment before issuing these final guidelines, OMB will refine
these guidelines as experience develops and further public comment is
obtained.
Dated: September 24, 2001.
John D. Graham,
Administrator, Office of Information and Regulatory Affairs.
Guidelines for Ensuring and Maximizing the Quality, Objectivity,
Utility, and Integrity of Information Disseminated by Federal
Agencies
I. OMB Responsibilities
Section 515 of the Treasury and General Government Appropriations
Act for FY2001 (Public Law 106-554) directs the Office of Management
and Budget to issue government-wide guidelines that provide policy and
procedural guidance to Federal agencies for ensuring and maximizing the
quality, objectivity, utility, and integrity of information, including
statistical information, disseminated by Federal agencies.
II. Agency Responsibilities
Section 515 directs agencies subject to the Paperwork Reduction Act
(44 U.S.C. 3502(a)) to--
1. Issue their own information quality guidelines ensuring and
maximizing the quality, objectivity, utility, and integrity of
information, including statistical information, disseminated by the
agency no later than one year after the date of issuance of the OMB
guidelines;
2. Establish administrative mechanisms allowing affected persons to
seek and obtain correction of information maintained and disseminated
by the agency that does not comply with these OMB guidelines; and
3. Report to the Director of OMB the number and nature of
complaints received by the agency regarding agency compliance with
these OMB guidelines concerning the quality, objectivity, utility, and
integrity of information and how such complaints were resolved.
III. Guidelines for Ensuring and Maximizing the Quality, Objectivity,
Utility, and Integrity of Information Disseminated by Federal Agencies
1. Overall, agencies shall adopt a basic standard of quality
(including objectivity, utility, and integrity) as a performance goal
and should take appropriate steps to incorporate information quality
criteria into agency information dissemination practices. Quality is to
be ensured and established at levels appropriate to the nature and
timeliness of the information to be disseminated. Agencies shall adopt
specific standards of quality that are appropriate for the various
categories of information they disseminate.
2. As a matter of good and effective agency information resources
[[Page 49724]]
management, agencies shall develop a process for reviewing the quality
(including the objectivity, utility, and integrity) of information
before it is disseminated. Agencies shall treat information quality as
integral to every step of an agency's development of information,
including creation, collection, maintenance, and dissemination. This
process shall enable the agency to substantiate the quality of the
information it has disseminated through documentation or other means
appropriate to the information.
3. To facilitate citizen review, agencies shall establish
administrative mechanisms allowing affected persons to seek and obtain,
where appropriate, timely correction of information maintained and
disseminated by the agency that does not comply with OMB or agency
guidelines. These administrative mechanisms shall be flexible,
appropriate to the nature and timeliness of the disseminated
information, and incorporated into agency information resources
management and administrative practices.
4. The agency's pre-dissemination review, under paragraph III.2,
shall apply to information that the agency first disseminates on or
after October 1, 2002. The agency's administrative mechanisms, under
paragraph III.3, shall apply to information that the agency
disseminates on or after October 1, 2002, regardless of when the agency
first disseminated the information.
IV. Agency Reporting Requirements
1. Agencies must designate the Chief Information Officer or another
official to be responsible for agency compliance with these guidelines.
2. The agency shall respond to complaints in a manner appropriate
to the nature and extent of the complaint. Examples of appropriate
responses include personal contacts via letter or telephone, form
letters, press releases or mass mailings that correct a widely
disseminated error or address a frequently raised complaint.
3. Each agency must prepare a draft report, no later than April 1,
2002, providing the agency's information quality guidelines and
explaining how such guidelines will ensure and maximize the quality,
objectivity, utility, and integrity of information, including
statistical information, disseminated by the agency. This report must
also detail the administrative mechanisms developed by that agency to
allow affected persons to seek and obtain appropriate correction of
information maintained and disseminated by the agency that does not
comply with the OMB or the agency guidelines.
4. The agency must publish a notice of availability of this draft
report in the Federal Register, and post this report on the agency's
website, to provide an opportunity for public comment.
5. Upon consideration of public comment and after appropriate
revision, the agency must submit this draft report to OMB for review
regarding consistency with these OMB guidelines no later than July 1,
2002. Upon completion of that OMB review and completion of this report,
agencies must publish notice of the availability of this report in the
Federal Register, and post this report on the agency's web site no
later than October 1, 2002.
6. On an annual fiscal-year basis, each agency must submit a report
to the Director of OMB providing information (both quantitative and
qualitative, where appropriate) on the number and nature of complaints
received by the agency regarding agency compliance with these OMB
guidelines and how such complaints were resolved. Agencies must submit
these reports no later than January 1 of each following year, with the
first report due January 1, 2004.
V. Definitions
1. ``Quality'' is an encompassing term comprising utility,
objectivity, and integrity. Therefore, the guidelines sometimes refer
to these four statutory terms, collectively, as ``quality.''
2. ``Utility'' refers to the usefulness of the information to its
intended users, including the public. In assessing the usefulness of
information that the agency disseminates to the public, the agency
needs to consider the uses of the information not only from the
perspective of the agency but also from the perspective of the public.
As a result, when reproducibility and transparency of information are
relevant for assessing the information's usefulness from the public's
perspective, the agency must take care to ensure that reproducibility
and transparency have been addressed in its review of the information.
3. ``Objectivity'' involves two distinct elements, presentation and
substance.
A. ``Objectivity'' includes whether disseminated information is
being presented in an accurate, clear, complete, and unbiased manner.
This involves whether the information is presented within a proper
context. Sometimes, in disseminating certain types of information to
the public, other information must also be disseminated in order to
ensure an accurate, clear, complete, and unbiased presentation. Also,
the agency needs to identify the sources of the disseminated
information (to the extent possible, consistent with confidentiality
protections) and, in a scientific or statistical context, the
supporting data and models, so that the public can assess for itself
whether there may be some reason to question the objectivity of the
sources. Where appropriate, supporting data should have full, accurate,
transparent documentation, and error sources affecting data quality
should be identified and disclosed to users.
B. In addition, ``objectivity'' involves a focus on ensuring
accurate, reliable, and unbiased information. In a scientific or
statistical context, the original or supporting data shall be
generated, and the analytical results shall be developed, using sound
statistical and research methods.
i. If the results have been subject to formal, independent,
external peer review, the information can generally be considered of
acceptable objectivity.
ii. In those situations involving influential scientific or
statistical information, the results must be capable of being
substantially reproduced, if the original or supporting data are
independently analyzed using the same models. Reproducibility does not
mean that the original or supporting data have to be capable of being
replicated through new experiments, samples or tests.
iii. Making the data and models publicly available will assist in
determining whether analytical results are capable of being
substantially reproduced. However, these guidelines do not alter the
otherwise applicable standards and procedures for determining when and
how information is disclosed. Thus, the objectivity standard does not
override other compelling interests, such as privacy, trade secret, and
other confidentiality protections.
4. ``Integrity'' refers to the security of information--protection
of the information from unauthorized access or revision, to ensure that
the information is not compromised through corruption or falsification.
5. ``Information'' means any communication or representation of
knowledge such as facts or data, in any medium or form, including
textual, numerical, graphic, cartographic, narrative, or audiovisual
forms. This definition includes information that an agency disseminates
from a web page, but does not include the provision of hyperlinks to
information that others disseminate. This definition does not include
opinions, where the agency's presentation makes it clear that what is
[[Page 49725]]
being offered is someone's opinion rather than fact or the agency's
views.
6. ``Government information'' means information created, collected,
processed, disseminated, or disposed of by or for the Federal
Government.
7. ``Information dissemination product'' means any book, paper,
map, machine-readable material, audiovisual production, or other
documentary material, regardless of physical form or characteristic, an
agency disseminates to the public. This definition includes any
electronic document, CD-ROM, or web page.
8. ``Dissemination'' means agency initiated or sponsored
distribution of information to the public (see 5 CFR 1320.3(d)
(definition of ``Conduct or Sponsor''). Dissemination does not include
distribution limited to government employees or agency contractors or
grantees; intra- or inter-agency use or sharing of government
information; and responses to requests for agency records under the
Freedom of Information Act, the Privacy Act, the Federal Advisory
Committee Act or other similar law. This definition also does not
include distribution limited to correspondence with individuals or
persons, press releases, archival records, public filings, subpoenas or
adjudicative processes.
9. ``Influential'' when used in the phrase ``influential scientific
or statistical information'' means the agency expects that information
in the form of analytical results will likely have an important effect
on the development of domestic or international government or private
sector policies or will likely have important consequences for specific
technologies, substances, products or firms.
10. ``Capable of being substantially reproduced'' means that
independent reanalysis of the original or supporting data using the
same methods would generate similar analytical results, subject to an
acceptable degree of imprecision.
[FR Doc. 01-24172 Filed 9-27-01; 8:45 am]
BILLING CODE 3110-01-P