Federal Research: Policies Guiding the Dissemination of
Scientific Research from Selected Agencies Should Be Clarified
and Better Communicated (17-MAY-07, GAO-07-653).
Researchers at federal agencies disseminate their research
results through a variety of approaches, including scientific
publications, presentations, press releases, and media
interviews. Because of recent concerns about some federal
researchers possibly being restricted from disseminating their
research on controversial topics, GAO determined (1) the policies
that guide the dissemination of federal research at the National
Aeronautics and Space Administration (NASA), the National
Institute of Standards and Technology (NIST), and the National
Oceanic and Atmospheric Administration (NOAA); (2) how
effectively these agencies have communicated their policies to
researchers; and (3) the extent to which researchers have been
restricted in disseminating their research. GAO conducted a
survey of 1,811 researchers randomly selected at the three
agencies, and had a 66 percent response rate.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-07-653
ACCNO: A69734
TITLE: Federal Research: Policies Guiding the Dissemination of
Scientific Research from Selected Agencies Should Be Clarified
and Better Communicated
DATE: 05/17/2007
SUBJECT: Employee training
Federal agencies
Federal regulations
Government information dissemination
Information management
Information resources management
Mass media
Policy evaluation
Research and development
Surveys
Policies and procedures
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-07-653
* [1]FEDERAL RESEARCH
* [2]Policies Guiding the Dissemination of Scientific Research fr
* [3]Contents
* [4]Letter
* [5]Results in Brief
* [6]Background
* [7]NASA, NIST, and NOAA Policies Generally Facilitate the
Disse
* [8]NASA Policies Provide Clear Guidance to Facilitate
Research
* [9]NIST and NOAA Policies Provide Clear Guidance to
Facilitate
* [10]Researchers at NIST and NOAA Have to Comply with
Commerce an
* [11]Researchers at NIST and NOAA Are Subject to
Outdated Commerc
* [12]Researchers at NIST Have Been Provided with
Supplemental Age
* [13]Researchers at NOAA Are Subject to Unclear
Agency-Level Medi
* [14]Despite Agencies' Efforts to Communicate Research
Disseminat
* [15]Most Researchers Learned about Agency Dissemination
Policies
* [16]Researchers Have More Concerns about Adhering to
Their Agenc
* [17]Researchers Are Often Unclear about Whether They
Can Discuss
* [18]Researchers Are Generally Unaware of Their
Agencies' Process
* [19]Six Percent of Researchers at NASA, NIST, and NOAA Had
Disse
* [20]Some Researchers at Each Agency Who Were Denied
Approval to
* [21]Researchers Believe That Their Agencies Provide
Greater Supp
* [22]Most Researchers Generally Believe That
Dissemination Polici
* [23]OSTP Provides Indirect Oversight of Agencies'
Dissemination
* [24]Conclusions
* [25]Recommendations for Executive Action
* [26]Agency Comments and Our Evaluation
* [27]Appendix I: Objectives, Scope, and Methodology
* [28]Appendix II: Selected Survey Results for NASA, NIST, and NOA
* [29]Appendix III: Comments from the Department of Commerce
* [30]Appendix IV: Comments from the National Aeronautics and Spac
* [31]Appendix V: Comments from the Office of Science and Technolo
* [32]Appendix VI: GAO Contact and Staff Acknowledgments
* [33]GAO Contact
* [34]Staff Acknowledgments
* [35]Order by Mail or Phone
* [36]PDF6-Ordering Information.pdf
* [37]Order by Mail or Phone
Report to the Chairman, Subcommittee on Commerce, Justice, Science, and
Related Agencies, Committee on Appropriations, U.S. Senate
United States Government Accountability Office
GAO
May 2007
FEDERAL RESEARCH
Policies Guiding the Dissemination of Scientific Research from Selected
Agencies Should Be Clarified and Better Communicated
GAO-07-653
Contents
Letter 1
Results in Brief 4
Background 7
NASA, NIST, and NOAA Policies Generally Facilitate the Dissemination of
Research, but Some Do Not 11
Despite Agencies' Efforts to Communicate Research Dissemination and
Dispute Resolution Policies, Many Researchers Are Not Confident about How
to Comply with Them 20
Six Percent of Researchers at NASA, NIST, and NOAA Had Dissemination
Requests Denied, and Many Believe That Their Agencies Are Less Supportive
of Media Interviews than Other Dissemination Methods 26
OSTP Provides Indirect Oversight of Agencies' Dissemination Policies 31
Conclusions 32
Recommendations for Executive Action 33
Agency Comments and Our Evaluation 33
Appendix I Objectives, Scope, and Methodology 35
Appendix II Selected Survey Results for NASA, NIST, and NOAA 40
Appendix III Comments from the Department of Commerce 88
Appendix IV Comments from the National Aeronautics and Space
Administration 93
Appendix V Comments from the Office of Science and Technology Policy 95
Appendix VI GAO Contact and Staff Acknowledgments 96
Figures
Figure 1: NOAA/Commerce News Release Review Process, as of March 2006 19
Figure 2: Estimated Percentage of Researchers Who Learned How to Comply
with Dissemination Policies through Various Methods 22
Figure 3: Estimated Percentage of Researchers Who Are Confident That They
Understand Policies Well Enough to Follow Them 23
Figure 4: Estimated Percentage of Researchers Who Believe That Their
Agency Supports Dissemination, by Dissemination Route 29
Figure 5: Estimated Percentage of Researchers Who Believe That Their
Agency Consistently Applies Policies, by Dissemination Route 31
Abbreviations
EOP Executive Office of the President
NASA National Aeronautics and Space Administration
NIST National Institute of Standards and Technology
NOAA National Oceanic and Atmospheric Administration
OMB Office of Management and Budget
OSTP Office of Science and Technology Policy
OPCIA Office of Public, Constituent, and Intergovernmental Affairs
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
United States Government Accountability Office
Washington, DC 20548
May 17, 2007
The Honorable Barbara A. Mikulski
Chairman
Subcommittee on Commerce, Justice, Science, and Related Agencies
Committee on Appropriations
United States Senate
Dear Madam Chairman:
Federal agencies strive to base science-related public policy on the best
available information provided through objective research, some of which
is conducted by federal researchers. Because progress in scientific
understanding depends on open communication, sharing research results with
the broader research community is necessary to help validate and expand
upon these results. While the results of some research cannot be
disseminated because of national security or intellectual property
concerns, dissemination of most federally funded research helps ensure a
maximum return on the public's investment in this research. Much
scientific research is either conducted by federally employed researchers
or supported through grants and contracts awarded by over a dozen federal
departments and agencies, such as the National Aeronautics and Space
Administration (NASA), the National Institute of Standards and Technology
(NIST), and the National Oceanic and Atmospheric Administration (NOAA).
This report focuses on the dissemination of research conducted by
federally employed researchers at these three agencies, as you requested.
Federal agencies strive to base science-related public policy on the best
available information provided through objective research, some of which
is conducted by federal researchers. Because progress in scientific
understanding depends on open communication, sharing research results with
the broader research community is necessary to help validate and expand
upon these results. While the results of some research cannot be
disseminated because of national security or intellectual property
concerns, dissemination of most federally funded research helps ensure a
maximum return on the public's investment in this research. Much
scientific research is either conducted by federally employed researchers
or supported through grants and contracts awarded by over a dozen federal
departments and agencies, such as the National Aeronautics and Space
Administration (NASA), the National Institute of Standards and Technology
(NIST), and the National Oceanic and Atmospheric Administration (NOAA).
This report focuses on the dissemination of research conducted by
federally employed researchers at these three agencies, as you requested.
The topics of research performed by NASA, NIST, and NOAA cover the gamut,
from space exploration equipment to voting technology to fishery ecosystem
health. For example, NASA's research is diverse and is spread among 10
centers across the United States. These centers perform work that covers
topics ranging from aeronautical research to exploration systems to Earth,
moon, and space exploration. NIST, an agency within the Department of
Commerce, conducts research that is used by a variety of government and
industrial customers in diverse areas, such as electronic voting
technology, secured automated banking, drinking water quality, and fuel
cell technology. Research at NIST is spread across various laboratories
that focus on particular scientific pursuits, such as information
technology, materials science and engineering, and building The topics of
research performed by NASA, NIST, and NOAA cover the gamut, from space
exploration equipment to voting technology to fishery ecosystem health.
For example, NASA's research is diverse and is spread among 10 centers
across the United States. These centers perform work that covers topics
ranging from aeronautical research to exploration systems to Earth, moon,
and space exploration. NIST, an agency within the Department of Commerce,
conducts research that is used by a variety of government and industrial
customers in diverse areas, such as electronic voting technology, secured
automated banking, drinking water quality, and fuel cell technology.
Research at NIST is spread across various laboratories that focus on
particular scientific pursuits, such as information technology, materials
science and engineering, and building and fire research. Similarly, the
breadth of research activities undertaken by NOAA, another Commerce
agency, includes providing weather, water, and climate services; managing
and protecting fisheries and sensitive marine ecosystems; conducting
atmospheric, climate, and ecosystems research; and promoting efficient and
environmentally safe commerce and transportation.
Research generated by NASA, NIST, and NOAA researchers can be disseminated
through a number of different routes to reach a variety of audiences. Some
dissemination routes, such as through publications, including
peer-reviewed journals and agency scientific and technical reports, are
useful for sharing information with the scientific community. Similarly,
researchers can present their findings to colleagues at conferences,
workshops, symposia, or professional society meetings. Other dissemination
routes can reach a still wider audience. For example, agency press
releases or postings on agency Web sites can help target media attention
to particular research findings, and media interviews can provide the
general public with access to scientific information.
In recent years, concerns have emerged regarding the possibility that
limits are being placed on the efforts of federally employed researchers
to share the results of their work through these various dissemination
routes. For example, in January 2006, the press reported allegations that
a NASA scientist was restricted from speaking with the press after
presenting his research findings related to climate change at a
conference. In addition, some researchers at NOAA have claimed that the
agency had put restrictions on their ability to speak to reporters, had
imposed delays in the approval and coordination process for speaking with
the media, or had censored their views in issued reports.
In this context, we (1) identified and evaluated the policies that guide
the dissemination of federal research results at NASA, NIST, and NOAA; (2)
determined how effectively the dissemination and dispute resolution
policies of these agencies have been communicated to researchers; and (3)
determined the extent to which researchers at these agencies have
experienced restrictions on the dissemination of their research results.
In addition, we obtained information on the role that the Office of
Science and Technology Policy (OSTP) has identified for itself in helping
agencies develop and implement policies related to the dissemination of
research results.
To identify and evaluate the policies that guide the dissemination of
federally funded research results at NASA, NIST, and NOAA, we obtained,
reviewed, and analyzed the dissemination policies for these agencies, in
addition to relevant policies from Commerce. In addition, we spoke with
managers and researchers at NASA, NIST, and NOAA as well as public affairs
officials at each of these agencies and Commerce. During these interviews,
we confirmed that we had copies of the relevant dissemination policies and
discussed how these policies were put into practice. To determine how
effectively these policies have been communicated to researchers and the
extent to which researchers at these agencies have experienced
restrictions on the dissemination of their research results, we surveyed a
stratified random sample of 1,811 researchers across scientific and
engineering disciplines at NASA, NIST, and NOAA. This sample included 578
NASA researchers, 699 NIST researchers, and 534 NOAA researchers. We
selected these 1,811 researchers from among the 1,794 researchers at NASA,
1,337 researchers at NIST, and 1,815 researchers at NOAA. Overall, we
received a 66 percent response rate to our survey, for a total of 1,177
respondents. We defined researchers to be included in our population as
federally employed scientists, engineers, or other researchers who are in
a position to disseminate their research results to a wider audience.
Through our survey, we sought the researchers' views on their agencies'
research dissemination policies, the level of agency support for
dissemination, and their experiences with dissemination. In addition, we
asked the researchers to provide examples of ways in which their agencies'
dissemination policies work well and ways in which these policies could be
improved. All estimates based on our survey allow us to project the
results of our survey to all researchers at these three agencies with a 95
percent level of confidence. Unless otherwise noted, all percentage
estimates have a 95 percent confidence interval within plus or minus 8
percentage points. In addition, any comparison between point estimates is
statistically significant at the 0.05 level, unless otherwise noted. In
one instance, the number of responses to three follow-up questions
designed to elicit elaborative information received too few responses to
generalize to the population. Although we cannot generalize from these
responses, we included this information because it provides context for
specific concerns that these respondents had about certain research
dissemination policies at their agencies. Where we use this information,
we highlight the fact that it cannot be generalized to the population by
attributing the statements as a survey response, rather than as researcher
perception. To determine OSTP's role in helping agencies develop and
implement policies for the dissemination of research results, we submitted
detailed questions to OSTP officials, to which we received written
responses. Appendix I contains an expanded explanation of our scope and
methodology, and appendix II contains selected survey results for NASA,
NIST, and NOAA. We conducted our work from June 2006 through March 2007 in
accordance with generally accepted government auditing standards.
Results in Brief
Most of the policies guiding the dissemination of scientific research at
NASA, NIST, and NOAA generally facilitate the dissemination process, but
some Commerce and NOAA policies lack clarity and can hinder dissemination.
Specifically, we found the following:
o At NASA, researchers must comply with both agencywide
dissemination policies and policies established by the specific
NASA centers to which they belong. We found that NASA policies are
generally clear and should help facilitate the dissemination of
research results. For example, NASA's recently revised media
policy clearly defines the roles and responsibilities for
managers, researchers, and public affairs staff; details steps in
the process for dissemination via press releases and interviews;
and describes a process to resolve disputes about agency decisions
regarding press releases. To supplement the policy, NASA has also
developed operating procedures for the release of public
information and a "Frequently Asked Questions" guide.
o At NIST, researchers must comply with both agency-level policies
for the dissemination of research through publications and
presentations and Commerce's department-level policies for
dissemination of research through media interviews and press
releases. NIST's policies generally facilitate the dissemination
of scientific results because they clearly describe the reviews
that are required before publications and presentations may be
released, and they describe the process that researchers may use
to appeal decisions made during the review process. In contrast,
Commerce's policies that apply to requests for media interviews
and press releases have not been revised for over 20 years, are
unrealistic, and may hinder dissemination efforts. For example,
Commerce requires all of its constituent agencies to submit
proposed media interviews and press releases to its
department-level Office of Public Affairs for review and approval;
it does not delegate authority to approve these activities to the
agency-level public affairs offices. Because of the potentially
high volume of requests, sometimes in the hundreds after a large
event, it is not realistic to expect the department-level Office
of Public Affairs to review every media interview and press
release request that each Commerce agency generates.
o At NOAA, researchers must comply with both agency-level policies
and policies established by individual NOAA program offices for
dissemination of research through publications and presentations.
These policies generally facilitate dissemination, because they
are clear and delegate authority to approve dissemination requests
to managers in the office to which the researcher belongs. For
dissemination of research through press releases and media
interviews, NOAA researchers must comply not only with Commerce's
department-level policies, which as previously mentioned are
outdated and may hinder dissemination, but also with NOAA's
agencywide policies, which are unclear and may further impede
dissemination. For example, the lack of clarity in NOAA's media
interview policy has led to different interpretations of the
policy by NOAA public affairs officials, which results in an
uneven application of the policy among researchers. Some public
affairs officials have interpreted the policy to mean that
researchers are required to obtain prior approval for all
interviews, while others believe researchers have to notify the
public affairs office only after the interview has occurred.
Despite the agencies' efforts to communicate their dissemination
policies to researchers, many researchers are not confident that
they know how to comply with some of these policies or how to
resolve disputes regarding agency decisions, according to our
survey. NASA, NIST, and NOAA have employed a variety of formal and
informal methods, including staff meetings and notifications on
agency Web sites, to communicate dissemination policy to their
researchers. However, most researchers learned how to comply with
their agency's policies through informal methods, such as e-mails
from management and on-the-job experience, rather than through
more formal means, such as training sessions. Moreover, many
researchers are not confident that they understand all of the
policies well enough to follow them. For example, according to our
survey, while an estimated 90 percent of researchers across all
three agencies believe they understand their agency's policy for
dissemination through publications well enough to comply with
them, only about 65 percent of researchers believe they understand
their agency's policies for media interviews and press releases.
Similarly, although NASA and NOAA leaders have told researchers
that they may discuss potential policy implications of their
research as long as they identify such views as their personal
opinions and not those of the agency, this communication has not
been effective, as fewer than one-half of the researchers at these
two agencies believe they are free to discuss their views. In
addition, only 25 percent of researchers across all three agencies
are aware of a process or procedure they are to follow if they
want to appeal denials of requests to disseminate their research.
On the basis of our survey results, we estimate that 6 percent
(about 200) of the researchers across NASA, NIST, and NOAA have
been denied approval to disseminate their research results in the
past 5 years. Despite the difficulties experienced by these
researchers, most researchers believe that their agency either
encourages dissemination of research results or insists on it.
However, researchers believe that, overall, their agencies tend to
be less supportive of dissemination through media interviews than
through other dissemination routes. Among the most common reasons
researchers mentioned for denials of their requests for
dissemination, other than those stemming from standard technical
review, is that the topic or issue of the research was sensitive
or that the research was restricted for security reasons. In some
instances, researchers reported that their agency gave them no
reason for the denial. As a result of these denials, many of these
researchers gave up trying to disseminate their results, while
others subsequently disseminated the results of their work using a
different dissemination route. For example, one researcher who had
been denied permission to present his research at a conference
went on to disseminate the research results by publishing a paper.
Regarding agency support for dissemination, over 90 percent of the
researchers at NASA, NIST, and NOAA believe that their agency
supports dissemination of research results through publications
and presentations; and significantly fewer researchers, ranging
from 54 percent of the researchers at NOAA to 68 percent of the
researchers at NIST, believe that their agency is supportive of
dissemination through media interviews. Finally, most researchers
at NIST and NOAA believe that their agencies consistently apply
the dissemination policies for each route of dissemination. In
contrast, more researchers at NASA believe that the agency
consistently applies its policies for publications than believe
that the agency consistently applies its policies for press
releases and media interviews.
According to OSTP, it does not conduct scientific research on its
own nor does it formulate or directly oversee the development of
dissemination policies or decisions at individual agencies.
However, OSTP has publicly affirmed the value of science as a
basis for federal action and recognizes the importance of timely,
complete, and accurate communication of scientific information.
The OSTP Director has on several occasions asked the leaders and
chief scientists of federal agencies to develop, revise, or
reemphasize their dissemination policies and to ensure that agency
employees and managers understand their rights and obligations
under these policies. The director has cited NASA's media policy
as a model for other agencies to consider in developing their own
dissemination policies.
Given the lack of clarity of some dissemination policies, the lack
of a process for appealing dissemination decisions, and researcher
uncertainty about how to comply with these policies, we are
recommending that Commerce and NOAA clarify their policies for
disseminating research results via press releases and media
interviews. Furthermore, we are recommending that Commerce, NASA,
NIST, and NOAA review their dissemination policies and ensure that
they clearly identify a process by which researchers can appeal
dissemination decisions, and provide formal training to inform,
reinforce, and update managers, researchers, and public affairs
staff on these policies. Commerce, on behalf of NIST, NOAA, and
itself, generally concurred with our findings and recommendations;
NASA and OSTP concurred with our recommendations.
Background
In recent years, there have been a number of allegations
concerning interference with federal researchers in their efforts
to disseminate their research results to a wider audience,
external to their agency.^1 Many of these allegations have
concerned interference with researchers' attempts to speak with
the media or present research findings at conferences. These
claims have frequently surrounded research on politically
sensitive topics, such as climate change and the role it may play
in connection to other weather events, such as hurricanes. For
example, some federal researchers have alleged that they were not
allowed to use the phrase "global warming," and others have
alleged that NOAA has limited scientific debate by not reporting
all of the research regarding a possible connection between global
warming and increased hurricane intensity.
These concerns have resulted in a number of inquiries into the
dissemination practices and policies at several federal agencies.
For example, in May 2006, in response to a congressional request,
the National Science Foundation's National Science Board examined
the existing policies of selected federal science agencies,
including NASA and NOAA, to determine if they contained steps to
ensure the credibility of research results and insulate these
results from suppression or distortion. The board found that there
was no consistent federal policy regarding the dissemination of
research results by federal employees and recommended that all
agencies that conduct research establish policies and procedures
to encourage an open exchange of data and results. The board also
recommended that the administration develop and issue an
overarching set of principles for the communication of scientific
information by government scientists, policymakers, and managers
that could be used as the framework under which each agency would
develop its specific policies and procedures.
^1GAO, Data Quality: Expanded Use of Key Dissemination Practices Would
Further Safeguard the Integrity of Federal Statistical Data,
[38]GAO-06-607 (Washington, D.C.: May 31, 2006); and Bureau of Justice
Statistics: Quality Guidelines Generally Followed for Police-Public
Contact Surveys, but Opportunities Exist to Help Assure Agency
Independence, [39]GAO-07-340 (Washington, D.C.: Mar. 30, 2007).
In addition to the National Science Board review, other groups
have sought information on the dissemination activities of federal
agencies. For example, in 2005, Environmental Science and
Technology, an online and print journal published by the American
Chemical Society, requested and received copies of NOAA Office of
Public Affairs documents through the Freedom of Information Act
pertaining to climate change, including exchanges between NOAA
scientists and public affairs staff. Furthermore, in September
2006, 14 Senators requested that the Inspectors General at
Commerce and NASA investigate reports of political interference
with the work of scientists at NASA and NOAA. In February 2007,
the Union of Concerned Scientists and the Government
Accountability Project, two nongovernmental advocacy
organizations, jointly issued a report examining the extent to
which politics play a role in scientific research. More recently,
over the first few months of 2007, Congress has held several
hearings exploring allegations of political interference with the
work of government climate change scientists.
^2Although these 13 entities are typically bureaus or administrations, for
the purposes of this report we refer to them as "agencies."
A number of federal agencies conduct scientific research that is
used to inform federal policy decisions. This report focuses on
NASA and 2 of the 13 agencies within Commerce--NIST and NOAA.^2
NASA's organic statute--the National Aeronautics and Space Act of
1958, as amended--calls for NASA to "provide for the widest
practicable and appropriate dissemination" of the scientific and
technical information resulting from NASA's research efforts. This
work is structured around four mission areas--aeronautics
research, exploration systems, science, and space operations--and
is performed at 10 research and flight centers across the United
States. Each of these centers performs a variety of research,
engineering, construction, and support functions related to the
mission areas, and each has a separate management structure,
including a center director, to guide its activities. Furthermore,
each center has its own public affairs office that is responsible
for managing the center's media contacts and communications. In
addition, NASA headquarters has its own public affairs office that
oversees agencywide programs and activities to coordinate and
maintain open and credible communication channels to the news
media and the public.
Each of Commerce's 13 agencies is headed by its own director,
undersecretary, or assistant secretary. Even though each of these
agencies has its own public affairs office, Commerce's Office of
Public Affairs is responsible for overseeing public affairs
interactions for the department, such as issuing press releases
and answering media inquiries, as well as for keeping abreast of
the activities at all 13 agencies, many of which disseminate
research results, as is the case with NIST and NOAA. NIST's
mission is "to promote U.S. innovation and industrial
competitiveness by advancing measurement science, standards, and
technology in ways that enhance economic security and improve our
quality of life." To a large extent, this mission is fulfilled by
the research undertaken by scientists, engineers, and technical
experts at the agency's Maryland and Colorado facilities. NIST is
specifically required to disseminate research conducted under
several of its programs. Most research at NIST is conducted at one
of its eight laboratories, covering research areas such as
manufacturing engineering, physics, and information technology. It
is NIST's policy to publish the results of these technical
programs openly, widely, and promptly, and this is accomplished
through scientific reports issued by researchers as well as press
releases and Web-based newsletters issued by NIST's Public and
Business Affairs Office. NOAA is the largest of the Commerce
agencies, and its work is structured around four program
goals--ecosystems, climate, weather and water, and commerce and
transportation--and seven operating organizations, or program
offices.^3 Similar to NASA and NIST, NOAA headquarters has a
public affairs office--the Office of Communications--which is led
by a director and tasked with facilitating media and public
communications with the agency.^4 Each of NOAA's program offices
also has a public affairs officer who reports to the director of
the Office of Communications and serves as the primary public
affairs contact for program office researchers.
^3These program offices are also referred to as "line offices" and include
the Office of Oceanic and Atmospheric Research; the National Environmental
Satellite, Data, and Information Service; the National Marine and Aviation
Operations; the National Marine Fisheries Service; the National Ocean
Service; the National Weather Service; and the Office of Program Planning
and Integration.
^4NOAA's public affairs office, previously called the "Office of Public,
Constituent, and Intergovernmental Affairs," was renamed the "Office of
Communications" in January 2007.
Research dissemination at each of these agencies is guided by
departmental or agency policies. In addition, the Office of
Management and Budget (OMB) has also issued guidelines and
bulletins that have an impact on the dissemination of research
results. For example, in October 2001, OMB issued its Guidelines
for Ensuring and Maximizing the Quality, Objectivity, Utility, and
Integrity of Information Disseminated by Federal Agencies, which
required agencies to issue their own guidelines to ensure the
quality of information being disseminated.^5 NASA, NIST, and NOAA
have all issued their own information quality guidelines.^6 In
addition, OMB issued the Final Information Quality Bulletin for
Peer Review in December 2004 that established governmentwide
guidance establishing that important scientific information shall
be peer-reviewed by qualified specialists before it is
disseminated by the federal government. Among other things, the
bulletin provides guidance to federal agencies on what information
is subject to peer review and requires agencies to begin a
systematic process of peer-review planning for influential
scientific information (including highly influential scientific
assessments) that the agency plans to disseminate in the
foreseeable future.
In addition to the research activities of specific agencies, OSTP
advises the President and others in the Executive Office of the
President (EOP) on the effects of science and technology on
matters being considered in the policymaking process. OSTP serves
as a source of scientific and technological information and advice
for the President with respect to major policies of the federal
government and acts as the scientific and technical arm of the EOP
policymaking process. As part of this work, OSTP and OMB
collaborate to help agencies develop research and development
budgets to address national priorities. OSTP meets regularly with
agency staff as well as representatives of industry, academia, and
the general public to seek or share information about science and
technology policy and leads interagency efforts to develop and
implement science and technology policies.
^5These guidelines implement section 515 of the Treasury and General
Government Appropriations Act for Fiscal Year 2001. Section 515 directs
OMB to issue governmentwide guidelines that "provide policy and procedural
guidance to Federal agencies for ensuring and maximizing the quality,
objectivity, utility, and integrity of information (including statistical
information) disseminated by Federal agencies." Within 1 year after OMB
issued these guidelines, agencies were to issue their own implementing
guidelines. Pub. L. No. 106-554 S 515, 114 Stat. 2763A-154 (2000).
^6Commerce issued its information quality guidelines in October 2002. In
addition, in light of the diversity of the department's mission, these
guidelines directed each of the department's operating units to issue
their own guidelines.
NASA, NIST, and NOAA Policies Generally Facilitate the
Dissemination of Research, but Some Do Not
Researchers at NASA, NIST, and NOAA must comply with both office
specific and agencywide or departmental-level policies, most of
which facilitate dissemination of research. NASA's researchers are
subject to agency-level policies and center-specific policies for
the dissemination of research through publications, presentations,
press releases, and media interviews, and these policies generally
facilitate dissemination. NIST and NOAA have agency-level policies
that generally facilitate the dissemination of research results
through publications and presentations. Researchers at NIST and
NOAA must follow both agency-level policies and guidance and
Commerce's department-level policies if they want to disseminate
their research through media interviews and press releases.
However, NOAA's agency-level and Commerce's department-level
policies may hinder dissemination through these routes.
NASA Policies Provide Clear Guidance to Facilitate Research
Dissemination
At NASA, researchers must comply with both agencywide policies and
center-specific policies when disseminating research results,
regardless of the form this dissemination takes. For example, for
publications and presentations, all researchers are directed to
comply with the agency's policy for approval, publication, and
dissemination of scientific and technical information.^7
Scientific and technical information is defined as the
results--including facts, analyses, and conclusions--of basic and
applied scientific, technical, and related engineering research
and development. This information can be disseminated through a
variety of channels, including NASA publications, outside
journals, presentations at meetings or workshops, and Web sites.
The policy directs that all scientific and technical information
released outside the agency through these means must be reviewed
to determine whether public access to it should be prohibited or
restricted, for national security or intellectual property
reasons. In addition, NASA's policies direct that all research
results that are to be disseminated undergo professional and
technical reviews to ensure that the information is being clearly
communicated, is technically accurate, and meets data quality
standards. These technical review and management approval
procedures vary depending upon the information content,
publication route, and intended audience--domestic or
international. To indicate that a document has completed the
necessary reviews, researchers generally must complete two
forms--one identifying what restrictions, if any, need to be
placed on the document prior to dissemination and the other
indicating that the document has undergone technical peer review.
NASA managers with whom we spoke at the centers said that they
found the agency's policies to be clear, easy to follow, and
similar to procedures researchers follow in the academic
community, although some said that the agency's reviews to
identify security restrictions can be cumbersome at times.
^7NASA, "Requirements for Documentation, Approval, and Dissemination of
NASA Scientific and Technical Information," NPR 2200.2B, March 25, 2005.
Regarding dissemination of research through media interviews and
press releases, researchers at NASA are subject to the agency's
policy on the release of information to news and information
media, hereafter referred to as the "media policy." This policy,
which was revised in March 2006, governs the release of
information to the media, especially information with the
potential to generate significant media or public interest,
including press releases, media advisories, news features, and Web
postings.^8 According to NASA, the media policy in place prior to
March 2006 had not been substantively modified since 1987 and
required clarification to be both useful and practical to
implement. Furthermore, the NASA public affairs policy review
team, composed of representatives from throughout NASA's
scientific and public affairs offices, found that the previous
policies were convoluted, bureaucratic, and resulted in a
breakdown between researchers and public affairs staff.
We found that NASA's revised 2006 media policy is generally clear
and should help facilitate dissemination of research results by
devolving decision making and providing a process to resolve
disputes. The policy, its accompanying "Frequently Asked
Questions" guide, and statements by the NASA Administrator assert
NASA's commitment to a culture of openness with the media and the
public and affirm that the agency values the free exchange of
ideas, data, and information as part of scientific and technical
inquiry. The policy defines the roles, responsibilities, and
methods of coordination for managers, researchers, and public
affairs staff and lays out guidelines for working with the public
affairs office, clearly stating what public affairs officials can
and cannot do. Specifically, regarding media interviews, the
updated policy clearly affirms that NASA employees may speak to
the media and public about their work without prior approval;
however, they must notify their immediate supervisor and their
center public affairs office immediately thereafter. While
researchers are encouraged to have a public affairs official
present during interviews, it is not required and the policy
clearly defines the role of public affairs staff as one of
logistical support and clarification, not interference. Finally,
the policy states that NASA employees may communicate conclusions
on the basis of their research to the media. However, NASA
employees who present personal views outside of their official
area of expertise or responsibility must make clear that they are
presenting their individual views and not those of the agency.
^8NASA, "NASA Policy on the Release of Information to News and Information
Media," 14 C.F.R. Part 1213.
In addition to clarifying the policy for conducting media
interviews, NASA's revised media policy also describes the process
for dissemination via press releases or other media items, such as
news features, media advisories, or news-related Web postings.
Specifically, the policy, augmented by accompanying operating
procedures, clearly outlines the coordination, review, and
approval process that is required prior to issuing a press
release. While some public affairs officials and managers told us
that some of the reviews for press releases can require more time
than available to meet the deadlines of the news cycle, they also
felt that the revised policy had improved the approval process. In
addition, the policy specifically delegates authority to NASA
centers, independent of headquarters, to issue public information
that is of local interest, among other things. Items eligible for
issuance by the centers include releases on specific research
topics that have a targeted audience, such as the development of a
new "superplastic" that would be of interest to people who use
polymers, or an announcement of an upcoming lecture series to be
held at the center. Delegating authority to the centers to issue
their own press releases allows NASA to better publicize the work
it does to targeted audiences of interest. Finally, the revised
media policy describes the process for researchers to use when
they disagree with the agency's decision regarding whether to
issue a press release or another type of public information. The
policy allows such appeals to be elevated to the Office of the
Administrator for resolution, if needed.
NIST and NOAA Policies Provide Clear Guidance to Facilitate the
Dissemination of Research through Publications and Presentations
NIST's publication and presentation policy, called the Technical
Communications Program, generally facilitates the dissemination of
scientific results. Similar to NASA's publications policy, NIST's
policy clearly describes the extent and type of reviews for
technical and scientific accuracy that are required before
publications and presentations may be released outside the agency.
For example, abstracts and outlines for talks that are not of a
sensitive nature and will be limited in distribution can be
approved by the researcher's division chief and do not need to go
through a more extensive review. However, manuscripts for journals
and books that are intended for public distribution must be
reviewed and approved by the laboratory to which the researcher
belongs as well as the NIST Editorial Review Board. This board,
composed of NIST technical authors and editors from all
organizational units, conducts a critical evaluation of the
technical content and methodology, among other things. If the
review board does not support publication, it generally recommends
changes, which if made by the author, would result in approval. If
the author chooses not to accept these changes, the policy
delineates the steps that the author should take to resolve any
disagreement. NIST managers said that the guidance and procedures
for getting potential publications through the editorial review
process and disseminated were adequate, clear, and readily
accessible to researchers on the internal NIST Web site.
Similar to NIST, NOAA's publication and presentation policy
facilitates research dissemination and clearly describes the
process for disseminating scientific and technical publications.
Specifically, NOAA's policy encourages each program office to
review, edit, and disseminate its own publications, and the policy
clearly delegates responsibility to each program office to ensure
the scientific and technical quality of its publications and
presentations.^9 Furthermore, NOAA has developed a standard form
to guide the review process and to document final approval for
release. In addition, several NOAA program offices have
supplemented these agencywide policies with office-specific ones.
For example, the National Weather Service's policy clearly
delineates the review and approval process for their
publications.^10 Similarly, the National Environmental Satellite,
Data, and Information Service created an office-specific form that
documents the multiple reviews a proposed manuscript must undergo.
This form, signed by the branch chief, division chief, and office
director or deputy director, certifies that a research paper is
worthy of scientific review or useful for education or outreach
purposes. Several NOAA managers told us that NOAA's publication
review process was clear and worked well.
^9NOAA, "Scientific and Technical Publications," NAO 201-32G, January 27,
1993.
^10National Weather Service, "Clearances for NWS Employee Papers,"
National Weather Service Policy Directive 100-1, December 10, 2002 and
"Science Review and Approval," National Weather Service Policy Directive
80-5, April 8, 2004.
Researchers at NIST and NOAA Have to Comply with Commerce and
Agency-Specific Policies for the Dissemination of Research through
Media Interviews and Press Releases
Researchers at NIST and NOAA are subject to Commerce's
department-level policies when disseminating research results
through media interviews and press releases; however, these
policies are outdated and unrealistic and can impede the
dissemination of research results. At NIST, researchers have been
provided with agency-specific guidance to supplement Commerce's
department-level media interview and press release policies. At
NOAA, researchers must also adhere to agency-level media and press
release policies, which are generally unclear and can lead to
inconsistent interpretations that can further hinder the
dissemination of research through these dissemination routes.
Researchers at NIST and NOAA Are Subject to Outdated Commerce
Policies for Media Interviews and Press Releases
Researchers at NIST and NOAA must follow Commerce's
department-level policies for dissemination of research results
through media interviews and press releases; however, these
policies, which have not been revised for over 20 years, are
outdated and unrealistic.^11 Commerce policies direct all of its
constituent agencies to submit all proposed media interviews and
press releases to its Office of Public Affairs for review and
approval, and the department does not delegate authority to
approve any of these activities to the agency-level public affairs
office. Because Commerce's public affairs office is responsible
for overseeing the media activities for 13 agencies, according to
Commerce officials, it is not realistic to expect this office to
review every media interview and press release request that each
of these agencies generates. For example, from a large agency like
NOAA, the volume of media requests can range from dozens on a
normal business day to hundreds following a major event, such as a
tsunami. In addition, Commerce's policy states that disputes
concerning a potential media interview or press releases that are
not resolved by the Director of Public Affairs shall be referred
to the Secretary of Commerce.
At a February 2007 Senate hearing, a Commerce official referred to
the department's communication policies as "contradictory" and
"woefully outdated." These policies, which have not been revised
since 1983, were implemented prior to the expansion and widespread
adoption of newer media outlets, such as the Internet and 24-hour
cable news stations, and therefore do not take these routes of
dissemination into consideration. While officials from Commerce's
Office of Public Affairs said that they would like to be informed
about all press releases and interviews, they realize that this
request is not realistic given the volume of releases from some
agencies. Consequently, Commerce public affairs officials
acknowledged that even though it is required, they do not
generally review low-level or routine releases.
^11Department of Commerce, "Outside Writing and Speaking; Departmental
Approval," DAO 219-1, April 1, 1982; and "Release of News, Clearance of
Speeches and Publications, and Media Coverage," DAO 19-2, November 3,
1980.
To streamline the approval process and respond to allegations by
some researchers about the department's interference with media
dissemination of research results, Commerce began an effort in
November 2005 to revise its media and communications policies.
According to a Commerce official, this effort included three
rounds of input. During the first round, Commerce solicited
informal feedback from managers and some scientists at selected
agencies, including at NIST and NOAA. As part of the subsequent
two rounds, Commerce solicited input from managers and public
affairs officials at all Commerce agencies. In addition, these
officials were encouraged to share the draft with researchers and
other public affairs officials within their agency to obtain their
feedback and input. We reviewed a draft of the revised policy in
March 2007. Because the final policy was released after our audit
work was completed, we did not conduct a detailed evaluation of
the new policy. However, we did note that the revised policy
included provisions to address some of the concerns we identified
with the prior policy. For example, the revised policy affirms the
department's support for open and free communication of scientific
and technical ideas, findings, and conclusions based on
researchers' official work and asserts that researchers are free
to participate in interviews, without prior approval, on matters
directly related to their research. In addition, the revised
policy takes other positive steps, such as the inclusion of an
appeals process and the clarification of the role of the public
affairs office. However, it also contains some unclear elements
that may cause confusion for the agencies and researchers who are
trying to adhere to the policy. The policy was officially released
on March 29, 2007, and, according to a Commerce official, it will
be accompanied by a training program to educate public affairs
officials on how to implement it.
Researchers at NIST Have Been Provided with Supplemental Agency
Guidance
To supplement Commerce policies for dissemination of research
results through media interviews and press releases, NIST has
issued Web-based guidance that provides an overview of the
Commerce policy and serves as a reference tool for NIST
researchers who interact with the media. The guidance on
responding to media inquiries explicitly states that interview
requests from the media are to be cleared in advance through
Commerce's Office of Public Affairs. In addition, this guidance
provides information on Commerce's media policy in a
question-and-answer format, including instructions that
researchers should discuss only the research facts and should not
express personal opinions when communicating with the media.
NIST managers commented that while NIST issues between 30 and 50
press releases a year, they have found that other dissemination
methods, such as e-mailing a biweekly publication called Tech
Beat, have been a more effective way to reach the media. In fact,
the managers estimated that 60 percent of the news articles about
NIST research are generated from Tech Beat articles. Like press
releases, Tech Beat articles go through the Commerce review and
approval process, and they are subject to NIST technical and
scientific review. NIST officials with whom we spoke said that
while they forward most proposed interview requests to Commerce
for review and approval, they generally do not seek approval for
interviews that result from a press release or from a Tech Beat
article previously approved by Commerce.
Researchers at NOAA Are Subject to Unclear Agency-Level Media
Interview and Press Release Dissemination Policies
In addition to adhering to Commerce's policies for media
interviews and press releases, researchers at NOAA must also
follow NOAA-specific policies and procedures, which are at times
unclear, leading to inconsistent interpretation. For example,
regarding media interviews, one part of NOAA's policy directs its
employees to inform NOAA headquarters or program office public
affairs officials prior to participating in interviews that are of
national news interest, concern regulatory or controversial
issues, or pertain to research having potential policy interest;
however, another part of this policy states that responsibility of
the NOAA headquarters' public affairs office is to approve and
coordinate media communications, including interviews and press
releases.^12 As a result of this unclear language, public affairs
officials have interpreted these requirements differently. Some
believe that researchers are required to obtain prior approval for
all interviews, while others believe researchers must notify the
public affairs office after an interview has occurred. This has
resulted in an uneven application of the policy among researchers.
Furthermore, the policy does not define what qualifies as "of
national news interest" or "controversial," thereby leaving the
interpretation to each individual.
^12NOAA, "NOAA Media Policy," NAO 219-6, June 22, 2004.
In addition, some of NOAA's procedures required for media
interview approvals can be burdensome and could delay
dissemination of research. To seek approval for a media interview,
NOAA program office public affairs staff are expected to complete
a form that contains information about the topic, the potential
questions, the media outlet, and the reporter, among other things.
The completed form is forwarded to the headquarters' public
affairs office for review and approval. However, while interview
approvals typically occur within a few hours, in some cases they
have arrived after the reporters' deadlines, resulting in missed
interviews, according to NOAA public affairs officials.
Furthermore, some of these officials expressed concern that
reporters may shy away from contacting NOAA researchers because of
the drawn out approval process, ultimately causing NOAA to miss
opportunities to inform the public about its work.
Regarding press releases, NOAA public affairs officials have also
interpreted approval requirements differently from one another.
For example, some officials believe that the policy requires all
press releases to be approved by the NOAA headquarters' public
affairs office, while others believe headquarters approval is not
required for releases of regional interest or involving routine
activities. NOAA's press release policy, unlike NASA's policy,
does not distinguish between press releases on local topics and
press releases on national topics and does not delegate authority
to program office public affairs officials to issue such press
releases. However, according to public affairs officials with whom
we spoke, routine releases may not warrant the same scrutiny as
ones likely to be of national interest or controversial. For
example, a NOAA public affairs official commented that routine
releases about StormReady--a community safety and emergency
preparedness program--should be treated differently from press
releases that have a broader scope, such as annual releases of
climate data.
According to NOAA public affairs officials, the approval process
for press releases is lengthy and burdensome, and approvals that
took 1 to 2 days a few years ago can now take as long as 2 weeks
to obtain. This is because the process involves 13 separate steps
for approval that begin after the press release has been drafted,
which itself can be a lengthy process. (See fig. 1.) In November
2006, another requirement was added to the approval process--that
is, a public affairs briefing memorandum must precede certain
requests to issue a press release, such as those involving a
controversial or "hot" topic. According to a senior NOAA public
affairs official, the intent of requiring the memorandum was to
inform NOAA management about upcoming announcements and help
ensure that the program office public affairs staff had gathered
enough information about the proposed release to judge its merit.
Figure 1: NOAA/Commerce News Release Review Process, as of March 2006
Note: OPCIA is also referred to as the NOAA headquarters office of public
affairs.
The myriad steps in the approval process can hinder the timely issuance of
press releases. As was the case with delays in approving interviews,
officials told us press releases that come out too late are in danger of
not making it into the news and cited several instances in which requests
to issue press releases never received an approval or a denial from
headquarters public affairs and, consequently, were never released.
Moreover, no explanation for the lack of response was offered to the
public affairs office or the researcher. Furthermore, officials at a NOAA
laboratory have become discouraged from seeking press releases for
research conducted at the laboratory because of delays and the lack of
responsiveness to such requests, further limiting NOAA's ability to
publicize its research.
Despite Agencies' Efforts to Communicate Research Dissemination and Dispute
Resolution Policies, Many Researchers Are Not Confident about How to Comply with
Them
According to our survey, researchers at NASA, NIST, and NOAA usually
learned about their agency's dissemination policies through informal
communication methods, rather than through formal training provided by
their agency. In addition, more researchers at the three agencies are
confident that they understand their agency's policies on the
dissemination of research through publications and presentations well
enough to comply with them, than are confident that they understand their
agency's media interview and press release policies. Furthermore, many
researchers at NASA, NIST, and NOAA are unsure whether their agency's
policies allow them to discuss research results with potential policy
implications or to express personal views related to federal policy
decisions. Finally, we estimate that only one-quarter of all researchers
across the three agencies are aware of their agency's processes to appeal
decisions made regarding requested dissemination.
Most Researchers Learned about Agency Dissemination Policies through Informal
Methods
According to our survey and agency officials with whom we spoke,
researchers at NASA, NIST, and NOAA were most likely to learn about their
agency's dissemination policies through informal communication methods.
Most researchers at each agency learned about these policies through a
variety of informal means, such as e-mails from their agency or operating
unit managers, or through on-the-job training with supervisors.
Researchers were less likely to learn about agency dissemination policies
through formal training sessions, a method that agency officials told us
was typically offered on a limited basis focused toward specific groups,
such as those likely to have media contact. Specifically:
o At NASA, we estimate that three-quarters of researchers learned
of their agency's dissemination policies through e-mails or other
correspondence from center or office managers, 70 percent through
on-the-job training, and about one-half through notifications on
internal agency Web sites. Managers and public affairs officials
said that limited training on dissemination of research through
publications, presentations, and media interviews is available. At
most centers, managers said that training on the publication and
presentation review process is done on the job by supervisors or
managers, although some centers have conducted limited formal
training on how to comply with export control regulations,^13
which are a required segment of NASA's scientific and technical
review process. Several NASA center public affairs officials said
they have also conducted limited formal training on media
interviews, but this training was usually targeted toward
researchers who are expected to have significant exposure to
reporters. Headquarters and several center officials told us that
little or no formal training had been conducted on NASA's recently
revised media policy.
^13Export Administration Regulations ("EAR") and International Traffic In
Arms Regulations ("ITAR") control the export of, among other things,
certain technology and technical data to foreign countries.
o At NIST, officials told us that they generally rely on the
agency's internal Web site to inform employees about NIST's
publication and presentation policies; however, they mentioned
that more specific, individual training is offered on an as-needed
basis. Media and communications training targeted toward
researchers with high media exposure is offered periodically,
covering the approval processes for media interviews and press
releases. However, according to our survey estimates, nearly 80
percent of researchers received information about their agency's
policies through on the job training, and nearly two-thirds
learned about them through e-mail communication from the agency.
Additionally, about one-half of researchers at NIST learned of the
dissemination policies through notifications on internal agency
Web sites or at staff or town-hall meetings; and fewer (about 24
percent) attended formal training sessions to learn about the
policies.
o At NOAA, researchers most often learned of the agency's
dissemination policies through informal methods, such as e-mails
or other correspondence from center or office managers (75
percent), on-the-job training (59 percent), and notifications on
internal agency Web sites (37 percent). These sentiments were
echoed by NOAA managers who told us that they typically inform
researchers about the agency's policies on dissemination of
publications and presentations through on-the-job training, on an
as-needed basis. They added that researchers are usually exposed
to standard scientific peer review processes during graduate
school, and that these processes are similar to the agency's
review processes. Regarding media interviews, at least one public
affairs branch office offered a formal training course that
included an overview of NOAA requirements in this area and offered
advice on effectively communicating with the media. According to
program office officials, this training was well received by those
who participated.
Figure 2 shows the methods by which researchers at NASA, NIST, and
NOAA generally learned about their agency's research dissemination
policies.
Figure 2: Estimated Percentage of Researchers Who Learned How to
Comply with Dissemination Policies through Various Methods
Researchers Have More Concerns about Adhering to Their Agency's
Media Interview and Press Release Policies than to Their Publication
and Presentation Policies
Most researchers at NASA, NIST, and NOAA feel confident that they
know how to comply with their agency's policies for publications
and presentations, but many do not feel as confident that they
know how to comply with their agency's policies on media
interviews and press releases. (See fig. 3.) According to our
survey, we estimate that more than 80 percent of researchers at
each agency are confident that they understand their agency's
policies on publications and presentations well enough to comply
with them. At NIST, about three-quarters of researchers are also
confident that they know how to follow their agency's policies on
media interviews and press releases. In contrast, nearly one-half
of researchers at NASA and over one-third of researchers at NOAA
are not confident that they understand their agency's policies on
media interviews well enough to follow them, and 40 percent of
NASA researchers and 35 percent of NOAA researchers are not
confident they understand their agency's policies on press
releases.
Figure 3: Estimated Percentage of Researchers Who Are Confident
That They Understand Policies Well Enough to Follow Them
Researchers Are Often Unclear about Whether They Can Discuss
Research Results with Potential Policy Implications or Express
Their Personal Views
Many researchers at NASA, NIST, and NOAA are unclear about whether
their agency allows them to discuss research results with policy
implications. This is particularly striking at NASA and NOAA,
given that the leaders of these two agencies have each assured
their researchers that they may discuss the policy implications of
their research. When research results are consistent with their
agency's policy or position statements, we estimate that
two-thirds of researchers across NASA, NIST, and NOAA believe that
they are allowed to discuss these potential policy implications.
However, when research results differ from their agency's policy
or positions statements, 23 percent of researchers across all
three agencies believe they may not discuss those results, and 40
percent are not sure. In addition, even though NASA's recently
revised media policy clearly states that researchers are allowed
to discuss conclusions drawn from their work with the media as
long as they disclose them as personal views and not views of the
agency, many researchers are unsure whether they are allowed to
discuss personal views related to potential policy implications of
their research results. On the basis of our survey, we estimate
that only one-half of researchers at NASA believe that they may
make such statements, and another 40 percent are not sure whether
they can.
Unlike NASA researchers, researchers at NIST are not allowed to
discuss their personal views on the policy implications of their
research, even if they disclose that the views are their own and
not those of their agency. This is because NIST and Commerce
policies state that when speaking publicly on topics related to
research conducted at the agency, it is not realistic for a
researcher to express personal views without the perception that
the employee is speaking on behalf of the agency. Nevertheless, we
estimate that 24 percent of researchers at NIST believe that they
are free to discuss potential policy implications of their
research provided they identify such views as their personal
opinions and not those of the agency, and 49 percent of
researchers at NIST are not sure whether they can discuss their
personal views.
NOAA researchers, like NIST researchers, are also subject to
Commerce's policy that restricts their ability to discuss their
personal views. However, in a February 2006 e-mail to NOAA
employees, the NOAA Administrator said that communicating personal
views was allowed provided that the researcher stated the views
were the researcher's opinion. Despite the clarifying instruction
from the administrator, we estimate that 42 percent of NOAA
researchers are uncertain whether they may discuss personal views
with the media, and another 40 percent believe they may do so if
they offer a disclaimer.
Researchers Are Generally Unaware of Their Agencies' Process for
Appealing Dissemination Decisions
According to our survey, only 33 percent of NASA researchers, 26
percent of NIST researchers, and 17 percent of NOAA researchers
are aware of their agencies' processes to address disputes over
dissemination requests. Even at NASA, where officials told us that
the agency rolled out its revised media policy, which includes a
dispute resolution process, with great fanfare, we estimate that
only one-third of researchers are aware of the appeals process for
dissemination of research. According to NASA headquarters and
center public affairs officials with whom we spoke, no one, to
date, has attempted to use the dispute resolution process for
media-related concerns. For other types of dissemination, such as
through publications, center managers generally told us that
formal processes for appeal did not exist; however, most managers
said that a researcher would likely raise concerns through the
chain of command or rely on other agency processes, such as those
for personnel disputes. Most NASA center managers told us that
disputes over disseminating scientific and technical information
are rare, and some managers said that they had never needed a
formal process to resolve disputes.
NIST managers told us that the NIST technical review policy
delineates a process for appeals of dissemination decisions for
publications and presentations. However, according to our survey,
we estimate that only 26 percent of researchers at NIST are aware
that such an appeals process existed. NIST officials said that no
disputes over the dissemination of information via publications
had occurred in the past 5 years. Regarding media contacts,
Commerce officials said existing Commerce policies include a
process to handle disputes. However, these officials acknowledged
that the process may not have been clear to staff, and they plan
to clarify this process in future policies.
At NOAA, we estimate that 17 percent of researchers are aware of
the agency's appeals process. NOAA managers told us that the
agency's policies do not include an appeals process. The managers
said that if a dispute arose, they expect the researcher to appeal
the decision through the chain of command or use the agency's
alternative dispute resolution process, which involves mediation
of the dispute by a neutral third party, often from another NOAA
program office.
Six Percent of Researchers at NASA, NIST, and NOAA Had Dissemination
Requests Denied, and Many Believe That Their Agencies Are Less
Supportive of Media Interviews than Other Dissemination Methods
On the basis of our survey, we estimate that across NASA, NIST,
and NOAA, 6 percent of researchers were denied the opportunity to
disseminate their research results in the past 5 years, sometimes,
they believe, without explanation. In general, however, most
researchers at these agencies believe that their agency either
insists on or encourages dissemination of research results. At the
same time, significantly more researchers at NASA, NIST, and NOAA
believe that their agency supports dissemination through
publications and presentations rather than interviews with the
media. Finally, most researchers at NIST and NOAA believe that
their agencies consistently applied the dissemination policies
across all routes of dissemination. In contrast, many researchers
at NASA believe that policies for press releases and media
interviews are not applied as consistently as its policies for
publications.
Some Researchers at Each Agency Who Were Denied Approval to
Disseminate Their Results Believe They Were Not Provided with an
Adequate Explanation
Six percent of the approximately 5,000 researchers, or about 200,
across the three agencies have had their requests for
dissemination denied for reasons other than those stemming from
standard technical review. According to our survey, we estimate
that 7 percent of NASA's 1,794 researchers, 4 percent of NIST's
1,337 researchers, and 6 percent of NOAA's 1,815 researchers have
in the past 5 years been denied the opportunity to disseminate
their research.^14 The denials involved a wide range of research
topics and occurred across all routes of dissemination, within
each NOAA branch office, and within nearly all of the NASA centers
and NIST laboratories included in our survey.^15
^14The number of researchers listed for each agency represents those who
met our criterion of being federally employed scientists, engineers, or
other researchers who are in a position to disseminate their research
results to a wider audience. We estimate with 95 percent confidence that
the total estimated number of researchers across all three agencies who
were denied is 5.9 percent, or 214 researchers, with a confidence interval
of 4.5 percent (162 researchers) to 7.8 percent (282 researchers). At
NASA, the estimated number of researchers denied is 7.0 percent (102
researchers), with a confidence interval of 4.6 percent (68 researchers)
to 10.4 percent (151 researchers). At NIST, the estimated number of
researchers denied is 3.7 percent (37 researchers), with a confidence
interval of 2.3 percent (23 researchers) to 5.9 percent (58 researchers).
At NOAA, the estimated number of researchers denied is 6.4 percent (76
researchers), with a confidence interval of 3.8 percent (45 researchers)
to 10.7 percent (127 researchers). For additional details on our survey
methodology, see appendix I.
Researchers at each agency were given a variety of reasons for the
denials, but some indicated that they were given no reason. For
NASA researchers, representing a variety of areas, including
aeronautics; biological or life sciences; and climate,
environment, or atmosphere research, among the most common reasons
reported for denials was that the information was restricted or
that the topic discussed or results presented were sensitive. The
subject of some restricted research involved government classified
information, or results that were covered by export control
regulations. For example, one researcher reported his request for
dissemination of research was denied because of restrictions on
dissemination of high frequency communications technology. As a
result of agency denials, most of these NASA researchers gave up
trying to disseminate these particular results, although some
subsequently disseminated the results of their work using a
different dissemination route.
Researchers at NIST who had their requests denied also represented
a range of research areas, including information technology,
electrical engineering, and physical or chemical sciences. Many of
these researchers responded that they either were not given a
reason for the denial or did not know the reason. In these
situations, most researchers either found another route to
disseminate these particular research results or gave up trying.
Other researchers who had requests denied at NIST reported that
the reasons given for the denial included that the results or
topic was sensitive or that there was a question about the
scientific merit of the results. In some of these cases, the
researchers were able to disseminate their results once they added
a disclaimer that the opinions expressed in the research results
did not reflect the views of the agency.
At NOAA, researchers who had requests denied represented a diverse
cadre of research areas, including climate, environment, or
atmosphere; oceans and coasts; and fisheries and ecosystems. Among
the most common reasons that researchers reported for the denial
of their requests to disseminate research were that the topic or
results were sensitive and that resources, such as money for
travel, were limited. For example, one researcher reported that
his request was denied because there was concern that the results
might be misinterpreted in light of recent hurricane events. In
addition, like NIST researchers, many NOAA researchers reported
that they were not given a reason for the denial or did not know
the reason. Those who indicated they were denied for budgetary
reasons had requested funding to travel to a conference, and
reported that after the denial they disseminated their work
through another route. Those who said their requests were denied
because their results were considered sensitive took various steps
to obtain subsequent approval to disseminate their results,
including resubmitting the same or a similar document or seeking
an alternative dissemination route. Of the researchers who either
did not know the reason for denial or were not given one, most
gave up trying to disseminate these particular results.
^15While the number of denials can be generalized to the population, the
reasons provided for the denial, the actions taken as a result, and the
field of research cannot be generalized to the population. Where we
describe the reasons for the denials and actions taken as a result in this
section, the information only represents the sentiments of individual
respondents and not the population of the researchers across the three
agencies.
Researchers Believe That Their Agencies Provide Greater Support
for Dissemination of Research through Publications and Presentations
than through Press Releases and Media Interviews
While researchers at NASA, NIST, and NOAA believe that, overall,
their agencies support dissemination of research results, many
believe that their agency is more supportive of some dissemination
routes than others. We estimate that at least 85 percent of
researchers at each agency believe that, overall, their agency
either encourages researchers to disseminate their research
results or insists that they do so. However, we also found that
researchers believe that their agency is more supportive of
sharing results within the scientific community through
publications, such as peer-reviewed journals, and presentations,
than sharing results with the general public through media
interviews or press releases. Specifically, we estimate that 91
percent of researchers at NASA and NOAA and 97 percent of
researchers at NIST believe that their agency supports
dissemination of research results through publications. However,
significantly fewer researchers at each agency believe their
agency supports dissemination through press releases, ranging from
73 percent of researchers at NOAA to 87 percent at NIST. (See fig.
4.) Similarly, regarding dissemination through media interviews,
significantly fewer researchers, ranging from 54 percent at NOAA
to 68 percent at NIST, believe that their agency is supportive of
this dissemination route.
Figure 4: Estimated Percentage of Researchers Who Believe That
Their Agency Supports Dissemination, by Dissemination Route
Researchers echoed these sentiments in their comments on ways the
dissemination of research results works well and how it could be
improved. Many researchers responded that agency support for
dissemination was good, particularly for scientific publications
and presentations, but could be improved for press releases and
media interviews. For example, one NIST researcher commented that
the agency has a strong cultural belief in the technical integrity
of peer-reviewed journals compared with general news media
outlets. A NOAA researcher said that although his division
supported and encouraged original, cutting-edge research and
dissemination through scientific literature, he did not believe
that NOAA or Commerce effectively disseminate research results to
the public or the Congress. Moreover, a representative of a NOAA
laboratory told us that although the laboratory issues over 100
peer-reviewed articles every year, only a few are publicized
through press releases. In addition, representatives of one NASA
center believe that the role of public affairs has shifted from
help to hindrance, and that some requests for press releases
announcing contentious research results had not been approved by
headquarters, sometimes without explanation. In addition, several
agency managers told us that support for dissemination depends on
the tone set by senior managers. More specifically, one manager
said that without commitment and support from agency leaders, even
good policies will not ensure that important research gets
disseminated.
Most Researchers Generally Believe That Dissemination Policies Are
Consistently Applied
According to our survey, over 72 percent of researchers at NIST
and NOAA believe that their agency consistently applies its
dissemination policies across all dissemination routes. In
contrast, more researchers at NASA believe that the agency
consistently applies its policies for publications than believe
the agency consistently applies its policies for press releases
and media interviews. We estimate that only 67 percent and 57
percent of researchers believe NASA consistently applies its press
release and media interview dissemination policies, respectively,
as opposed to 84 percent and 74 percent, respectively, of
researchers who believe the agency consistently applies its policy
for publications and presentations.^16 (See fig. 5.)
In contrast, most researchers at NIST and NOAA believe their
agencies consistently apply all of their dissemination policies.
According to our survey, over 85 percent of the researchers within
NIST and approximately 75 percent of the researchers at NOAA
believe their agencies consistently apply their dissemination
policies for publications, presentations, press releases, and
media interviews.
^16There was no statistical difference between presentations and press
releases.
Figure 5: Estimated Percentage of Researchers Who Believe That
Their Agency Consistently Applies Policies, by Dissemination Route
OSTP Provides Indirect Oversight of Agencies' Dissemination
Policies
According to OSTP, the office does not conduct any scientific
research on its own nor does it formulate or directly oversee the
development of dissemination policies or decisions at individual
agencies. However, OSTP told us that it has affirmed to agency
leaders the value of science as the basis for effective federal
action and recognizes the importance of timely, complete, and
accurate communication of scientific information. According to
OSTP, the director of the office has on several occasions asked
the leaders and chief scientists of federal agencies to develop,
revise, or reemphasize their dissemination policies and to ensure
that both employees and managers understand their rights and
obligations under the policies. For example, in an April 2006
letter that transmitted copies of NASA's policy to agency chief
scientists, the director praised NASA's media policy as a model
and cited the Frequently Asked Questions supplement as a best
practice. In addition, according to OSTP, the director instructed
agencies that federal scientists, like federal employees
generally, are obliged to distinguish their personal views from
the official positions of their agencies but are in no way to be
restricted in their ability to openly communicate their scientific
findings.
With regard to disseminating the results of interagency research,
OSTP sometimes plays a role in notifying agencies when research
reports of interest are published by another agency, and it will
disseminate research results to members of various National
Science and Technology Council committees, subcommittees, and
working groups, according to OSTP officials. For example, to
ensure accuracy and readability, OSTP reviews interagency reports
on the scale, quality, and effectiveness of the federal science
and technology effort as well as all reports from interagency
working groups and task forces. In addition, OSTP may, at its own
initiative or in response to an agency's request, review
presentations developed by individual agencies that are based on
interagency reports. According to OSTP, the office is part of the
interagency review process when federal scientists testify before
Congress, and, accordingly, OSTP officials review all of the
statements prepared by federal scientists who are testifying in
their official capacities. Regarding presentations and articles
prepared by OSTP employees, OSTP officials told us that the office
has a process by which these presentations and articles are
reviewed and for making decisions about whether its employees can
participate in policy-related media interviews.
In discussing OSTP's role with NASA, NIST, and NOAA public affairs
officials, we found that some of these agencies may inform OSTP of
certain dissemination events, but their policies do not call for
routine OSTP review or approval. For example, Commerce and NOAA
public affairs officials told us that OSTP often receives
informational copies of press releases that are particularly
sensitive or involve regulatory issues.
Conclusions
More and more of the major policy debates of the day hinge on the
results of scientific research. Therefore, timely and thorough
dissemination of research results within the research community
and to the public at large is crucial. To help ensure that this
kind of dissemination is taking place, NASA, NIST, and NOAA have
each taken some steps to develop policies to guide the
dissemination of research through publications and presentations
for the scientific community. However, the lack of clarity and
consistency in the application of agency-level policies for
requests to disseminate research to the public through press
releases and media interviews continues to generate significant
concern among some agency researchers, such as those at NASA. For
researchers at NIST and NOAA, this situation is further
exacerbated because they must comply with Commerce's outdated and
unrealistic department-level policies, which can further impede
the dissemination of research results through media interviews and
press releases. Moreover, when disputes arise about an agency's
decision regarding a dissemination request, few researchers are
aware of the dispute resolution processes that exist within their
agency. To some extent, this situation may have been partly caused
by a lack of formal training provided by the agencies on the
dissemination and dispute resolution policies to their research
staff. Instead most researchers at these agencies learn about
their agency's policies through informal means and, therefore, are
not particularly confident that they know how to comply with all
of them.
Recommendations for Executive Action
To ensure that the policies to guide researchers and public
affairs officials in their efforts to disseminate research to the
scientific community and the public are clear, transparent,
consistently applied, and completely understood, we recommend that
the Secretary of Commerce and the NOAA Administrator each take the
following action:
o Clarify their policies for disseminating research results via
press releases and media interviews.
To ensure that (1) researchers have a clear process to follow for
appealing decisions regarding dissemination of research results
and (2) all researchers are aware of and understand how to comply
with their agencies' policies for disseminating research, we
recommend that the Secretary of Commerce, the NASA Administrator,
the NIST Director, and the NOAA Administrator each take the
following two actions:
o Review their dissemination policies and ensure that they clearly
identify a process by which researchers can appeal dissemination
decisions.
o Provide formal training to inform, reinforce, and update
managers, researchers, and public affairs staff on these policies.
Agency Comments and Our Evaluation
We provided Commerce, NASA, NIST, NOAA, and OSTP with a copy of
this report for review and comment. Commerce, on behalf of NIST,
NOAA, and itself, generally concurred with our findings and
recommendations. In its letter, Commerce noted that the draft
report highlighted some ongoing challenges for the department,
NIST, and NOAA that they will address. Furthermore, in an addendum
from NOAA, the agency stated that it is updating its public
communications policies to ensure consistent understanding by
researchers and public affairs officials. In addition, NIST
suggested some technical changes that we have incorporated in this
report as appropriate. NASA also concurred with the
recommendations we made to the agency. In its letter, NASA stated
that it plans to direct the Chief of Strategic Communications and
the Assistant Administrator for Public Affairs to, among other
things, review its current dissemination policies to ensure they
clearly identify a process for appealing dissemination decisions
and to provide biannual training to reinforce dissemination
guidelines. Finally, OSTP generally agreed with the report's
recommendations made to Commerce, NASA, NIST, and NOAA. The office
commented that the report is fair and accurately describes OSTP
and its role in this area. The comment letters from Commerce,
NASA, and OSTP appear in appendixes III through V.
As agreed with your office, unless you publicly announce the
contents of this report earlier, we plan no further distribution
until 30 days from the report date. At that time, we will send
copies to interested congressional committees and Members of
Congress, the Secretary of Commerce, the NASA Administrator, the
NIST Director, and the NOAA Administrator. We also will make
copies available to others upon request. In addition, the report
will be available at no charge on the GAO Web site at
http://www.gao.gov.
If you or your staff have questions about this report, please
contact me at (202) 512-3841 or [email protected]. Contact points
for our Offices of Congressional Relations and Public Affairs may
be found on the last page of this report. GAO staff who made key
contributions to this report are listed in appendix VI.
Sincerely yours,
Ms. Anu Mittal
Director, Natural Resources and Environment
Appendix I: Objectives, Scope, and Methodology
Our objectives for this review were to (1) identify and evaluate
the policies that guide the dissemination of federal research
results at the National Aeronautics and Space Administration
(NASA), the National Institute of Standards and Technology (NIST),
and the National Oceanic and Atmospheric Administration (NOAA);
(2) determine how effectively the dissemination and dispute
resolution policies of these agencies have been communicated to
researchers; (3) determine the extent to which researchers at
these agencies have experienced restrictions on the dissemination
of their research results, and (4) provide an overview of the role
of the Office of Science and Technology Policy (OSTP) in helping
agencies develop and implement policies related to the
dissemination of research results.
To identify and evaluate the policies that guide the dissemination
of federally funded research results at NASA, NIST, and NOAA, we
obtained, reviewed, and analyzed the dissemination policies for
these agencies, in addition to relevant policies from the
Department of Commerce. In addition, we spoke with managers and
researchers at NASA, NIST, and NOAA as well as public affairs
officials at each of these agencies and Commerce. Specifically, at
NASA, we spoke with managers at each of nine research, space, and
space flight centers, public affairs officials from NASA
headquarters and the four largest centers in terms of researchers
who would be in a position to disseminate research, and the
agency's Chief of Staff.^1 At NIST, we spoke with agency
officials, including the Chief of Staff, the program office
director, and the director of the public and business affairs
office. At NOAA, we spoke with managers in each of the NOAA
program offices, as well as each of the laboratories in the
Oceanic and Atmospheric Research program office and each of the
National Marine Fisheries Service fisheries science centers.^2 We
also spoke with public affairs officials in each of the NOAA
program offices. We followed a standard set of questions for each
of these interviews, confirmed that we had copies of the relevant
dissemination policies, and discussed at length the ways in which
these policies were put into practice.
^1The NASA centers included in this review were the Ames Research Center,
Dryden Flight Research Center, Glenn Research Center, Goddard Space Flight
Center, Johnson Space Center, Kennedy Space Center, Langley Research
Center, Marshall Space Flight Center, and Stennis Space Center. The Jet
Propulsion Laboratory, the 10th NASA center, was scoped out of this review
because research at this center is primarily performed by contract
researchers.
^2The NOAA program offices we spoke with and included in our scope were
the Office of Oceanic and Atmospheric Research; the National Environmental
Satellite, Data, and Information Service; the National Marine Fisheries
Service; the National Ocean Service; and the National Weather Service. We
also spoke with managers in the NOAA Office of Marine and Aviation
Operations and the Office of Program Planning and Integration, but these
offices were removed from our scope because they did not have researchers
who would be in a position to disseminate research.
To determine how effectively the dissemination and dispute
resolution policies have been communicated to researchers and to
determine the extent to which researchers at these agencies had
experienced restrictions on the dissemination of their research,
we conducted a Web-based survey of a stratified random sample of
1,811 researchers across scientific and engineering disciplines at
NASA, NIST, and NOAA. This sample included 578 NASA researchers,
699 NIST researchers, and 534 NOAA researchers. We selected these
researchers from among the 1,794 researchers at NASA, 1,337
researchers at NIST, and 1,815 researchers at NOAA. Overall, we
received a 66 percent response rate to our survey, for a total of
1,177 respondents. We stratified by agency subcomponent--such as
the NASA center, NIST laboratory, and NOAA program office--and
produced estimates at the agency level. Each subcomponent has a
known nonzero probability of being selected, and each subcomponent
is weighted in the analysis to account statistically for all
subcomponents in the population, including those that were not
selected. We sought the researchers' views on their agency's
research dissemination policies, the level of agency support for
dissemination, and their experiences with dissemination. In
addition, we asked the researchers to provide examples of ways in
which dissemination of results works well and ways in which it
could be improved. We defined researchers to be included in our
population as federally employed scientists, engineers, or other
researchers who are in a position to disseminate their research
results to a wider audience. To identify these names, we asked
center or program managers at NASA, NIST, and NOAA to provide a
list of researchers in their organization who would meet our
defined criterion. We cross-referenced this list with other
published lists of researchers in each facility, where available.
We further refined the list by sending out notification e-mails
that allowed recipients who did not meet our criteria to notify us
prior to issuing the survey.
Information about accessing the survey was provided via e-mail for
all survey participants. The survey was activated, and researchers
were informed of its availability on November 16, 2006; it was
available through December 31, 2006. To ensure security and data
integrity, we provided all participants with a user name and a
personal password that allowed them to access and complete the
survey. No one else could access that survey or edit its data. To
reduce survey nonresponse, we sent out e-mail reminder messages
and a contractor was used to make follow-up telephone calls to all
nonrespondants to encourage them to complete the survey. From the
initial notification, we identified 21 individuals who were
outside the target population. For example, some individuals had
retired from the agency. In all, we received a 66 percent response
rate. Response rates for our survey population by agency were as
follows: NASA, 63.6 percent; NIST, 61.5 percent; and NOAA, 73.7
percent.
All sample surveys are subject to sampling error--that is, the
extent to which the survey results differ from what would have
been obtained if the whole population had been observed. Measures
of sampling error are defined by two elements, the width of the
confidence intervals around the estimate (sometimes called the
precision of the estimate) and the confidence level at which the
intervals are computed. Because we followed a probability
procedure based on random selections, our sample is only one of a
large number of samples that we might have drawn. Because each
sample could have provided different estimates, we expressed our
confidence in the precision of our particular sample's results as
a 95 percent confidence interval. This is the interval that would
contain the actual population value for 95 percent of the samples
we could have drawn. As a result, we are 95 percent confident that
each of the confidence intervals based on the survey includes the
true values in the sample population. Unless otherwise noted, all
percentage estimates have a 95 percent confidence interval within
plus or minus 8 percentage points. The 95 percent confidence
intervals for numeric estimates are presented along with those
estimates in the body of the report. In addition, any comparison
between point estimates is statistically significant at the 0.05
level, unless otherwise noted. In some instances, we used general
modifiers (i.e., few, some, many, and most) to characterize
percentage point estimates or to indicate the number of center or
office managers who made a particular statement. We used the
following method to assign these modifiers to our statements:
"few" represents less than 10 percent of respondents or managers
representing centers or offices, "some" or "many" represents 10 to
50 percent of respondents or managers representing centers or
offices, and "most" represents over 50 percent of respondents or
managers representing centers or offices. These ranges do not
represent a standard; rather, we assigned these numeric ranges on
the basis of natural breaks in our data. In all cases where we use
these modifiers to characterize point estimates from our survey,
their use implies statistical significance. For example, where we
use the term "most," the lower bound of the confidence interval of
the point estimate is greater than 50 percent.
Regarding the survey data on instances in which researchers were
denied the opportunity to disseminate, the percentage of
researchers reporting such instances can be generalized to the
population. However, because the number of denials is small, the
details concerning the reason for denial, the researcher's primary
field of research, and the actions taken in response to the denial
cannot be generalized. Because the issue of a researcher being
denied is a salient piece of our analysis, we included this
nongeneralizable information to provide context to this important
issue. To analyze select, open-ended questions on our survey,
including those given as the reason researchers said they were
denied, we conducted a content analysis to develop our agreement
statistics. Reviewers (two per question) collaboratively developed
content categories based on survey responses, and then
independently assessed and coded each survey response into those
categories. Intercoder reliability (agreement) statistics were
electronically generated in the coding process, and agreement
statistics for all categories were 90 percent or above. Coding
disagreements were resolved through reviewer discussion or a
third-party arbiter.
In addition to the reported sampling errors, as previously
indicated, the practical difficulties of conducting any survey may
introduce errors, commonly referred to as "nonsampling errors."
For example, differences in how a particular question is
interpreted, the information sources available to respondents, or
the types of sample members who do not respond can introduce
unwanted variability into the survey results. Our estimation
method assumes that nonrespondents are missing at random. If
characteristics of respondents are different than nonrespondents
on key items, it could introduce a bias not accounted for in our
analysis. We took extensive steps in questionnaire development,
data collection, and the editing and analysis of the survey data
to minimize nonsampling errors. For example, the survey was
developed by a GAO survey specialist in conjunction with subject
matter experts, and then reviewed by a second, independent survey
specialist. In addition, we pretested the survey by telephone with
researchers from all three agencies. During these pretests, we
asked each researcher to complete the survey as they would when
they received it. We then interviewed the respondents to ensure
that (1) the questions were clear and unambiguous, (2) the terms
used were precise, (3) the survey did not place an undue burden on
the researchers completing it, and (4) the survey was independent
and unbiased. On the basis of the feedback from the pretests, we
revised the questions, as appropriate. The SAS and SUDAAN programs
that produced our survey estimates, including estimates of
categories derived from content analysis, were reviewed by a
second, independent programmer to ensure accuracy in the logic and
syntax of the program.
To determine OSTP's role in helping agencies develop and implement
policies for the dissemination of research results, we submitted
written questions. They subsequently replied to these questions in
writing.
We conducted our work from June 2006 through March 2007 in
accordance with generally accepted government auditing standards.
Appendix II: Selected Survey Results for NASA, NIST, and NOAA
The following tables contain summary results of selected questions
from our survey of researchers at NASA, NIST, and NOAA. For each
question reported below, the estimated percentage is presented
along with its 95 percent confidence interval. These tables do not
include summary-level data for the demographic questions and do
not include the results from any open-ended questions.
Q8. Overall, how familiar or unfamiliar are you with the policies that
your agency currently has in place regarding the dissemination of research
results outside of the agency?
Estimated 95 PercentConfidence
Agency Question Choices Percentage Interval
Very or Moderately
Total Familiar 87.2 84.9-89.2
Slightly or Not at
Total All Familiar 12.8 10.8-15.1
Very or Moderately
NASA Familiar 91.9 88.4-94.3
Slightly or Not at
NASA All Familiar 8.1 5.7-11.6
Very or Moderately
NIST Familiar 85.4 81.8-88.4
Slightly or Not at
NIST All Familiar 14.6 11.6-18.2
Very or Moderately
NOAA Familiar 83.6 78.6-87.7
Slightly or Not at
NOAA All Familiar 16.4 12.3-21.4
Q9. Has your agency used any of the following means to help you understand
how to comply with its policies regarding the dissemination of research
results outside of the agency?
a. Formal training sessions
Agency Question Choices Estimated Percentage 95 PercentConfidence Interval
Total Yes 21.6 19.3-24.1
Total No 61.6 58.6-64.5
Total Not sure 16.8 14.5-19.4
NASA Yes 32.0 27.3-37.2
NASA No 49.5 44.2-54.8
NASA Not sure 18.5 14.6-23.2
NIST Yes 23.6 20.0-27.7
NIST No 54.8 50.3-59.2
NIST Not sure 21.6 18.1-25.5
NOAA Yes 9.4 6.7-13.1
NOAA No 79.0 73.5-83.6
NOAA Not sure 11.6 8.0-16.6
Q9. Has your agency used any of the following means to help you understand
how to comply with its policies regarding the dissemination of research
results outside of the agency?
b. On-the-job training (including mentoring)
Agency Question Choices Estimated Percentage 95 PercentConfidence Interval
Total Yes 67.9 64.8-70.9
Total No 24.3 21.7-27.2
Total Not sure 7.8 6.2-9.7
NASA Yes 69.2 64.1-73.8
NASA No 21.8 17.8-26.4
NASA Not sure 9.0 6.4-12.6
NIST Yes 78.7 74.8-82.1
NIST No 13.3 10.6-16.6
NIST Not sure 8.0 5.8-10.9
NOAA Yes 58.6 52.3-64.6
NOAA No 35.2 29.5-41.3
NOAA Not sure 6.2 3.8-10.2
Q9. Has your agency used any of the following means to help you understand
how to comply with its policies regarding the dissemination of research
results outside of the agency?
c. Staff or town hall meetings
Agency Question Choices Estimated Percentage 95 PercentConfidence Interval
Total Yes 40.7 37.8-43.8
Total No 43.3 40.2-46.4
Total Not sure 16.0 13.7-18.5
NASA Yes 48.2 42.9-53.6
NASA No 34.4 29.5-39.5
NASA Not sure 17.4 13.7-22.0
NIST Yes 49.2 44.7-53.7
NIST No 32.3 28.2-36.6
NIST Not sure 18.5 15.4-22.1
NOAA Yes 26.9 22.0-32.4
NOAA No 60.6 54.4-66.4
NOAA Not sure 12.6 8.8-17.6
Q9. Has your agency used any of the following means to help you understand
how to comply with its policies regarding the dissemination of research
results outside of the agency?
d. E-mails or other correspondence from [AGENCY] Administrator-level
offices sent to all staff
Agency Question Choices Estimated Percentage 95 PercentConfidence Interval
Total Yes 65.3 62.2-68.3
Total No 19.3 16.9-21.9
Total Not sure 15.4 13.2-17.9
NASA Yes 62.5 57.3-67.4
NASA No 21.8 17.7-26.4
NASA Not sure 15.8 12.2-20.2
NIST Yes 66.0 61.7-70.1
NIST No 17.9 14.7-21.5
NIST Not sure 16.1 13.1-19.7
NOAA Yes 67.7 61.6-73.2
NOAA No 17.8 13.6-23.1
NOAA Not sure 14.5 10.5-19.6
Q9. Has your agency used any of the following means to help you understand
how to comply with its policies regarding the dissemination of research
results outside of the agency?
e. E-mails or other correspondence from Center or Office management sent
to all Center or Office staff
Agency Question Choices Estimated Percentage 95 PercentConfidence Interval
Total Yes 70.9 68.2-73.5
Total No 14.7 12.8-16.9
Total Not sure 14.3 12.4-16.6
NASA Yes 78.5 74.0-82.3
NASA No 11.6 9.0-14.8
NASA Not sure 9.9 7.1-13.7
NIST Yes 54.7 50.2-59.1
NIST No 21.6 18.2-25.5
NIST Not sure 23.7 20.1-27.7
NOAA Yes 74.7 69.3-79.5
NOAA No 13.0 9.6-17.4
NOAA Not sure 12.2 8.9-16.6
Q9. Has your agency used any of the following means to help you understand
how to comply with its policies regarding the dissemination of research
results outside of the agency?
f. Notifications on internal agency websites
Agency Question Choices Estimated Percentage 95 PercentConfidence Interval
Total Yes 47.2 44.0-50.4
Total No 26.1 23.3-29.0
Total Not sure 26.8 24.0-29.7
NASA Yes 50.7 45.3-56.0
NASA No 22.0 18.1-26.5
NASA Not sure 27.4 22.8-32.4
NIST Yes 55.8 51.3-60.2
NIST No 20.6 17.1-24.5
NIST Not sure 23.7 20.0-27.8
NOAA Yes 37.2 31.2-43.7
NOAA No 34.4 28.6-40.7
NOAA Not sure 28.4 23.2-34.3
Q9. Has your agency used any of the following means to help you understand
how to comply with its policies regarding the dissemination of research
results outside of the agency?
g. Other - Please specify below.
Agency Question Choices Estimated Percentage 95 PercentConfidence Interval
Total Yes 18.2 14.3-22.9
Total No 44.8 39.2-50.4
Total Not sure 37.0 31.6-42.7
NASA Yes 22.1 15.2-31.0
NASA No 40.6 32.1-49.7
NASA Not sure 37.3 28.6-47.0
NIST Yes 18.7 13.3-25.6
NIST No 42.5 35.1-50.2
NIST Not sure 38.8 31.7-46.5
NOAA Yes 13.7 7.8-22.9
NOAA No 51.1 39.8-62.3
NOAA Not sure 35.2 25.1-46.8
Q10. Overall, how effective or ineffective have your agency's efforts been
to help you understand how to comply with its policies regarding
dissemination of research results though each of the following routes?
a. Publications (such as peer reviewed publications or non-peer reviewed
publications)
Estimated 95 PercentConfidence
Agency Question Choices Percentage Interval
Extremely to
Total Moderately Effective 91.9 90.1-93.3
Slightly or Not at
Total All Effective 8.1 6.7-9.9
Extremely to
NASA Moderately Effective 90.6 87.2-93.1
Slightly or Not at
NASA All Effective 9.4 6.9-12.8
Extremely to
NIST Moderately Effective 93.7 91.1-95.6
Slightly or Not at
NIST All Effective 6.3 4.4-8.9
Extremely to
NOAA Moderately Effective 91.9 88.5-94.4
Slightly or Not at
NOAA All Effective 8.1 5.6-11.5
Q10. Overall, how effective or ineffective have your agency's efforts been
to help you understand how to comply with its policies regarding
dissemination of research results though each of the following routes?
b. Presentations (such as keynote addresses, conferences, workshops,
symposia, professional society meetings, or Congressional hearings)
Estimated 95 PercentConfidence
Agency Question Choices Percentage Interval
Extremely to
Total Moderately Effective 86.4 83.9-88.5
Slightly or Not at
Total All Effective 13.6 11.5-16.1
Extremely to
NASA Moderately Effective 88.0 84.1-91.0
Slightly or Not at
NASA All Effective 12.0 9.0-15.9
Extremely to
NIST Moderately Effective 88.6 85.2-91.3
Slightly or Not at
NIST All Effective 11.4 8.7-14.8
Extremely to
NOAA Moderately Effective 83.2 78.0-87.4
Slightly or Not at
NOAA All Effective 16.8 12.6-22.0
Q10. Overall, how effective or ineffective have your agency's efforts been
to help you understand how to comply with its policies regarding
dissemination of research results though each of the following routes?
c. Agency releases (such as press releases, web posting on an agency
website, or agency reports or fact sheets)
Estimated 95 PercentConfidence
Agency Question Choices Percentage Interval
Extremely to
Total Moderately Effective 82.9 80.1-85.4
Slightly or Not at
Total All Effective 17.1 14.6-19.9
Extremely to
NASA Moderately Effective 80.8 75.4-85.2
Slightly or Not at
NASA All Effective 19.2 14.8-24.6
Extremely to
NIST Moderately Effective 84.8 80.8-88.1
Slightly or Not at
NIST All Effective 15.2 11.9-19.2
Extremely to
NOAA Moderately Effective 83.5 78.3-87.6
Slightly or Not at
NOAA All Effective 16.5 12.4-21.7
Q10. Overall, how effective or ineffective have your agency's efforts been
to help you understand how to comply with its policies regarding
dissemination of research results though each of the following routes?
d. Media interviews
Estimated 95 PercentConfidence
Agency Question Choices Percentage Interval
Extremely to
Total Moderately Effective 78.7 75.4-81.7
Slightly or Not at
Total All Effective 21.3 18.3-24.6
Extremely to
NASA Moderately Effective 73.0 66.8-78.4
Slightly or Not at
NASA All Effective 27.0 21.6-33.2
Extremely to
NIST Moderately Effective 85.0 80.8-88.5
Slightly or Not at
NIST All Effective 15.0 11.5-19.2
Extremely to
NOAA Moderately Effective 79.0 72.9-84.1
Slightly or Not at
NOAA All Effective 21.0 15.9-27.1
Q10. Overall, how effective or ineffective have your agency's efforts been
to help you understand how to comply with its policies regarding
dissemination of research results though each of the following routes?
e. Other - Please specify below.
Estimated 95 PercentConfidence
Agency Question Choices Percentage Interval
Extremely to
Total Moderately Effective 62.9 43.9-78.6
Slightly or Not at
Total All Effective 37.1 21.4-56.1
Extremely to
NASA Moderately Effective 61.2 31.3-84.5
Slightly or Not at
NASA All Effective 38.8 15.5-68.7
Extremely to
NIST Moderately Effective 90.1 61.5-98.1
Slightly or Not at
NIST All Effective 9.9 1.9-38.5
Extremely to
NOAA Moderately Effective 38.1 15.3-67.7
Slightly or Not at
NOAA All Effective 61.9 32.3-84.7
Q11. Regardless of how effective or ineffective your agency's efforts have
been, how confident, if at all, are you that you understand your agency's
policies regarding the dissemination of research results through each of
the following routes well enough to follow them?
a. Publications
Estimated 95 PercentConfidence
Agency Question Choices Percentage Interval
Very or Moderately
Total confident 89.8 87.7-91.5
Slightly or Not at
Total all confident 10.2 8.5-12.3
Very or Moderately
NASA confident 90.3 86.8-93.0
Slightly or Not at
NASA all confident 9.7 7.0-13.2
Very or Moderately
NIST confident 91.7 88.8-93.9
Slightly or Not at
NIST all confident 8.3 6.1-11.2
Very or Moderately
NOAA confident 87.8 83.4-91.1
Slightly or Not at
NOAA all confident 12.2 8.9-16.6
Q11. Regardless of how effective or ineffective your agency's efforts have
been, how confident, if at all, are you that you understand your agency's
policies regarding the dissemination of research results through each of
the following routes well enough to follow them?
b. Presentations
Estimated 95 PercentConfidence
Agency Question Choices Percentage Interval
Very or Moderately
Total confident 85.8 83.3-88.0
Slightly or Not at
Total all confident 14.2 12.0-16.7
Very or Moderately
NASA confident 88.3 84.4-91.4
Slightly or Not at
NASA all confident 11.7 8.6-15.6
Very or Moderately
NIST confident 88.3 85.0-91.0
Slightly or Not at
NIST all confident 11.7 9.0-15.0
Very or Moderately
NOAA confident 81.4 76.0-85.9
Slightly or Not at
NOAA all confident 18.6 14.1-24.0
Q11. Regardless of how effective or ineffective your agency's efforts have
been, how confident, if at all, are you that you understand your agency's
policies regarding the dissemination of research results through each of
the following routes well enough to follow them?
c. Agency releases
Estimated 95 PercentConfidence
Agency Question Choices Percentage Interval
Very or Moderately
Total confident 66.2 62.8-69.5
Slightly or Not at
Total all confident 33.8 30.5-37.2
Very or Moderately
NASA confident 60.9 54.9-66.6
Slightly or Not at
NASA all confident 39.1 33.4-45.1
Very or Moderately
NIST confident 75.1 70.6-79.1
Slightly or Not at
NIST all confident 24.9 20.9-29.4
Very or Moderately
NOAA confident 64.8 58.3-70.7
Slightly or Not at
NOAA all confident 35.2 29.3-41.7
Q11. Regardless of how effective or ineffective your agency's efforts have
been, how confident, if at all, are you that you understand your agency's
policies regarding the dissemination of research results through each of
the following routes well enough to follow them?
d. Media interviews
Estimated 95 PercentConfidence
Agency Question Choices Percentage Interval
Very or Moderately
Total confident 63.1 59.6-66.5
Slightly or Not at
Total all confident 36.9 33.5-40.4
Very or Moderately
NASA confident 55.1 49.0-61.0
Slightly or Not at
NASA all confident 44.9 39.0-51.0
Very or Moderately
NIST confident 74.7 70.1-78.7
Slightly or Not at
NIST all confident 25.3 21.3-29.9
Very or Moderately
NOAA confident 62.4 55.8-68.6
Slightly or Not at
NOAA all confident 37.6 31.4-44.2
Q11. Regardless of how effective or ineffective your agency's efforts have
been, how confident, if at all, are you that you understand your agency's
policies regarding the dissemination of research results through each of
the following routes well enough to follow them?
e. Other - Please specify below.
Estimated 95 PercentConfidence
Agency Question Choices Percentage Interval
Very or Moderately
Total confident 54.5 35.6-72.2
Slightly or Not at
Total all confident 45.5 27.8-64.4
Very or Moderately
NASA confident 32.2 10.3-66.3
Slightly or Not at
NASA all confident 67.8 33.7-89.7
Very or Moderately
NIST confident 63.3 35.9-84.2
Slightly or Not at
NIST all confident 36.7 15.8-64.1
Very or Moderately
NOAA confident 67.6 30.9-90.7
Slightly or Not at
NOAA all confident 32.4 9.3-69.1
Q12. Overall, in your opinion, how consistently or inconsistently does
your NASA Center, NIST Laboratory, or NOAA Branch/Line Office apply its
policies regarding the dissemination of research results through each of
the following routes?
a.Publications
Estimated 95 PercentConfidence
Agency Question Choices Percentage Interval
Total Very or Somewhat Consistent 90.5 88.2-92.3
Total Neither Consistent Nor Inconsistent 3.4 2.4-4.9
Total Very or Somewhat Inconsistent 6.1 4.6-8.0
NASA Very or Somewhat Consistent 91.4 88.1-93.8
NASA Neither Consistent Nor Inconsistent 3.2 1.8-5.4
NASA Very or Somewhat Inconsistent 5.4 3.5-8.3
NIST Very or Somewhat Consistent 95.7 93.2-97.3
NIST Neither Consistent Nor Inconsistent 1.4 0.7-2.8
NIST Very or Somewhat Inconsistent 2.9 1.6-5.3
NOAA Very or Somewhat Consistent 85.5 80.3-89.6
NOAA Neither Consistent Nor Inconsistent 5.2 3.0-9.0
NOAA Very or Somewhat Inconsistent 9.2 6.1-13.8
Q12. Overall, in your opinion, how consistently or inconsistently does
your NASA Center, NIST Laboratory, or NOAA Branch/Line Office apply its
policies regarding the dissemination of research results through each of
the following routes?
b. Presentations
Estimated 95 PercentConfidence
Agency Question Choices Percentage Interval
Total Very or Somewhat Consistent 80.5 77.6-83.1
Neither Consistent Nor
Total Inconsistent 10.6 8.7-13.0
Total Very or Somewhat Inconsistent 8.9 7.1-11.1
NASA Very or Somewhat Consistent 80.3 75.6-84.3
Neither Consistent Nor
NASA Inconsistent 9.1 6.4-12.7
NASA Very or Somewhat Inconsistent 10.6 7.7-14.5
NIST Very or Somewhat Consistent 85.9 82.2-88.9
Neither Consistent Nor
NIST Inconsistent 10.2 7.6-13.4
NIST Very or Somewhat Inconsistent 4.0 2.4-6.4
Very or Somewhat Consistent 76.9 Question 70.8-82.0Estimated
NOAA Agency Choices Percentage
Neither Consistent Nor
NOAA Inconsistent 12.8 8.9-18.0
NOAA Very or Somewhat Inconsistent 10.3 7.0-15.0
Q12. Overall, in your opinion, how consistently or inconsistently does
your NASA Center, NIST Laboratory, or NOAA Branch/Line Office apply its
policies regarding the dissemination of research results through each of
the following routes?
c. Agency releases
95
Estimated PercentConfidence
Agency Question Choices Percentage Interval
Very or Somewhat
Total Consistent 80.9 77.4-83.9
Neither Consistent Nor
Total Inconsistent 10.5 8.1-13.4
Very or Somewhat
Total Not sure Inconsistent 0.0 8.7 6.7-11.2
Very or Somewhat 71.3
NASA Yes Consistent 15.4 5.1-38.1 64.4-77.4
Neither Consistent Nor 15.9
NASA No Inconsistent 74.7 49.5-89.9 11.3-21.8
Very or Somewhat 12.8
NASA Not sure Inconsistent 9.9 1.9-38.2 8.6-18.5
Very or Somewhat 94.5
NIST Yes Consistent 7.9 2.5-22.7 91.4-96.5
Neither Consistent Nor 2.8
NIST No Inconsistent 92.1 77.3-97.5 1.5-5.0
Very or Somewhat
NIST Not sure Inconsistent 0.0 2.7 1.4-5.4
Very or Somewhat
NOAA e. Resubmitted Consistent [If Question
the same or similar 19 is Yes and Question 73.9-84.9Question
document 22 is Yes] 80.0 Agency Choices
NOAA 95
PercentConfidence Neither Consistent Nor
Interval Inconsistent 10.9 Total 7.1-16.4Yes
Very or Somewhat
NOAA 6.2-35.4 Inconsistent 9.1 Total 6.2-13.2No
Q12. Overall, in your opinion, how consistently or inconsistently does
your NASA Center, NIST Laboratory, or NOAA Branch/Line Office apply its
policies regarding the dissemination of research results through each of
the following routes? 63.4-92.4
d. Media interviews
Estimated 95 PercentConfidence
Agency Question Choices Percentage Interval
Very or Somewhat
Total NASA Consistent Yes 74.9 3.7 71.0-78.40.7-16.9
Neither Consistent
Total NASA Nor Inconsistent No 12.4 96.3 9.9-15.583.1-99.3
Very or Somewhat
Total NASA Inconsistent Not sure 12.7 0.0 10.1-15.8
Very or Somewhat
NASA NIST Consistent Yes 62.3 7.7 54.9-69.11.6-29.9
Neither Consistent
NASA NIST Nor Inconsistent No 16.0 82.4 11.5-21.956.6-94.4
Very or Somewhat
NASA NIST Inconsistent Not sure 21.7 9.9 16.1-28.61.9-38.2
Very or Somewhat
NIST NOAA Consistent Yes 89.7 39.5 85.5-92.813.7-72.8
Neither Consistent
NIST NOAA Nor Inconsistent No 6.6 60.5 4.3-10.127.2-86.3
Very or Somewhat
NIST NOAA Inconsistent Not sure 3.7 0.0 2.0-6.8
NOAA Q25. Did you
take any of the
following actions 75.4 [If
when you did not Question 19
receive approval to Very or Somewhat is Yes and
disseminate your Consistent f. Gave up Question 22
research results? trying is Yes] 68.6-81.2Agency
Neither Consistent
Nor Inconsistent 95
NOAA Estimated PercentConfidence
Percentage Interval 13.2 8.9-19.1Total
Very or Somewhat
Inconsistent
NOAA 66.5 51.5-78.7 11.4 7.6-16.7Total
Q12. Overall, in your opinion, how consistently or inconsistently does
your NASA Center, NIST Laboratory, or NOAA Branch/Line Office apply its
policies regarding the dissemination of research results through each of
the following routes? 28.9
e. Other - Please specify below. 17.4-44.0
95 PercentConfidence
Agency Question Choices Estimated Percentage Interval
Very or Somewhat
Total Consistent NASA 56.0 Yes 36.9-73.470.2
Neither Consistent Nor
Total Inconsistent NASA 10.0 No 1.7-42.029.8
Very or Somewhat
Total Inconsistent NASA 34.0 Not sure 19.9-51.70.0
Very or Somewhat
NASA Consistent NIST 50.8 Yes 20.6-80.446.7
Neither Consistent Nor
NASA Inconsistent NIST 21.5 No 3.7-66.435.0
Very or Somewhat
NASA Inconsistent NIST 27.7 Not sure 9.4-58.818.2
Very or Somewhat
NIST Consistent NOAA 71.8 Yes 46.3-88.372.2
Neither Consistent Nor
NIST Inconsistent NOAA 0.0 No 24.6
Very or Somewhat
NIST Inconsistent NOAA 28.2 Not sure 11.7-53.73.3
Very or Somewhat
Consistent Q25. Did you
take any of the 44.6 g. Received
following actions when approval following 19.4-72.9[If
you did not receive other types of Question 19 is Yes
approval to disseminate revisions - Please and Question 22 is
NOAA your research results? specify below. Yes]
NOAA 0.0 95
Question Neither Consistent Nor PercentConfidence
Choices Inconsistent Estimated Interval95
Question Percentage Estimated PercentConfidence
Choices Percentage Interval
Very or Somewhat
NOAA Yes Inconsistent 7.0 55.4 3.0-15.9 27.1-80.6
Q13. Overall, in your opinion, how consistently or inconsistently does
[AGENCY] as a whole apply its policies regarding the dissemination of
research results through each of the following routes? No
a. Publications 89.0
Estimated 95 PercentConfidence
Agency Question Choices Percentage Interval
Total Very or Somewhat
1.5-10.1 Consistent 85.6 NASA 82.8-88.0Yes
Total Neither Consistent
2.6-26.8 Nor Inconsistent 5.1 NASA 3.7-6.9No
Total Very or Somewhat
73.2-97.4 Inconsistent 9.4 NASA 7.4-11.8Not sure
Very or Somewhat
NASA Consistent 83.8 NIST 79.1-87.7Yes
Neither Consistent
NASA 1.6-29.9 Nor Inconsistent 4.3 NIST 2.5-7.1No
NASA Very or Somewhat
56.6-94.4 Inconsistent 11.9 NIST 8.6-16.3Not sure
Very or Somewhat
NIST 1.9-38.2 Consistent 95.1 NOAA 92.4-96.9Yes
Neither Consistent
NIST 0.8-16.8 Nor Inconsistent 1.2 NOAA 0.5-2.6No
NIST Very or Somewhat
75.4-96.3 Inconsistent 3.7 NOAA 2.1-6.3Not sure
78.9 Q25. Did you
take any of the
following actions
when you did not
receive approval to
Very or Somewhat disseminate your 72.4-84.2h. Other -
NOAA 2.0-17.8 Consistent research results? Please specify below.
Neither Consistent 6.4-14.795
Nor Inconsistent 9.8 Estimated PercentConfidence
NOAA Agency Question Choices Percentage Interval
Very or Somewhat
NOAA Total Inconsistent Yes 11.3 24.8 7.4-16.99.5-50.9
Q13. Overall, in your opinion, how consistently or inconsistently does
[AGENCY] as a whole apply its policies regarding the dissemination of
research results through each of the following routes? Total
b. Presentations No
95 PercentConfidence
Agency Question Choices Estimated Percentage Interval
Very or Somewhat
Consistent
Total 22.4 7.2-51.7 77.2 74.0-80.1NASA
Neither
Consistent Nor
Inconsistent
Total 51.1 9.8-90.9 11.7 9.6-14.3NASA
Very or Somewhat
Inconsistent
Total 48.9 9.1-90.2 11.1 8.9-13.7NASA
Very or Somewhat
NASA 0.0 Consistent 73.5 68.0-78.4NIST
Neither
Consistent Nor
NASA 0.0 Inconsistent 12.2 8.8-16.8NIST
Very or Somewhat
Inconsistent
NASA 69.7 22.7-94.8 14.2 10.5-19.0NIST
Very or Somewhat
Consistent
NIST 30.3 5.2-77.3 86.0 82.3-89.1NOAA
Neither
Consistent Nor
Inconsistent
NIST 33.5 10.1-69.3 10.2 7.6-13.6NOAA
Very or Somewhat
Inconsistent
NIST 40.6 13.4-75.2 3.8 2.3-6.1NOAA
67.3-79.8Q26. To the best
of your knowledge, does
your agency have a
process or procedure in
place to appeal decisions
Very or Somewhat made regarding the
Consistent dissemination of research
NOAA 25.9 5.3-68.7 74.1 results?
Neither
Consistent Nor
NOAA Inconsistent 12.4 95
Question Estimated PercentConfidence
Choices Percentage Interval95
Question Estimated PercentConfidence
Choices Percentage Interval 8.5-17.8
NOAA Yes,
familiar Very or Somewhat
with it Inconsistent 7.4 13.5 5.9-9.1 9.3-19.2
Q13. Overall, in your opinion, how consistently or inconsistently does
[AGENCY] as a whole apply its policies regarding the dissemination of
research results through each of the following routes? Yes, but not
familiar with it
c. Agency releases 18.0
Estimated
Agency Question Choices Percentage 95 PercentConfidence Interval
Total Very or Somewhat
15.0-19.9 Consistent 76.7 Total 72.9-80.1Not sure
Total Neither Consistent
54.2-60.4 Nor Inconsistent 8.9 NASA 6.8-11.6Yes, familiar with it
Total Very or Somewhat 11.6-17.7Yes, but not familiar
5.6-11.5 Inconsistent 14.4 NASA with it
NASA Very or Somewhat
20.9-30.0 Consistent 66.9 NASA 59.7-73.3No
NASA Neither Consistent
11.9-19.3 Nor Inconsistent 10.9 NASA 7.0-16.5Not sure
NASA Very or Somewhat
46.3-56.7 Inconsistent 22.2 NIST 16.8-28.9Yes, familiar with it
Very or Somewhat 90.1-95.7Yes, but not familiar
NIST 7.3-12.6 Consistent 93.4 NIST with it
NIST Neither Consistent
13.7-20.1 Nor Inconsistent 4.6 NIST 2.7-7.6No
NIST Very or Somewhat
13.5-20.0 Inconsistent 2.0 NIST 1.0-4.1Not sure
NOAA Very or Somewhat
52.8-61.5 Consistent 72.7 NOAA 65.3-79.1Yes, familiar with it
Neither Consistent 7.0-15.8Yes, but not familiar
NOAA 3.0-8.3 Nor Inconsistent 10.6 NOAA with it
Very or Somewhat
NOAA 8.1-16.1 Inconsistent 16.6 NOAA 11.6-23.4No
Q13. Overall, in your opinion, how consistently or inconsistently does
[AGENCY] as a whole apply its policies regarding the dissemination of
research results through each of the following routes? 15.5-25.6
d. Media interviews
Question Estimated 95 PercentConfidence
Agency Choices Percentage Interval
Total Q27. In your
opinion, how effective or
ineffective is this
appeals process or
procedure in terms of its Very or
ability to resolve Somewhat
dissemination issues in a Consistent [If 69.5-77.0Question
fair and reasonable Question 26 is 73.4 Agency Choices Question
manner? Yes] Agency Choices
Total 95 PercentConfidence Neither
Interval95 Consistent Nor 6.3-10.5Very or
PercentConfidence Interval Inconsistent 8.1 Total Somewhat Effective
Very or 15.2-22.1Neither
Somewhat Effective Nor
Total 78.8-92.9 Inconsistent 18.4 Total Ineffective
Very or
Somewhat 49.5-64.4Very or
NASA 2.4-10.6 Consistent 57.1 Total Somewhat Ineffective
Neither
Consistent Nor 8.0-18.1Very or
NASA 3.3-16.1 Inconsistent 12.1 NASA Somewhat Effective
Very or 24.2-38.2Neither
Somewhat Effective Nor
NASA 62.5-89.0 Inconsistent 30.8 NASA Ineffective
Very or
Somewhat 84.7-92.3Very or
NIST 1.7-18.1 Consistent 89.1 NASA Somewhat Ineffective
Neither
Consistent Nor 5.1-11.3Very or
NIST 6.9-31.6 Inconsistent 7.6 NIST Somewhat Effective
Very or 1.7-6.5Neither
Somewhat Effective Nor
NIST Inconsistent 3.3 NIST Ineffective
Very or
Somewhat 70.0-82.5Very or
NOAA Consistent 76.8 NIST Somewhat Ineffective
Neither
Consistent Nor 3.0-7.3Very or
NOAA Inconsistent 4.7 NOAA Somewhat Effective
Very or 13.2-25.1Neither
Somewhat Effective Nor
NOAA 78.5-96.2 Inconsistent 18.5 NOAA Ineffective
Q13. Overall, in your opinion, how consistently or inconsistently does
[AGENCY] as a whole apply its policies regarding the dissemination of
research results through each of the following routes? 3.8-21.5
e. Other - Please specify below.
Estimated 95 PercentConfidence
Agency Question Choices Percentage Interval
38.4-79.1
Very or Somewhat
Total Consistent 60.6
Very or Somewhat
Total Inconsistent 39.4 20.9-61.6
Very or Somewhat
NASA Consistent 26.4 4.2-74.6
Very or Somewhat
Inconsistent
NASA 73.6 25.4-95.8
73.8-98.4
Very or Somewhat
NIST Consistent 92.9
.6-26.2
Very or Somewhat
NIST Inconsistent 7.1
11.0-64.6
Very or Somewhat 32.2
Consistent
Very or Somewhat
NOAA Inconsistent 67.8 35.4-89.0
Q14. Overall, do you believe your agency's dissemination policies are
effective or ineffective in ensuring access to the results of research
conducted at your agency?
Estimated 95 PercentConfidence
Agency Question Choices Percentage Interval
Extremely to
Total Moderately Effective 90.0 87.7-91.8
Slightly or Not at
Total All Effective 10.0 8.2-12.3
Extremely to
NASA Moderately Effective 88.8 85.1-91.6
Slightly or Not at
NASA All Effective 11.2 8.4-14.9
Extremely to
NIST Moderately Effective 95.8 93.4-97.4
Slightly or Not at
NIST All Effective 4.2 2.6-6.6
Extremely to
NOAA Moderately Effective 87.0 82.1-90.7
Slightly or Not at
NOAA All Effective 13.0 9.3-17.9
Q15. How supportive, if at all, is your agency toward disseminating
research results through each of the following routes?
a. Through publications such as peer reviewed journals, or non-peer
reviewed journals
Estimated 95 PercentConfidence
Agency Question Choices Percentage Interval
Very or Moderately
Total Supportive 92.7 90.7-94.2
Slightly or Not at
Total All Supportive 7.3 5.8-9.3
Very or Moderately
NASA Supportive 91.2 87.6-93.8
Slightly or Not at
NASA All Supportive 8.8 6.2-12.4
Very or Moderately
NIST Supportive 97.2 95.3-98.3
Slightly or Not at
NIST All Supportive 2.8 1.7-4.7
Very or Moderately
NOAA Supportive 91.0 87.0-93.9
Slightly or Not at
NOAA All Supportive 9.0 6.1-13.0
Q15. How supportive, if at all, is your agency toward disseminating
research results through each of the following routes?
b. Through presentations, such as at conferences or at congressional
hearings
Estimated 95 PercentConfidence
Agency Question Choices Percentage Interval
Very or Moderately
Total Supportive 92.1 90.0-93.8
Slightly or Not at
Total All Supportive 7.9 6.2-10.0
Very or Moderately
NASA Supportive 91.6 88.1-94.2
Slightly or Not at
NASA All Supportive 8.4 5.8-11.9
Very or Moderately
NIST Supportive 96.1 94.0-97.5
Slightly or Not at
NIST All Supportive 3.9 2.5-6.0
Very or Moderately
NOAA Supportive 89.8 85.1-93.0
Slightly or Not at
NOAA All Supportive 10.2 7.0-14.9
Q15. How supportive, if at all, is your agency toward disseminating
research results through each of the following routes?
c. Through agency releases, such as press releases, web postings, and
agency reports and fact sheets
Estimated 95 PercentConfidence
Agency Question Choices Percentage Interval
Very or Moderately
Total Supportive 77.7 74.3-80.7
Slightly or Not at
Total All Supportive 22.3 19.3-25.7
Very or Moderately
NASA Supportive 75.5 69.6-80.5
Slightly or Not at
NASA All Supportive 24.5 19.5-30.4
Very or Moderately
NIST Supportive 87.0 83.1-90.1
Slightly or Not at
NIST All Supportive 13.0 9.9-16.9
Very or Moderately
NOAA Supportive 73.0 66.4-78.7
Slightly or Not at
NOAA All Supportive 27.0 21.3-33.6
Q15. How supportive, if at all, is your agency toward disseminating
research results through each of the following routes?
d. Through interviews with media
Estimated 95 PercentConfidence
Agency Question Choices Percentage Interval
Very or Moderately
Total Supportive 60.2 56.0-64.3
Slightly or Not at
Total All Supportive 39.8 35.7-44.0
Very or Moderately
NASA Supportive 61.8 54.8-68.3
Slightly or Not at
NASA All Supportive 38.2 31.7-45.2
Very or Moderately
NIST Supportive 68.2 62.4-73.5
Slightly or Not at
NIST All Supportive 31.8 26.5-37.6
Very or Moderately
NOAA Supportive 54.2 46.6-61.6
Slightly or Not at
NOAA All Supportive 45.8 38.4-53.4
Q16. Overall, which one of the following statements best characterizes the
extent to which your agency supports the dissemination of research
results?
Estimated 95 PercentConfidence
Agency Question Choices Percentage Interval
Insists on / Encourages
Total Dissemination 89.1 86.8-90.9
Total Neither Encourages nor Discourages 7.7 6.2-9.6
Discourages / Does Not Allow
Total Dissemination 1.6 0.9-2.8
Total Not sure 1.7 1.0-2.7
Insists on / Encourages
NASA Dissemination 88.6 84.8-91.5
NASA Neither Encourages nor Discourages 9.5 6.7-13.1
Discourages / Does Not Allow
NASA Dissemination 1.3 0.7-2.6
NASA Not sure 0.7 0.2-2.7
Insists on / Encourages
NIST Dissemination 95.8 93.7-97.3
NIST Neither Encourages nor Discourages 1.9 1.0-3.4
Discourages / Does Not Allow
NIST Dissemination 0.0
NIST Not sure 2.3 1.3-4.2
Insists on / Encourages
NOAA Dissemination 84.6 79.8-88.4
NOAA Neither Encourages nor Discourages 10.2 7.3-14.0
Discourages / Does Not Allow
NOAA Dissemination 3.0 1.3-6.6
NOAA Not sure 2.2 1.0-5.0
Q17. Does your area of research have the potential to impact federal
policy decisions?
Agency Question Choices Estimated Percentage 95 PercentConfidence Interval
Total Yes 38.8 36.0-41.6
Total No 37.5 34.8-40.3
Total Not sure 23.7 21.3-26.4
NASA Yes 25.3 21.0-30.1
NASA No 48.7 43.6-53.9
NASA Not sure 26.0 21.6-31.0
NIST Yes 22.5 19.1-26.3
NIST No 44.1 39.8-48.4
NIST Not sure 33.5 29.5-37.7
NOAA Yes 64.9 59.3-70.1
NOAA No 20.9 16.8-25.7
NOAA Not sure 14.2 10.6-18.7
Q18. When disseminating research results that have potential federal
policy implications, does each of the following statements correctly or
not correctly represent your agency's position on discussing these
research results?
a. I can discuss research results with potential policy implications when
the results are consistent with the agency's policy or position
statements.
[If Question 17 is Yes]
Estimated 95 PercentConfidence
Agency Question Choices Percentage Interval
Total Statement Represents Policy 68.9 63.2-74.1
Total Statement Does NOT Represent Policy 12.3 8.9-16.9
Total Not sure 18.8 14.5-23.9
NASA Statement Represents Policy 71.3 60.4-80.1
NASA Statement Does NOT Represent Policy 11.7 6.2-21.2
NASA Not sure 17.0 10.1-27.2
NIST Statement Represents Policy 67.7 58.0-76.1
NIST Statement Does NOT Represent Policy 7.8 3.9-14.9
NIST Not sure 24.5 17.0-34.0
NOAA Statement Represents Policy 68.3 60.1-75.4
NOAA Statement Does NOT Represent Policy 13.7 8.9-20.4
NOAA Not sure 18.1 12.4-25.5
Q18. When disseminating research results that have potential federal
policy implications, does each of the following statements correctly or
not correctly represent your agency's position on discussing these
research results?
b. I can discuss research results with potential policy implications when
the results differ from the agency's policy or position statements.
[If Question 17 is Yes]
Estimated 95 PercentConfidence
Agency Question Choices Percentage Interval
Total Statement Represents Policy 37.4 31.9-43.1
Total Statement Does NOT Represent Policy 22.6 18.3-27.6
Total Not sure 40.0 34.4-45.9
NASA Statement Represents Policy 35.0 25.8-45.5
NASA Statement Does NOT Represent Policy 29.5 20.8-40.1
NASA Not sure 35.5 26.0-46.2
NIST Statement Represents Policy 32.1 23.7-41.7
NIST Statement Does NOT Represent Policy 22.9 15.8-32.0
NIST Not sure 45.0 35.4-54.9
NOAA Statement Represents Policy 39.6 31.9-47.8
NOAA Statement Does NOT Represent Policy 19.8 14.4-26.7
NOAA Not sure 40.6 32.8-48.9
Q18. When disseminating research results that have potential federal
policy implications, does each of the following statements correctly or
not correctly represent your agency's position on discussing these
research results?
c. I can discuss research results with potential policy implications on
issues for which my agency does not have a policy or position statement.
[If Question 17 is Yes]
Estimated 95 PercentConfidence
Agency Question Choices Percentage Interval
Total Statement Represents Policy 53.9 48.1-59.6
Total Statement Does NOT Represent Policy 10.5 7.5-14.5
Total Not sure 35.6 30.2-41.4
NASA Statement Represents Policy 61.0 50.3-70.7
NASA Statement Does NOT Represent Policy 10.5 5.1-20.2
NASA Not sure 28.5 20.0-39.0
NIST Statement Represents Policy 47.7 38.1-57.4
NIST Statement Does NOT Represent Policy 11.7 6.8-19.5
NIST Not sure 40.6 31.3-50.6
NOAA Statement Represents Policy 52.7 44.4-60.7
NOAA Statement Does NOT Represent Policy 10.2 6.3-16.0
NOAA Not sure 37.1 29.5-45.5
Q18. When disseminating research results that have potential federal
policy implications, does each of the following statements correctly or
not correctly represent your agency's position on discussing these
research results?
d. I can discuss potential policy implications of research results
provided that I state the policy implications as my personal views and not
those of the agency.
[If Question 17 is Yes]
Estimated
Agency Question Choices Percentage 95 PercentConfidence Interval
Statement
Represents
Total Policy 40.3 34.8-46.0
Statement Does
NOT Represent
Total Policy 17.3 13.3-22.1
Total Not sure 42.4 36.8-48.3
Statement
Represents
NASA Policy 48.6 38.4-59.0
Statement Does
NOT Represent
NASA Policy 11.6 6.8-18.9
NASA Not sure 39.8 30.0-50.5
Statement
Represents
NIST Policy 24.3 Research
Statement Does
NOT Represent
NIST Policy 26.9 19.1-36.6
NIST Not sure 48.8 39.1-58.6
Statement
Represents
NOAA Policy 40.8 33.2-49.0
Statement Does
NOT Represent
NOAA Policy 17.2 11.7-24.5
NOAA Not sure 42.0 34.1-50.3
Q18. When disseminating research results that have potential federal
policy implications, does each of the following statements correctly or
not correctly represent your agency's position on discussing these
research results?
e. I am not allowed to discuss the potential policy implications of
research results.
[If Question 17 is Yes]
Estimated 95 PercentConfidence
Agency Question Choices Percentage Interval
Total Statement Represents Policy 10.5 7.2-15.0
Total Statement Does NOT Represent Policy 50.2 44.4-56.1
Total Not sure 39.3 33.6-45.3
NASA Statement Represents Policy 3.4 1.2-9.0
NASA Statement Does NOT Represent Policy 57.7 46.9-67.9
NASA Not sure 38.9 29.0-49.8
NIST Statement Represents Policy 11.7 6.4-20.3
NIST Statement Does NOT Represent Policy 46.2 36.6-56.0
NIST Not sure 42.2 32.8-52.2
NOAA Statement Represents Policy 13.1 8.2-20.3
NOAA Statement Does NOT Represent Policy 48.1 39.9-56.5
NOAA Not sure 38.7 30.7-47.4
Q19. In the past 5 years, have you sought to disseminate the results of
your own scientific research outside of the agency?
Agency Question Choices Estimated Percentage 95 PercentConfidence Interval
Total Yes 86.1 83.6-88.2
Total No 13.9 11.8-16.4
NASA Yes 90.6 87.1-93.2
NASA No 9.4 6.8-12.9
NIST Yes 88.1 84.8-90.8
NIST No 11.9 9.2-15.2
NOAA Yes 80.0 74.4-84.6
NOAA No 20.0 15.4-25.6
Q20. Which of the following reasons describe why in the past 5 years you
have not sought to disseminate the results of your scientific research
outside the agency?
a. I don't conduct research that requires dissemination
[If Question 19 is No]
Agency Question Choices Estimated Percentage 95 PercentConfidence Interval
Total Checked 19.1 12.9-27.4
Total Not checked 80.9 72.6-87.1
NASA Checked 15.0 5.9-33.3
NASA Not checked 85.0 66.7-94.1
NIST Checked 15.8 8.5-27.5
NIST Not checked 84.2 72.5-91.5
NOAA Checked 22.5 13.0-36.1
NOAA Not checked 77.5 63.9-87.0
Q20. Which of the following reasons describe why in the past 5 years you
have not sought to disseminate the results of your scientific research
outside the agency
b. I don't conduct my own research
[If Question 19 is No]
Agency Question Choices Estimated Percentage 95 PercentConfidence Interval
Total Checked 25.9 18.7-34.7
Total Not checked 74.1 65.3-81.3
NASA Checked 14.6 7.6-26.3
NASA Not checked 85.4 73.7-92.4
NIST Checked 18.8 11.2-29.9
NIST Not checked 81.2 70.1-88.8
NOAA Checked 34.5 22.3-49.0
NOAA Not checked 65.5 51.0-77.7
Q20. Which of the following reasons describe why in the past 5 years you
have not sought to disseminate the results of your scientific research
outside the agency?
c. Someone else disseminates research results on my behalf
[If Question 19 is No]
Agency Question Choices Estimated Percentage 95 PercentConfidence Interval
Total Checked 18.4 12.3-26.5
Total Not checked 81.6 73.5-87.7
NASA Checked 32.5 18.2-51.0
NASA Not checked 67.5 49.0-81.8
NIST Checked 19.8 11.1-32.7
NIST Not checked 80.2 67.3-88.9
NOAA Checked 10.9 4.7-23.0
NOAA Not checked 89.1 77.0-95.3
Q20. Which of the following reasons describe why in the past 5 years you
have not sought to disseminate the results of your scientific research
outside the agency?
d. I sought to disseminate results in the past and was not allowed to
[If Question 19 is No]
Agency Question Choices Estimated Percentage 95 PercentConfidence Interval
Total Checked 2.4 0.8-7.2
Total Not checked 97.6 92.8-99.2
NASA Checked 5.2 0.8-26.2
NASA Not checked 94.8 73.8-99.2
NIST Checked 2.4 0.5-11.7
NIST Not checked 97.6 88.3-99.5
NOAA Checked 1.0 0.2-4.8
NOAA Not checked 99.0 95.2-99.8
Q20. Which of the following reasons describe why in the past 5 years you
have not sought to disseminate the results of your scientific research
outside the agency?
e. My research is ongoing and is not ready to be released
[If Question 19 is No]
Agency Question Choices Estimated Percentage 95 PercentConfidence Interval
Total Checked 24.2 17.3-32.8
Total Not checked 75.8 67.2-82.7
NASA Checked 24.8 13.6-40.9
NASA Not checked 75.2 59.1-86.4
NIST Checked 27.2 16.9-40.6
NIST Not checked 72.8 59.4-83.1
NOAA Checked 22.6 12.6-37.1
NOAA Not checked 77.4 62.9-87.4
Q20. Which of the following reasons describe why in the past 5 years you
have not sought to disseminate the results of your scientific research
outside the agency?
f. My research is not eligible for public dissemination
[If Question 19 is No]
Agency Question Choices Estimated Percentage 95 PercentConfidence Interval
Total Checked 5.4 2.5-11.2
Total Not checked 94.6 88.8-97.5
NASA Checked 19.2 8.7-37.3
NASA Not checked 80.8 62.7-91.3
NIST Checked 0.0
NIST Not checked 100.0
NOAA Checked 1.0 0.2-4.8
NOAA Not checked 99.0 95.2-99.8
Q20. Which of the following reasons describe why in the past 5 years you
have not sought to disseminate the results of your scientific research
outside the agency?
g. Agency policies did not allow me to disseminate
[If Question 19 is No]
Agency Question Choices Estimated Percentage 95 PercentConfidence Interval
Total Checked 5.5 2.2-13.1
Total Not checked 94.5 86.9-97.8
NASA Checked 7.0 1.7-24.6
NASA Not checked 93.0 75.4-98.3
NIST Checked 1.6 0.4-6.4
NIST Not checked 98.4 93.6-99.6
NOAA Checked 6.5 1.8-21.1
NOAA Not checked 93.5 78.9-98.2
Q20. Which of the following reasons describe why in the past 5 years you
have not sought to disseminate the results of your scientific research
outside the agency?
h. Other Please specify below.
[If Question 19 is No]
Agency Question Choices Estimated Percentage 95 PercentConfidence Interval
Total Checked 12.0 7.7-18.1
Total Not checked 88.0 81.9-92.3
NASA Checked 12.7 5.8-25.6
NASA Not checked 87.3 74.4-94.2
NIST Checked 19.5 10.5-33.2
NIST Not checked 80.5 66.8-89.5
NOAA Checked 8.4 3.6-18.4
NOAA Not checked 91.6 81.6-96.4
Q21. Over the past 5 years, through which methods did you seek to
disseminate your research results?
a. Publications (such as peer reviewed publications or non-peer reviewed
publications)
[If Question 19 is Yes]
Agency Question Choices Estimated Percentage 95 PercentConfidence Interval
Total Yes 97.0 95.7-97.9
Total No 3.0 2.1-4.2
Total Not sure 0.1 0.0-0.3
NASA Yes 96.8 94.3-98.2
NASA No 3.2 1.8-5.7
NASA Not sure 0.0
NIST Yes 97.3 95.5-98.4
NIST No 2.4 1.4-4.2
NIST Not sure 0.3 0.1-1.2
NOAA Yes 96.8 94.1-98.3
NOAA No 3.2 1.7-5.9
NOAA Not sure 0.0
Q21. Over the past 5 years, through which methods did you seek to
disseminate your research results?
b. Presentations (such as keynote addresses, conferences, workshops,
symposia, professional society meetings, or Congressional hearings)
[If Question 19 is Yes]
Agency Question Choices Estimated Percentage 95 PercentConfidence Interval
Total Yes 96.8 95.1-97.9
Total No 3.2 2.1-4.9
NASA Yes 97.1 95.1-98.3
NASA No 2.9 1.7-4.9
NIST Yes 98.5 96.6-99.4
NIST No 1.5 0.6-3.4
NOAA Yes 95.0 90.1-97.5
NOAA No 5.0 2.5-9.9
Q21. Over the past 5 years, through which methods did you seek to
disseminate your research results?
c. Agency releases (such as press releases, web posting on an agency
website, or agency reports or fact sheets)
[If Question 19 is Yes]
Agency Question Choices Estimated Percentage 95 PercentConfidence Interval
Total Yes 46.4 43.1-49.8
Total No 48.5 45.1-52.0
Total Not sure 5.0 3.7-6.8
NASA Yes 40.8 35.6-46.1
NASA No 51.0 45.5-56.5
NASA Not sure 8.2 5.5-12.1
NIST Yes 54.7 50.0-59.4
NIST No 40.7 36.2-45.4
NIST Not sure 4.6 2.9-7.1
NOAA Yes 46.2 39.4-53.2
NOAA No 52.1 45.2-58.9
NOAA Not sure 1.7 0.6-4.5
Q21. Over the past 5 years, through which methods did you seek to
disseminate your research results?
d. Media interviews
[If Question 19 is Yes]
Agency Question Choices Estimated Percentage 95 PercentConfidence Interval
Total Yes 27.8 24.9-30.8
Total No 65.0 61.8-68.1
Total Not sure 7.2 5.6-9.3
NASA Yes 27.8 23.5-32.5
NASA No 61.2 55.9-66.3
NASA Not sure 11.0 7.9-15.3
NIST Yes 25.5 21.6-29.8
NIST No 68.2 63.7-72.5
NIST Not sure 6.3 4.3-9.1
NOAA Yes 29.6 24.0-36.0
NOAA No 66.9 60.3-72.9
NOAA Not sure 3.5 1.7-7.0
Q21. Over the past 5 years, through which methods did you seek to
disseminate your research results?
e. Other - Please specify below.
[If Question 19 is Yes]
Agency Question Choices Estimated Percentage 95 PercentConfidence Interval
Total Yes 18.9 13.9-25.2
Total No 55.5 48.1-62.7
Total Not sure 25.6 19.5-32.9
NASA Yes 18.6 10.8-30.2
NASA No 53.2 41.1-64.9
NASA Not sure 28.2 18.4-40.6
NIST Yes 18.1 12.2-26.0
NIST No 61.0 51.3-69.8
NIST Not sure 20.9 13.9-30.3
NOAA Yes 20.3 10.9-34.6
NOAA No 52.9 37.3-68.0
NOAA Not sure 26.8 14.8-43.5
Q22. Other than for standard technical review reasons, over the past 5
years, have you ever encountered a situation when your agency did not
allow you to disseminate your research results?
[If Question 19 is Yes]
Agency Question Choices Estimated Percentage 95 PercentConfidence Interval
Total Yes 5.9 4.5-7.8
Total No 94.1 92.2-95.5
NASA Yes 7.0 4.6-10.4
NASA No 93.0 89.6-95.4
NIST Yes 3.7 2.3-5.9
NIST No 96.3 94.1-97.7
NOAA Yes 6.4 3.8-10.7
NOAA No 93.6 89.3-96.2
Q23. Through which method(s) were you seeking to disseminate your research
results when you were denied agency approval?
a. Publication (such as peer reviewed publications or non-peer reviewed
publications)
[If Question 19 is Yes and Question 22 is Yes]
Agency Question Choices Estimated Percentage 95 PercentConfidence Interval
Total Checked 43.9 30.4-58.3
Total Not checked 56.1 41.7-69.6
NASA Checked 53.8 33.3-73.0
NASA Not checked 46.2 27.0-66.7
NIST Checked 36.8 18.3-60.2
NIST Not checked 63.2 39.8-81.7
NOAA Checked 34.1 13.1-63.8
NOAA Not checked 65.9 36.2-86.9
Q23. Through which method(s) were you seeking to disseminate your research
results when you were denied agency approval?
b. Presentations (such as keynote addresses, conferences, workshops,
symposia, professional society meetings, or Congressional hearings)
[If Question 19 is Yes and Question 22 is Yes]
Agency Question Choices Estimated Percentage 95 PercentConfidence Interval
Total Checked 38.9 26.0-53.5
Total Not checked 61.1 46.5-74.0
NASA Checked 39.9 22.5-60.2
NASA Not checked 60.1 39.8-77.5
NIST Checked 29.9 12.1-57.1
NIST Not checked 70.1 42.9-87.9
NOAA Checked 41.8 19.1-68.7
NOAA Not checked 58.2 31.3-80.9
Q23. Through which method(s) were you seeking to disseminate your research
results when you were denied agency approval?
c. Agency releases (including press releases, web posting on an agency
website, or agency reports or fact sheets)
[If Question 19 is Yes and Question 22 is Yes]
Agency Question Choices Estimated Percentage 95 PercentConfidence Interval
Total Checked 21.5 13.1-33.2
Total Not checked 78.5 66.8-86.9
NASA Checked 21.7 9.3-42.7
NASA Not checked 78.3 57.3-90.7
NIST Checked 31.1 14.3-55.1
NIST Not checked 68.9 44.9-85.7
NOAA Checked 16.6 7.6-32.5
NOAA Not checked 83.4 67.5-92.4
Q23. Through which method(s) were you seeking to disseminate your research
results when you were denied agency approval?
d. Media interviews
[If Question 19 is Yes and Question 22 is Yes]
Agency Question Choices Estimated Percentage 95 PercentConfidence Interval
Total Checked 15.0 7.9-26.5
Total Not checked 85.0 73.5-92.1
NASA Checked 12.8 3.5-37.1
NASA Not checked 87.2 62.9-96.5
NIST Checked 24.0 9.6-48.4
NIST Not checked 76.0 51.6-90.4
NOAA Checked 13.5 5.5-29.6
NOAA Not checked 86.5 70.4-94.5
Q23. Through which method(s) were you seeking to disseminate your research
results when you were denied agency approval?
e. Other - Please specify below.
[If Question 19 is Yes and Question 22 is Yes]
Agency Question Choices Estimated Percentage 95 PercentConfidence Interval
Total Checked 2.8 1.2-6.4
Total Not checked 97.2 93.6-98.8
NASA Checked 0.0
NASA Not checked 100.0
NIST Checked 0.0
NIST Not checked 100.0
NOAA Checked 7.8 3.0-18.8
NOAA Not checked 92.2 81.2-97.0
Q25. Did you take any of the following actions when you did not receive
approval to disseminate your research results?
a. Appealed the decision using established procedures
[If Question 19 is Yes and Question 22 is Yes]
Agency Question Choices Estimated Percentage 95 PercentConfidence Interval
Total Yes 29.7 17.1-46.3
Total No 67.0 50.8-80.0
Total Not sure 3.3 1.0-10.2
NASA Yes 20.9 9.5-39.9
NASA No 79.1 60.1-90.5
NASA Not sure 0.0
NIST Yes 7.7 1.6-29.9
NIST No 82.4 56.6-94.4
NIST Not sure 9.9 1.9-38.2
NOAA Yes 50.0 24.8-75.3
NOAA No 45.8 22.1-71.5
NOAA Not sure 4.2 0.8-19.5
Q25. Did you take any of the following actions when you did not receive
approval to disseminate your research results?
b. Disseminated the results anyway
[If Question 19 is Yes and Question 22 is Yes]
Agency Question Choices Estimated Percentage 95 PercentConfidence Interval
Total Yes 15.7 6.2-34.5
Total No 79.9 62.6-90.5
Total Not sure 4.4 1.7-10.8
NASA Yes 4.1 0.8-18.5
NASA No 95.9 81.5-99.2
NASA Not sure 0.0
NIST Yes 0.0
NIST No 90.1 61.8-98.1
NIST Not sure 9.9 1.9-38.2
NOAA Yes 35.3 13.2-66.2
NOAA No 58.2 29.9-82.0
NOAA Not sure 6.4 2.0-18.6
Q25. Did you take any of the following actions when you did not receive
approval to disseminate your research results?
c. Disseminated the results through a different route
[If Question 19 is Yes and Question 22 is Yes]
Agency Question Choices Estimated Percentage 95 PercentConfidence Interval
Total Yes 36.0 22.6-51.9
Total No 61.1 45.5-74.7
Total Not sure 2.9 0.9-9.0
NASA Yes 17.0 7.0-35.7
NASA No 83.0 64.3-93.0
NASA Not sure 0.0
NIST Yes 43.5 22.6-67.0
NIST No 47.4 25.7-70.1
NIST Not sure 9.1 1.8-36.0
NOAA Yes 53.9 28.0-77.8
NOAA No 43.0 20.1-69.3
NOAA Not sure 3.2 0.7-13.7
Q25. Did you take any of the following actions when you did not receive
approval to disseminate your research results?
d. Added a disclaimer that the opinions expressed in the research results
do not reflect the views of the agency
[If Question 19 is Yes and Question 22 is Yes]
Agency Question Choices Estimated Percentage 95 PercentConfidence Interval
Total Yes 8.1 3.8-16.7
Total No 90.0 80.7-95.1
Total Not sure 1.9 0.4-9.4
NASA Yes 5.3 0.9-25.2
NASA No 94.7 74.8-99.1
95 PercentConfidence Interval
NASA
NIST
NIST
NIST
NOAA
NOAA
NOAA
Q25. Did you take any of the following actions when you did not receive
approval to disseminate your research results?
Estimated Percentage
16.0
82.1
Question Choices
Yes
No
1.8-11.5
47.7-85.9
14.1-52.3
25.2-69.6
16.1-60.2
5.9-44.3
42.3-90.2
7.8-55.8
0.7-14.2
Agency Agency
Total
Total
4.0
9.0
91.0
0.0
7.7
82.4
9.9
3.9
89.9
6.2
[If Question 19 is Yes and Question 22 is Yes]
Not sure
Yes
No
Not sure
Yes
No
Not sure
Yes
No
Not sure
Agency Agency
Total
Total
17.3
57.3
8.0
25.2
15.3
51.5
9.6
16.6
16.5
57.2
5.0
11.5
20.0
Estimated Percentage Estimated Percentage
87.5
5.1
7.4
78.6
5.8
15.6
100.0
0.0
0.0
90.6
9.4
Appendix III: Comments from the Department of Commerce
Appendix IV: Comments from the National Aeronautics and Space
Administration
Appendix V: Comments from the Office of Science and Technology Policy
Appendix VI: GAO Contact and Staff Acknowledgments
GAO Contact
Ms. Anu Mittal, 202-512-3841 or [email protected]
Staff Acknowledgments
In addition to the contact person named above, Cheryl Williams
(Assistant Director), Allen Chan, Nancy Crothers, Elizabeth Erdmann,
Stuart Kaufman, Matthew LaTour, Grant Mallie, Lisa Mirel, and Rebecca
Shea made key contributions to this report.
(360724)
www.gao.gov/cgi-bin/getrpt?GAO-07-653 .
To view the full product, including the scope
and methodology, click on the link above.
For more information, contact Ms. Anu Mittal at (202) 512-3841 or
[email protected].
Highlights of [41]GAO-07-653 , a report to the Chairman, Subcommittee on
Commerce, Justice, Science, and Related Agencies, Committee on
Appropriations, U.S. Senate
May 2007
FEDERAL RESEARCH
Policies Guiding the Dissemination of Scientific Research from Selected
Agencies Should Be Clarified and Better Communicated
Researchers at federal agencies disseminate their research results through
a variety of approaches, including scientific publications, presentations,
press releases, and media interviews. Because of recent concerns about
some federal researchers possibly being restricted from disseminating
their research on controversial topics, GAO determined (1) the policies
that guide the dissemination of federal research at the National
Aeronautics and Space Administration (NASA), the National Institute of
Standards and Technology (NIST), and the National Oceanic and Atmospheric
Administration (NOAA); (2) how effectively these agencies have
communicated their policies to researchers; and (3) the extent to which
researchers have been restricted in disseminating their research. GAO
conducted a survey of 1,811 researchers randomly selected at the three
agencies, and had a 66 percent response rate.
[42]What GAO Recommends
GAO is making recommendations to the Department of Commerce, NASA, NIST,
and NOAA to clarify their dissemination polices for press releases and
media interviews, ensure researchers have a policy for appealing
dissemination decisions, and provide training on these policies.
In commenting on a draft of this report, Commerce, responding for NIST,
NOAA, and itself, generally concurred with GAO's findings and
recommendations, as did NASA.
Most of the NASA, NIST, and NOAA policies that guide the dissemination of
federally funded research generally facilitate the dissemination process,
but some do not. GAO found that overall NASA's policies, including its
recently revised media policy, are clear and should help facilitate
dissemination regardless of the dissemination approach used. At NIST and
NOAA, GAO found that the agencies' policies for dissemination through
publications and presentations were generally clear and should facilitate
dissemination; but their policies for disseminating research through media
interviews and press releases may hinder it. For example, because both
NIST and NOAA are part of Commerce, researchers at these agencies must
comply with department-level policies to disseminate their research
results through media interviews or press releases, but Commerce's
policies are outdated and can prevent researchers from meeting media
schedules. Moreover, requests by NOAA researchers to share their research
via media interviews and press releases may be further hampered because
these researchers must also comply with their own agency's media interview
and press release policies in addition to the Department of Commerce's.
NOAA officials told GAO that because its media interview and press release
policies lack clarity, they have been inconsistently interpreted by NOAA
public affairs officials.
According to GAO's survey, NASA, NIST, and NOAA have made efforts to
communicate their dissemination policies to their research staff, but many
researchers are not confident that they know how to comply with some of
the policies. The agencies have communicated their dissemination policies
through staff meetings, on agency Web sites, and in limited formal
training. While 90 percent of researchers are confident that they
understand the policies for publications, only about 65 percent are
confident they understand their agency's media interview and press release
policies well enough to comply with them. Similarly, almost half of the
researchers across the agencies are unsure whether their agency's policy
allows them to discuss their personal views on the policy implications of
their research. Finally, only 25 percent of researchers across the
agencies are aware of a process to follow to appeal denials of requests to
disseminate their research.
On the basis of responses to GAO's survey, 6 percent--or about 200
researchers--across NASA, NIST, and NOAA had dissemination requests denied
during the last 5 years. One of the most common reasons researchers
mentioned for these denials was that the topic of the research was
sensitive or restricted for security reasons; in some cases, no reason was
given. Most researchers at these agencies believe that their agency is
more supportive of dissemination of research through publications and
presentations, than dissemination through the media. Most NIST and NOAA
researchers believe that their agency consistently applies the
dissemination policies for each route of dissemination, while more
researchers at NASA believe the agency consistently applies its policies
for publications than believe the agency consistently applies its policies
for press releases and interviews.
GAO's Mission
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting its
constitutional responsibilities and to help improve the performance and
accountability of the federal government for the American people. GAO
examines the use of public funds; evaluates federal programs and policies;
and provides analyses, recommendations, and other assistance to help
Congress make informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony
The fastest and easiest way to obtain copies of GAO documents at no cost
is through GAO's Web site ( www.gao.gov ). Each weekday, GAO posts
newly released reports, testimony, and correspondence on its Web site. To
have GAO e-mail you a list of newly posted products every afternoon, go to
www.gao.gov and select "Subscribe to Updates."
Order by Mail or Phone
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent of
Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more
copies mailed to a single address are discounted 25 percent. Orders should
be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
D.C. 20548
To order by Phone: Voice: (202) 512-6000
TDD: (202) 512-2537
Fax: (202) 512-6061
To Report Fraud, Waste, and Abuse in Federal Programs
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm
E-mail: [email protected]
Automated answering system: (800) 424-5454 or (202) 512-7470
Congressional Relations
Gloria Jarmon, Managing Director, [email protected] (202) 512-4400 U.S.
Government Accountability Office, 441 G Street NW, Room 7125 Washington,
D.C. 20548
Public Affairs
Paul Anderson, Managing Director, [email protected] (202) 512-4800
U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, D.C. 20548
References
Visible links
38. http://www.gao.gov/cgi-bin/getrpt?GAO-06-607
39. http://www.gao.gov/cgi-bin/getrpt?GAO-07-340
41. http://www.gao.gov/cgi-bin/getrpt?GAO-07-653
*** End of document. ***