[Federal Register Volume 88, Number 223 (Tuesday, November 21, 2023)]
[Notices]
[Pages 81108-81110]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 2023-25718]


=======================================================================
-----------------------------------------------------------------------

NATIONAL SCIENCE FOUNDATION


Notice of Intent To Seek Approval To Establish an Information 
Collection

AGENCY: National Science Foundation.

ACTION: Notice and request for comments.

-----------------------------------------------------------------------

SUMMARY: Under the Paperwork Reduction Act of 1995, and as part of its 
continuing effort to reduce paperwork and respondent burden, the 
Evaluation and Assessment Capability (EAC), National Science Foundation 
(NSF) Office of Integrative Activities (OIA) is inviting the general 
public or other Federal agencies to comment on this proposed 
information collection.

DATES: Written comments on this notice must be received by January 22, 
2024 to be assured consideration. Comments received after that date 
will be considered to the extent practicable. Send comments to the 
address below.

FOR FURTHER INFORMATION CONTACT: Suzanne H. Plimpton, Reports Clearance 
Officer, National Science Foundation, 2415 Eisenhower Avenue, Suite 
E7400, Alexandria, Virginia 22314; telephone (703) 292-7556; or send 
email to [email protected]. Individuals who use a telecommunications 
device for the deaf (TDD) may call the Federal Information Relay 
Service (FIRS) at 1-

[[Page 81109]]

800-877-8339, which is accessible 24 hours a day, 7 days a week, 365 
days a year (including federal holidays).

SUPPLEMENTARY INFORMATION: 
    Comments: Comments are invited on: (1) whether the proposed 
collection of information is necessary for the proper performance of 
the functions of the Foundation, including whether the information will 
have practical utility; (2) the accuracy of the Foundation's estimate 
of the burden of the proposed collection of information; (3) ways to 
enhance the quality, utility, and clarity of the information to be 
collected; and (4) ways to minimize the burden of collecting the 
information on the respondents, including through the use of automated 
collection techniques or other forms of information technology.
    Title of Collection: Generic Clearance for the Evaluation of the 
National Science Foundation's (NSF) Broader Impacts Review Criterion.
    OMB Number: 3145-NEW.
    Expiration Date of Approval: Not applicable.
    Type of Request: New information collection.
    Description: NSF is conducting an evaluation to assess (1) how 
NSF's Broader Impacts review criterion is applied across the Foundation 
and (2) its effectiveness in meeting the goals established in section 
526 of the America Creating Opportunities to Meaningfully Promote 
Excellence in Technology, Education, and Science Reauthorization Act of 
2010 (42 U.S.C. 1862p-14) (America COMPETES Reauthorization Act of 
2010). This evaluation is congressionally directed in section 10341 of 
the Creating Helpful Incentives to Produce Semiconductors [CHIPS] for 
America Fund Act 2022. As part of the evaluation, NSF is conducting a 
literature review, document analysis, extant data analysis, interviews 
with NSF staff, and focus groups with NSF principal investigators (PIs) 
and reviewers. NSF will map findings from the evaluation activities to 
current NSF policies and practices to identify strategies for improving 
how NSF applies the review criterion.
    The subject of this request is related to the planned focus groups 
with PIs and reviewers. The focus groups will answer the following 
research questions (RQs):
     RQ1. In what ways do the interpretations of the Broader 
Impacts review criterion among PIs and reviewers vary, and what factors 
might contribute to these variations?
     RQ2. How do external reviewers assess the Broader Impact 
review criterion?
     RQ3. In what ways do PIs and reviewers perceive that 
variations in interpretation and assessment can advance or hinder the 
merit review of proposals?
    Findings from the focus groups described in this request will be 
used to inform interpretation of other evaluation activities within the 
larger project (including informing interpretation of interviews with 
NSF staff, document review analyses, and interpretation of extant data 
analysis of review analyses). For example, we anticipate that 
participants in these focus groups may raise issues around their 
understanding and interpretation of Broader Impacts, which can be 
compared to perceptions that NSF staff report during interviews.
    Background:
    NSF sets forth an ambitious vision for the United States: a nation 
that leads the world in science and engineering research and 
innovation, to the benefit of all, without barriers to participation. 
Toward this end, NSF promotes the progress of science by investing in 
research and capacity-building activities that expand knowledge in 
science, engineering, and education. In fiscal year (FY) 2022, NSF 
evaluated almost 40,000 proposals for research and education 
activities, making nearly 11,000 new awards totaling more than $8.5 
billion.
    At the cornerstone of NSF's mission and its investments is its 
merit review process. NSF program directors with technical and 
programmatic expertise lead this process, with support from external 
experts who help evaluate submitted proposals for two main criteria: 
(1) Intellectual Merit--the potential to advance knowledge; and (2) 
Broader Impacts--the potential to contribute to society and achieve 
specific, desired societal outcomes. With these two criteria, NSF has 
established a commitment to projects that provide tangible benefits to 
society beyond advancing knowledge.
    It is critically important that NSF implement its merit review 
process in a way that is fair, thorough, competitive, and transparent, 
and that those internal and external to NSF recognize the process as 
such. However, as NSF noted, PIs and reviewers might lack clarity about 
the Broader Impacts criterion, despite NSF's efforts to provide 
additional guidance. NSF has also noted a lack of consistency in how 
NSF implements the criterion across directorates, divisions, and 
programs. Specific challenges related to the understanding and 
application of Broader Impacts include a lack of consensus on how to 
define Broader Impacts, and a disconnect between the Broader Impacts 
requirements stated in the NSF Proposal & Award Policies & Procedures 
Guide and how panelists review these activities (National Alliance for 
Broader Impacts 2018). The purpose of this work, then, is to ``assess 
how the Broader Impact review criterion is applied across the 
Foundation and make recommendations for improving the effectiveness for 
meeting the goals established in section 526 of the America Creating 
Opportunities to Meaningfully Promote Excellence in Technology, 
Education, and Science Reauthorization Act of 2010 (42 U.S.C. 1862p-
14)'' (America COMPETES Reauthorization Act of 2010). This evaluation 
is congressionally directed in section 10341 of the Creating Helpful 
Incentives to Produce Semiconductors [CHIPS] for America Fund Act 2022.
    Methodology: Focus groups will be conducted with two types of 
respondents: Pls and reviewers.
    The evaluation will include three PI focus groups of up to seven 
people each. Participants in these groups will be PIs who submitted a 
proposal within the last five years. The study team will select 
participants via a stratified random sample by NSF directorate, 
institutional characteristics (such as Carnegie classification, MSI 
status, and locale), and participant characteristics (such as race/
ethnicity, gender, years since terminal degree, and new investigator 
status). PIs have firsthand experience addressing the Broader Impacts 
review criterion in their proposals. Among this group, key insights 
include the following:
    1. Questions they have about how to address the Broader Impacts 
review criterion in their research and proposals.
    2. Strategies they have employed as a PI in addressing the Broader 
Impacts review criterion in their research and proposals.
    3. Resources or supports received from their respective 
institutions for developing well-thought-out proposals that address the 
Broader Impacts review criterion.
    Reviewer focus groups will consist of three focus groups of up to 
seven people each. Participants in these groups will be people who 
served on a review panel within the last five years. The study team 
will select participants via a stratified random sample by directorate 
and participant characteristics (such as how long they have been 
reviewing NSF proposals). Reviewers have firsthand knowledge about 
applying the Broader Impacts review criterion. Among this group, key 
insights include the following:

[[Page 81110]]

    1. Interpretating and applying the criterion as a reviewer (and 
compared with as a PI).
    2. Reviewer training and guidance.
    Affected Public: NSF reviewers and PIs.

                                                      Average Expected Annual Number of Activities
--------------------------------------------------------------------------------------------------------------------------------------------------------
                                                                                                    Estimated
                                                           Estimated lower     Estimated upper       average       Approximate lower   Approximate upper
                   Collection method                      bound  (number of   bound  (number of   response time     bound response      bound response
                                                             responses)          responses)           (min)         burden (hours)      burden (hours)
--------------------------------------------------------------------------------------------------------------------------------------------------------
Focus groups...........................................             4*6=24              7*6=42               90       (24*90)/60=36       (42*90)/60=72
--------------------------------------------------------------------------------------------------------------------------------------------------------

    Respondents: Lower-bound estimate of 24 individuals and upper-bound 
estimate of 48 individuals.
    Average Minutes per Response: 90.
    Burden Hours: Lower- and upper-bound estimates of approximately 36 
and 72 hours.

    Dated: November 16, 2023.
Suzanne H. Plimpton,
Reports Clearance Officer, National Science Foundation.
[FR Doc. 2023-25718 Filed 11-20-23; 8:45 am]
BILLING CODE 7555-01-P