[Federal Register Volume 66, Number 14 (Monday, January 22, 2001)]
[Notices]
[Pages 6593-6597]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 01-1782]


-----------------------------------------------------------------------

DEPARTMENT OF ENERGY


Office of Science; Office of Science Financial Assistance Program 
Notice 01-11: Scientific Discovery through Advanced Computing in High 
Energy and Nuclear Physics Research

AGENCY: U.S. Department of Energy (DOE).

ACTION: Notice inviting research grant applications.

-----------------------------------------------------------------------

SUMMARY: The Office of High Energy and Nuclear Physics (HENP) of the 
Office of Science (SC), U.S. Department of Energy (DOE), hereby 
announces its interest in receiving grant applications for the 
Department's Scientific Discovery

[[Page 6594]]

through Advanced Computing Program (SciDAC). The goal of this program 
is to enable the use of terascale computers to dramatically extend our 
exploration of the fundamental processes of nature as well as to 
advance our ability to predict the behavior of a broad range of complex 
natural and engineered systems. This goal is to be achieved through the 
creation of scientific simulation codes that achieve high performance 
on a single node, scale to hundreds of nodes and thousands of 
processors, and have the potential to adapt over time and to be ported 
to future generations of high performance computers. Projects should 
address a problem of national scientific or engineering significance 
clearly related to the mission of DOE. They are expected to have high 
visibility and to present a long-term vision of how their work will 
fundamentally impact scientific discovery in specific areas of High 
Energy Physics or Nuclear Physics research.
    The full text of Program Notice 01-11 is available via the Internet 
at the following web site address: http://www.science.doe.gov/production/grants/grants.html.

DATES: Preapplications referencing this program notice must be received 
by 4:30 P.M. EST, February 7, 2001. A response encouraging or 
discouraging the submission of a formal application will be 
communicated by E-mail within 14 days.
    Formal applications submitted in response to this notice must be 
received no later than 4:30 P.M., March 15, 2001, to be accepted for 
merit review and consideration for award in Fiscal Year 2001.

ADDRESSES: Preapplications referencing Program Notice 01-11 should be 
forwarded to: U.S. Department of Energy, Office of Science, Office of 
High Energy and Nuclear Physics, SC-20, 19901 Germantown Road, 
Germantown, Maryland 20874-1290, ATTN: Peter Rosen. Preapplications can 
also be submitted via E-mail at the following E-mail address: 
[email protected].
    Formal applications referencing Program Notice 01-11 should be 
forwarded to: U.S. Department of Energy, Office of Science, Grants and 
Contracts Division, SC-64, 19901 Germantown Road, Germantown, Maryland 
20874-1290, ATTN: Program Notice 01-11. The above address must be used 
when submitting applications by U.S. Postal Service Express Mail, any 
commercial mail delivery service, or when hand-carried by the 
applicant. An original and seven copies of the application must be 
submitted.

FOR FURTHER INFORMATION CONTACT: Dr. S. Peter Rosen, Office of High 
Energy and Nuclear Physics, SC-20, U.S. Department of Energy, 19901 
Germantown Road, Germantown, MD 20874-1290, E-mail: 
[email protected].

SUPPLEMENTARY INFORMATION:

Background: Scientific Discovery Through Advanced Computing

    Advanced scientific computing will be a key contributor to 
scientific research in the 21st Century. Within the Office of Science 
(SC), scientific computing programs and facilities are already 
essential to progress in many areas of research critical to the nation. 
Major scientific challenges exist in all SC research programs that can 
best be addressed through advances in scientific supercomputing, e.g., 
designing materials with selected properties, elucidating the structure 
and function of proteins, understanding and controlling plasma 
turbulence, and designing new particle accelerators. To help ensure its 
missions are met, SC is bringing together advanced scientific computing 
and scientific research in an integrated program entitled ``Scientific 
Discovery through Advanced Computing.''

The Opportunity and the Challenge

    Extraordinary advances in computing technology in the past decade 
have set the stage for a major advance in scientific computing. Within 
the next five to ten years, computers 1,000 times faster than today's 
computers will become available. These advances herald a new era in 
scientific computing. Using such computers, it will be possible to 
dramatically extend our exploration of the fundamental processes of 
nature (e.g., the structure of matter from the most elementary 
particles to the building blocks of life) as well as advance our 
ability to predict the behavior of a broad range of complex natural and 
engineered systems (e.g., the earth's climate or an automobile engine).
    To exploit this opportunity, these computing advances must be 
translated into corresponding increases in the performance of the 
scientific codes used to model physical, chemical, and biological 
systems. This is a daunting problem. Current advances in computing 
technology are being driven by market forces in the commercial sector, 
not by scientific computing. Harnessing commercial computing technology 
for scientific research poses problems unlike those encountered in 
previous supercomputers, in magnitude as well as in kind. As noted in 
the 1998 report \1\ from the NSF/DOE ``National Workshop on Advanced 
Scientific Computing'' and the 1999 report \2\ from the President's 
Information Technology Advisory Committee, this problem will only be 
solved by increased investments in computer software--in research and 
development of scientific simulation codes as well as on the 
mathematical and computing systems software that underlie these codes.
---------------------------------------------------------------------------

    \1\ This workshop was sponsored by the National Science 
Foundation and the Department of Energy and hosted by the National 
Academy of Sciences on July 30-31, 1998. Copies of the report may be 
obtained from: http://www.er.doe.gov/production/octr/mics/index.html
    \2\ Copies of the PITAC report may be obtained from http://www.cclc.gov/ac/report/.
---------------------------------------------------------------------------

Investment Plan of the Office of Science

    To meet the challenge posed by the new generation of terascale 
computers, SC will fund a set of coordinated investments as outlined in 
its long-range plan for scientific computing, Scientific Discovery 
through Advanced Computing,\3\ submitted to Congress on March 30, 2000. 
First, it will create a Scientific Computing Software Infrastructure 
that bridges the gap between the advanced computing technologies being 
developed by the computer industry and the scientific research programs 
sponsored by the Office of Science. Specifically, the SC effort 
proposes to:
---------------------------------------------------------------------------

    \3\ Copies of the SC computing plan, Scientific Discovery 
through Advanced Computing, can be downloaded from the SC web site 
at: http://www.sc.doe.gov/production/octr/index.html.
---------------------------------------------------------------------------

     Create a new generation of Scientific Simulation Codes 
that take full advantage of the extraordinary computing capabilities of 
terascale computers.
     Create the Mathematical and Computing Systems Software to 
enable the Scientific Simulation Codes to effectively and efficiently 
use terascale computers.
     Create a Collaboratory Software Environment to enable 
geographically separated scientists to effectively work together as a 
team and to facilitate remote access to both facilities and data.
    These activities are supported by a Scientific Computing Hardware 
Infrastructure that will be tailored to meet the needs of SC's research 
programs. The Hardware Infrastructure is robust, to provide the stable 
computing resources needed by the scientific applications; agile, to 
respond to innovative advances in computer technology that impact 
scientific computing; and flexible, to allow the

[[Page 6595]]

most appropriate and economical resources to be used to solve each 
class of problems. Specifically, the SC proposes to support:
     A Flagship Computing Facility, the National Energy 
Research Scientific Computing Center (NERSC), to provide the robust, 
high-end computing resources needed by a broad range of scientific 
research programs.
     Topical Computing Facilities to provide computing 
resources tailored for specific scientific applications and to serve as 
the focal point for an application community as it strives to optimize 
its use of terascale computers.
     Experimental Computing Facilities to assess the promise of 
new computing technologies being developed by the computer industry for 
scientific applications.
    Both sets of investments will create exciting opportunities for 
teams of researchers from laboratories and universities to create new 
revolutionary computing capabilities for scientific discovery.

The Benefits

    The Scientific Computing Software Infrastructure, along with the 
upgrades to the hardware infrastructure, will enable laboratory and 
university researchers to solve the most challenging scientific 
problems faced by the Office of Science at a level of accuracy and 
detail never before achieved. These developments will have significant 
benefits to all of the government agencies that rely on high-
performance scientific computing to achieve their mission goals as well 
as to the U.S. high-performance computing industry.

Background: Scientific Simulation in High Energy Physics and 
Nuclear Physics Research

    The Office of High Energy and Nuclear Physics supports a program of 
research into the fundamental nature of matter and energy. In carrying 
out this mission it:
     Builds and operates large, world class charged-particle 
accelerator facilities for the nation and for the international 
scientific research community;
     Builds detectors and instruments, for accelerator and non-
accelerator based experiments, designed to answer fundamental questions 
about the nature of matter and energy; and
     Carries out a program of scientific research based on 
experimental data, theoretical studies, and scientific simulation.
    This solicitation is focused on proposals to accelerate progress 
through the use of scientific simulation codes.
    Computational modeling and simulation are among the most 
significant developments in the practice of scientific inquiry in the 
20th century. The coming advances in computing performance, if they can 
be realized for scientific problems, herald a new era in scientific 
computing. If computers capable of 100 teraflops or more become 
available in the next few years, it will be possible to dramatically 
extend our exploration of the fundamental processes of nature. It will 
also be possible to predict the behavior of a broad range of complex 
systems, such as charged-particle accelerator components, and 
eventually entire accelerators.
    However, it is clear that the development of scientific codes that 
are capable of utilizing terascale computers efficiently and are 
adaptable, portable and re-usable is a massive undertaking that could 
take as long as 8-10 years to achieve its most ambitious scientific 
goals. This may require efforts of hundreds of person-years of work.
    It is also apparent that the most appropriate, cost-effective 
computing resources for scientific simulations vary significantly from 
application to application. Therefore, much work is needed to 
understand the optimal configuration of computing hardware for each 
task and to design operating environments best able to foster 
significant scientific discoveries.
    This solicitation is for proposals that articulate the long-term 
vision and potential for scientific progress through simulation, whilst 
laying out a concrete step-wise program of work and scientific research 
for the next 3 to 5 years.
    The scope and complexity of the proposed projects will require 
close collaboration among researchers from computational and 
theoretical physics, computer science, and applied mathematics 
disciplines. Accordingly, this solicitation calls for the creation of 
scientific simulation teams, or collaborations, as the organizational 
basis for a successful application. A scientific simulation team is a 
multi-institutional, multi-disciplinary group of people who will:
     Create scientific simulation codes that take full 
advantage of terascale computers,
     Work closely with other SciDAC teams and centers to ensure 
that the best available mathematical algorithms and computer science 
methods are employed, and
     Manage the work of the team in a way that will foster good 
communication and decision making (see section on Collaboration and 
Coordination below).
    Partnerships among universities, national laboratories, and 
industry are encouraged. Applications are being sought in the broad 
topical areas listed below.

Accelerator Science and Simulation

    The successful development of large accelerator facilities involves 
enormous investments in theory, experiment and simulation. Optimizing 
the performance of current accelerators and the design of future 
accelerators will require unprecedented precision in accelerator 
component design and beam dynamics and control. Applicants should 
explain how the proposed program of work will facilitate important 
design decisions, increase safety and reliability, optimize performance 
and reduce the cost of accelerators.
    The development of a comprehensive, coherent terascale simulation 
environment for the U.S. particle accelerator community will involve 
development of new computational models and codes, mathematical models, 
program frameworks and visualization techniques. The scientific 
software, while making good use of existing codes for (a) calculations 
for the design of complex electromagnetic components and systems and 
(b) beam dynamics calculations for predicting beam halo, must provide 
high performance on terascale computers and be capable of scaling to 
100 teraflops or more. New codes will need to be developed for 
problems, such as electromagnetic modeling of lossy structures and 
wakefields, parallel static computation for electric and magnetic 
component design, and parallel modeling of intense beams in injectors, 
linear and circular machines. Models need to be developed to include a 
range of physical phenomena such as collisions, synchrotron radiation, 
and surface emissions. In order to simulate accelerator components and 
entire accelerators, the scientific simulation codes will need to work 
together to carry out simulations of complex systems involving tight 
coupling of beam dynamics and electromagnetics.
    Collaborative work with Fusion Energy Scientists may also be useful 
since there are some common problems related to modeling 
electromagnetic fields and beam dynamics.

Theoretical Research

    In the past few years, several areas of theoretical research have 
demonstrated

[[Page 6596]]

the potential to further scientific knowledge by efficiently using 
scientific simulation codes on terascale computers to:
     Provide a major quantitative tool for simulations of 
quantum chromodynamics (QCD) on a lattice, which will:
    (a) Provide crucial information in support of the experimental 
programs in high energy and nuclear physics.
    (b) Make accurate determinations of a number of fundamental 
quantities, such as the coupling constant that determines the strength 
of quark-gluon interactions, and the underlying masses of the quarks.
    (c) Explore the limitations, if any, of the ``Standard Model'' of 
particle interactions.
    (d) Explore how quarks and gluons provide the binding and spin of 
the nucleon.
     Develop theoretical models of complex systems under 
extreme conditions, such as:
    (a) Exploration of complex theoretical models of supernovae and 
comparison of the predictions with experimental results.
    (b) Study of the behavior of supersymmetric and other quantum field 
theories.
    Particular areas of interest include, but are not limited to:

Quantum Chromodynamics (QCD)

    The development of a coherent terascale simulation environment for 
the study of QCD that will permit evolution of scientific codes to take 
advantage of 100 teraflop computers is a challenging problem. It 
demands a coordinated effort to provide the computer software 
infrastructure, the detailed scientific codes and algorithms, together 
with effective ways of using computing hardware now and in the future.

Simulations of Complex Nuclear Structure, Such as Found in Core-
Collapse Supernovae

    The development of a comprehensive model that brings together 
nuclear physics, particle physics, fluid dynamics, radiation transport, 
and general relativity is an equally challenging problem. Data from 
next-generation neutrino detectors, gravitational wave observatories, 
ground and space-based observatories, new radioactive beam facilities, 
and other experimental facilities will provide opportunities to 
evaluate and refine the many underlying physical models in the 
simulation.

Testbeds and Collaboratory Software Environments

    Collaboratories link geographically dispersed researchers, data and 
tools, via high performance networks, to enable remote access to 
facilities, access to large datasets and shared environments. They 
enable geographically separated scientists to effectively work together 
as a team and facilitate remote access to both computing facilities and 
data.
    As the size and complexity of high energy and nuclear physics 
experiments has increased so have the number and geographical 
dispersion of the researchers and the amount of data that must be 
collected, simulated and analyzed. Thus future experiments critically 
depend on the existence of such distributed hardware and software 
environments for their success. The scientific simulation applications 
that are the focus of this solicitation will also consist of 
geographically dispersed researchers, and will require high performance 
networks, to enable remote access to computing facilities, and multi-
terabyte datasets.
    Proposals for testbeds and collaborations across organizations that 
include network researchers, middleware developers and high energy and 
nuclear physicists are encouraged. However, they should be submitted in 
response to Notice 01-06 of the Office of Advanced Scientific Computing 
Research (ASCR). Copies should also be submitted to the Office of High 
Energy and Nuclear Physics, and joint funding can be considered.

Collaboration and Coordination

    It is expected that all applications submitted in response to this 
notice will be for scientific simulation teams involving more than one 
institution. Applications from different institutions, directed at a 
common research activity, must include a common technical description 
of the overall research project. Each participating institution must 
have a qualified principal investigator, who is responsible for the 
part of the effort at that institution, and separate face pages and 
budget pages for each institution. The distinct scope of work proposed 
for each institution must be clearly specified. Any work proposed in 
computer science or applied mathematics should also be described 
separately. Applicants should include cost sharing whenever feasible. 
Synergistic collaborations with researchers in federal laboratories and 
Federally Funded Research and Development Centers (FFRDCs), including 
the DOE National Laboratories are encouraged, although funds will not 
be provided to these organizations under this particular Notice. 
Further information on preparation of collaborative proposals is 
available in the Application Guide for the Office of Science Financial 
Assistance Program that is available via the Internet at: http://www.science.doe.gov/production/grants/Colab.html.

Preapplications

    Potential applicants are strongly encouraged, but not required, to 
submit a brief preapplication consisting of two or three pages of 
narrative describing the research objectives, technical approaches and 
management plan. Each preapplication should include a cover sheet with 
the title of the project, project principal investigator, institutions 
involved, and their principal investigators and senior personnel. The 
name, telephone number, and e-mail address of each principal 
investigator should also be provided. In addition, brief, one-page 
curriculum vitae should be submitted for the principal investigators 
and other senior personnel involved. Preapplications will be evaluated 
to assess their programmatic relevance, and a response will be provided 
to the principal investigator within 14 days of receipt. However, 
notification of a successful preapplication is not an indication that 
an award will be made in response to a formal application.

Program Funding

    Up to $2,500,000 of Fiscal Year 2001 funding will be available for 
grant awards in FY 2001. Additional funding for each proposed project 
may be available through the Office of Advanced Scientific Computing 
Research for closely related research in computer science and/or 
applied mathematics. Applications may request support for up to three 
years, with out-year support contingent on the availability of funds 
and satisfactory progress. To support multi-disciplinary, multi-
institutional efforts, funding levels of up to $1.0 million per project 
may be requested, under this notice, for the first year of the project. 
Requests for increased funding levels in future years will be 
entertained subject to availability of funds, progress of the funded 
activity, and programmatic needs.
    As required by the SC Grant Application Guide, applicants must 
submit their budgets using the Budget Page (DOE Form 4620.1) with one 
Budget Page for each year of requested funding. The requested funding 
for the

[[Page 6597]]

proposed work in computer science and applied mathematics should be 
included with the other project costs on the Budget Page. However, 
applicants are also requested to list the proposed computer science and 
applied mathematics costs separately in an appendix, as the Office of 
Advanced Scientific Computing Research may support this part of the 
work (up to 20-25% of the total project cost). The Office of High 
Energy and Nuclear Physics expects to fund three or four successful 
projects, depending on the size of the awards.

Evaluation Criteria

    Applications will be subjected to scientific merit review (peer 
review) and will be evaluated against the following criteria listed in 
descending order of importance as codified in 10 CFR 605.10(d) 
(www.science.doe.gov/production/grants/605index.html):
    1. Scientific and/or technical merit of the project,
    2. Appropriateness of the proposed method or approach,
    3. Competency of the applicant's personnel and adequacy of the 
proposed resources,
    4. Reasonableness and appropriateness of the proposed budget.
    The evaluation of applications under item 1, Scientific and 
Technical Merit, will pay particular attention to:
    (a) the potential of the proposed project to achieve a major 
advance in high energy and/or nuclear physics;
    (b) the potential of the proposed project to advance the state-of-
the-art in computational modeling and simulation in areas pertinent to 
high energy and nuclear physics research;
    (c) the need for extraordinary computing resources to address 
problems of critical scientific importance to the high energy physics 
or nuclear physics program and the demonstrated abilities of the 
applicants to exploit terascale computers;
    (d) knowledge of and coupling to previous efforts in scientific 
simulation;
    (e) the extent to which the project incorporates broad community 
(industry/academia/other federal programs) interaction;
    (f) the extent to which the results of the project are likely to be 
extensible to other program or discipline areas; and
    (g) the importance of the proposed project to the mission of the 
Office of High Energy and Nuclear Physics and its impact on overall DOE 
goals.
    The evaluation under item 2, Appropriateness of the Proposed Method 
or Approach, will also consider the following elements related to 
appropriateness of the proposed Scientific Computing Hardware 
Infrastructure to be used and of the quality of planning:
    (a) Viability of the plan with respect to the scale and nature of 
current and future Computing Hardware Infrastructure needed;
    (b) clarity of the plan in detailing areas of work to be addressed 
by discipline scientists, computational scientists, applied 
mathematicians, computer scientists and computer programmers;
    (c) quality of the plan for effective collaboration among 
participants;
    (d) quality of the plan for ensuring communication with other 
advanced computation and simulation efforts;
    (e) viability of the plan for deploying the software and for 
assuring long-term maintenance, support, and re-use of the scientific 
codes and software infrastructure developed;
    (f) viability of the plan for verifying and validating the models 
developed, including verification using experiment results; and
    (g) quality and clarity of the proposed work schedule and project 
deliverables.
    The evaluation will include program policy factors such as the 
relevance of the proposed research to the terms of the announcement and 
the agency's programmatic needs.
    Note, that external peer reviewers are selected with regard to both 
their scientific expertise and the absence of conflict-of-interest 
issues. Non-federal reviewers may be used, and submission of an 
application constitutes agreement that this is acceptable to the 
investigator(s) and the submitting institution.
    General information about development and submission of 
applications, eligibility, limitations, evaluations and selection 
processes, and other policies and procedures may be found in the 
Application Guide for the Office of Science (SC) Financial Assistance 
Program and in 10 CFR part 605. Electronic access to SC's Financial 
Assistance Guide and required forms is made available via the Internet 
using the following Web site address: http://www.science.doe.gov/production/grants/grants.html.
    In addition, for this notice, project descriptions must be 25 pages 
or less, including tables and figures, but excluding attachments. The 
application must also contain an abstract or project summary, letters 
of intent from all non-funded collaborators, and short curriculum vitae 
of all senior personnel. On the SC grant Face Page (DOE Form 4650.2), 
in block 15, also provide the Principal Investigator's phone number, 
FAX number, and E-mail address.
    The Catalog of Federal Domestic Assistance Number for this program 
is 81.049, and the solicitation control number is ERFAP 10 CFR Part 
605.

    Issued in Washington DC on January 10, 2001.
John Rodney Clark,
Associate Director of Science for Resource Management.
[FR Doc. 01-1782 Filed 1-19-01; 8:45 am]
BILLING CODE 6450-01-U