Science And Technology: Air Force's Planning Process Meets	 
Statutory Requirement (14-FEB-02, GAO-02-273).			 
                                                                 
Congress and the scientific community are concerned that the Air 
Force's investment in science and technology may be too low to	 
meet the challenges presented by new and emerging threats. The	 
National Defense Authorization Act for Fiscal Year 2001 requires 
the Air Force to review its science and technology programs to	 
assess the budgetary resources currently used and those needed to
adequately address the challenges and objectives. GAO found that 
the Air Force complied with the requirements of section 252 of	 
the act. The Air Force established an integrated product team to 
identify long-term science and technology challenges and a task  
force to identify short-term objectives. For each challenge or	 
objective that was identified, the Air Force established teams to
identify technological capabilities needed to achieve these	 
goals. Each team chose research projects that addressed the	 
criteria specified in the act. The Air Force also complied with  
the act's process provisions. The Deputy Assistant Director for  
Science, Technology and Engineering is required to review the	 
teams' results and to identify any science and technology	 
research not currently funded.					 
-------------------------Indexing Terms------------------------- 
REPORTNUM:   GAO-02-273 					        
    ACCNO:   A02750						        
  TITLE:     Science And Technology: Air Force's Planning Process     
Meets Statutory Requirement					 
     DATE:   02/14/2002 
  SUBJECT:   Air Force procurement				 
	     Defense capabilities				 
	     Military operations				 
	     Tactical air forces				 
	     Weapons systems					 
	     F-117 Aircraft					 
	     Stealth Aircraft					 

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO Product.                                                 **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
******************************************************************
GAO-02-273
     
United States General Accounting Office

GAO Report to Congressional Committees

February 2002

                                SCIENCE AND
                                 TECHNOLOGY

Air Force's Planning Process Meets Statutory Requirement

                                      a

Contents

  Letter                                                                                 1
                                        Results in Brief                                 2
                                           Background                                    2
                              Long-Term Planning Complies with Act                       3
                           Short-Term Planning Satisfies Requirements                    6
                       Program and Resource Assessment Complies with Act                 9
                                        Recommendations                                 11
                                        Agency Comments                                 11

Appendixes

Appendix
I:
Scope and Methodology

Tables   Table 1:  Long-Term  Challenge  Checklist  4   Table 2:  Short-Term
Challenge Checklist 7 Table 3: Resource Assessment Checklist 10

A

United States General Accounting Office
Washington, D.C. 20548

February 14, 2002

The Honorable Carl Levin Chairman The Honorable John W. Warner Ranking
Minority Member Committee on Armed Services United States Senate

The Honorable Bob Stump Chairman The Honorable Ike Skelton Ranking Minority
Member Committee on Armed Services House of Representatives

The successful outcome of military operations often depends on weapons
systems that emerged from research conducted decades earlier. For example,
the Air Force's investment in low observable stealth technology in the 1970s
led to the F-117 fighter, which was very effective in Operation Desert
Storm. Since the end of the Cold War, however, the focus of science and
technology has narrowed as a consequence of steady funding declines. The Air
Force has been criticized for focusing its research more on existing or
emerging weapons and less on long-term technology development. There is a
growing concern in the scientific community that the Air Force's investment
in science and technology may be too low to meet the challenges presented by
new and emerging threats.

The Congress has also been concerned about the Air Force's declining
investment in science and technology. The National Defense Authorization Act
for Fiscal Year 20011 requires that the secretary of the Air Force review
the long-term challenges and short-term objectives of the Air Force's
science and technology programs. The act further requires that the review
assess the budgetary resources currently used and those needed to adequately
address the challenges and objectives. The act also requires that we assess
the extent to which the Air Force has complied with the provisions of the
act and report to the Congress on the results of our review. In discussions
with your offices, we agreed to focus our review on whether the Air Force
complied with the criteria specified in the act and

1 Public Law 106-398, Oct. 30, 2000.

the process-related requirements of  the act and not on the technical merits
of the research projects identified. This report addresses the three primary
areas specified in the act:

* long-term challenge identification and planning,

* short-term objective identification and planning,

* program and budgetary resource assessment.

Results in Brief The Air Force complied with the requirements of section 252
of the Fiscal Year 2001 National Defense Authorization Act. As required, the
Air Force established an integrated product team to identify long-term
science and technology challenges and a task force to identify short-term
objectives. For each challenge or objective that was identified, the Air
Force complied with the provision to establish teams to identify potential
technological capabilities needed to achieve these goals. Each team selected
research projects that addressed the criteria specified in the act. For
example, each short-term objective represented a compelling Air Force
requirement, as the act specified. The Air Force also complied with the
act's process provisions. For example, the long-term challenge teams met the
requirement to conduct workshops to identify promising areas of research.
Upon completion of the planning process, the secretary of the Air Force
delegated to the deputy assistant secretary of the Air Force for Science,
Technology and Engineering, the requirement to review the results of the
teams' work and identify any science and technology research not currently
funded. The deputy complied with the act's review provisions. The Department
of Defense has reviewed this report and concurs with its contents.

Background Science and technology is traditionally divided into three broad
categories: basic research, applied research, and advanced technology
development. Basic research attempts to produce new knowledge in a
scientific or technological area. This research is not associated with a
specific weapon system. Applied research supports the development and
maturation of new technologies for a defined military application. Advanced
development entails large-scale hardware development and technology
integration in more operationally realistic settings. Research and

development beyond these categories is done in support of a specific weapon
system.

In the Air Force, the focal point for science and technology investments is
the Air Force Research Laboratory. It was created in 1997 to centrally
manage all Air Force science and technology efforts. Previously, the Air
Force operated 13 different laboratories across the country. The present Air
Force Research Laboratory, headquartered at Wright Patterson Air Force Base,
comprises 10 technology directorates. Nine directorates handle applied and
advanced development projects. The 10th directorate, the Office of
Scientific Research, manages the Air Force's basic research projects. The
Air Force Research Laboratory biennially generates a comprehensive strategic
plan that supports the national military strategy and the Air Force
Strategic Plan.

In the past, the Air Force was a leader in high-technology exploration.
According to a January 2000 Air Force Association study, the Air Force was
the unquestioned leader in science and technology investments at the end of
the Cold War.2 In the 1990s, however, it dropped to third place, behind the
Army and Navy. The Congress has been concerned about the Air Force's level
of investment in science and technology. For fiscal year 2000, the House and
Senate Armed Services Committees noted that the Air Force in particular, had
failed to comply with the science and technology funding objective specified
in the prior year's authorization act, thus jeopardizing the stability of
the technology base and increasing the risk of failure to maintain
technological superiority in future weapons systems. In 2001, the Scientific
Advisory Board found that the Air Force's science and technology program
needed to improve its planning process and generate stronger user support
and sponsorship.3 It also found weaknesses in the connection between
operational requirements and science and technology programs, which
inhibited the prioritization of investments.

Long-Term Planning The Air Force complied with the overall requirements of
the National Defense Authorization Act regarding long-term challenges. (See
table 1 forComplies with Act the checklist of provisions.)

2See AFA Special Report: Shortchanging the Future, Air Force Association
(Jan. 2000).

3 See  Report on Science & Technology and the  Air Force Vision: Achieving a
More Effective S&T Program, Scientific Advisory Board (May 2001).

Table 1: Long-Term Challenge Checklist

The act defined a long-term challenge as a high-risk, high-payoff effort
that will provide a focus for research in the next 20 to 50 years. To
identify potential long-term challenges, an Air Force review team obtained
over 140 ideas from a variety of sources in the scientific community. Ideas
ranged from cloaking technologies (the deceptive masking of assets) and
holodeck command capabilities (virtual reality battlespace control) to micro
weapons like ubiquitous "battle bees" (miniaturized unmanned air vehicles)
and cyber warfare technologies.

The team evaluated these ideas to ensure that they complied with the three
primary criteria specified in the act. The potential long-term challenges
had to involve (1) compelling Air Force requirements; (2) high-risk,
high-payoff areas of exploration; and (3) very difficult but achievable
results. Yet another provision in the act required that the team should
avoid selecting projects that are linear extensions of ongoing science and
technology projects. This provision was more difficult to assess, but after

additional deliberations, the team determined that the following six
challenges satisfied the criteria in the act:

* Finding and Tracking. To provide the decision maker with target quality
information from anywhere in near real-time.

* Command and Control. To assess, plan, and direct aerospace operations from
anywhere or from multiple locations in near real-time.

* Controlled Effects. To create precise effects rapidly, with the ability to
retarget quickly against complex target sets anywhere, anytime, for as long
as required.

* Sanctuary. To protect our total force from natural and man-made hazards or
threats, allowing us to operate anywhere with the lowest risk possible.

* Rapid Aerospace Response. To respond as quickly as necessary to support
peacetime operations or crises and move this response to another location
very rapidly if needed.

* Effective Aerospace Persistence. To sustain the flow of equipment and
supplies as well as the application of force for as long as required.

Once the long-term challenges were identified, the Air Force followed the
planning process specified in the act. For example, it established six work
groups tasked with identifying possible approaches to address these
challenges. The groups had about 9 weeks to complete their work. As
required, a technical coordinator, assisted by a management coordinator,
headed each group. Each group also complied with the requirement to hold a
workshop within the science and technology community to obtain suggestions
on possible approaches and promising areas of research. The workshop
participants satisfied the requirement to identify current work that
addresses the challenge, deficiencies in current work, and promising areas
of research.

Finally, the groups were also expected to select projects that were not
linear extensions of current science and technology work. This particular
provision was not easy for some groups to define. Some pondered the relative
nature of the term. For example, a user would perceive "nonlinearity"
differently than a scientist. Another group characterized it as a quantum
leap in capability. Another definition associated nonlinear

projects with multiple-capability dimensions. For example, if doubling the
payload capacity of a weapon is a linear extension, then doubling the
payload, speed, and range of the weapon would also be a nonlinear extension.
Regardless of the definition selected, each group addressed the issue in its
planning process.

Each group summarized the results of its workshop in a briefing that
contained enabling capabilities, research areas, technology roadmaps, and
associated funding requirements. In many cases, the level of funding
projections was double or triple the level of the planned budget. For
example, the level of funding projections for basic research in physics,
materials, mathematics, and computer science was more than triple the
planned investment levels.

Short-Term Planning Satisfies Requirements

The Air Force complied with the overall provisions of the National Defense
Authorization Act regarding short-term objectives. (See table 2 for the
checklist of provisions.) As required, the Air Force established a task
force consisting of representatives from the Air Force Chief of Staff and
combatant commands to identify short-term objectives. The task force
obtained about 58 ideas from the requirements, user, and acquisition
communities as specified in the act. Because of the mandated short-term
focus, most of the input involved enhancing or accelerating ongoing research
efforts-not initiating entirely new areas of research. These ideas included
maintaining aging aircraft, combat identification, and time-critical
targeting. While these are not new concepts, they still present significant
technological challenges. We have recently reported on weaknesses in each of
these areas.4

4See U. S. General Accounting Office, Tactical Aircraft: Modernization Plans
Will Not Reduce Average Age of Aircraft, GAO-01-163 (Washington, D. C.:
2001); Combat Identification Systems: Strengthened Management Efforts Needed
to Ensure Required Capabilities, GAO-01-632 (Washington, D. C.: 2001); and
Joint Warfighting: Attacking Time-Critical Targets, GAO-02-204R (Washington,
D.C.: 2001).

Table 2: Short-Term Challenge Checklist

The task force reviewed each idea to ensure that it complied with the
criteria in the act: (1) to involve compelling Air Force requirements, (2)
to have support within the user community, and (3) to likely attain the
desired benefits within 5 years. To ensure that each idea represented a
compelling Air Force requirement, the task force evaluated each idea against
the Air Force's core competencies and critical future capabilities. To meet
the user support requirement, the task force linked each potential
short-term objective to specific mission needs and requirements documents.
The objectives were reviewed and approved by the Air Force's corporate

structure. To ensure that the projects selected would achieve results in 5
years, the task force decided to use the technology maturity levels
highlighted in a recent GAO report.5 The following is a list of the eight
short-term objectives.

* Target Location, Identification, and Track. To detect, locate, track, and
identify air/ground targets anytime in countermeasure environments in near
real time.

* Command, Control, Communication, Computers, and Intelligence. To
dynamically assess, plan, and execute global missions.

* Precision Attack. To engage air and ground targets from manned and
unmanned vehicles with the precision and speed necessary to bring about
decisive results.

* Space Control. To increase the survivability of critical space assets.

* Access to Space. To improve access to space through responsive,
cost-effective launch systems.

* Aircraft Survivability and Countermeasures. To improve the ability to
survive and operate against airborne and ground threats in all environments.

* Sustaining Aging Systems. To extend the service life of aging aircraft and
space launch systems with reduced manpower, reduced total ownership costs,
and enhanced reliability.

* Air Expeditionary Forces Support. To provide air expeditionary forces with
the ability to operate with highly responsive and agile combat support
forces.

After the objectives were identified, the Air Force complied with the
planning process specified in the act. As required, it established an
integrated product team to address each short-term objective. Each team was
composed of a cross-cutting mix of officials from the requirements,

5See U. S. General Accounting Office, Best Practices: Better Management of
Technology Development Can Improve Weapon System Outcomes, GAO/NSIAD-99-162
(Washington, D.C.: 1999).

user, and science and technology communities, as the act specified.
According to many of the short-term objective team leaders, the
cross-cutting nature of the teams was very productive. Not only did they
believe that their planning was enhanced by the direct input from users and
requirements officials, they also believed that the expertise and assistance
from scientists in other laboratory directorates improved the process.

Each team satisfied the requirement to identify, define, and prioritize the
enabling capabilities necessary to meet the objectives. As required, each
team identified the deficiencies in the enabling capabilities and projects
necessary to eliminate the deficiencies. The teams summarized their work in
briefings that contained prioritized lists of enabling capabilities, a
definition of the objectives, technology roadmaps, and budget spreadsheets.
The spreadsheets detailed the current and additional funding required to
achieve the objectives. Obtaining the additional funding was a concern to
many teams. Many teams identified funding requirements that greatly exceeded
current funding levels; it was not uncommon for proposed annual funding
levels to double or triple the level currently projected. For example, the
Command, Control, Communication, Computer, and Intelligence team proposed
programs that would require from 2.6 to over 4 times the planned annual
investment.

Another concern was the 15-year gap between the short-term objective and
long-term challenge planning. According to the act's provisions, the
short-term teams were required to focus on technologies that would be mature
in 5 years; the long-term teams focused on technologies needed 20 to 50
years in the future. According to laboratory officials, this mid-term gap
constitutes much of the normal science and technology planning effort and
represents a critical point in science and technology project development.
This time frame is where science and technology can have a significant
impact. The Air Force currently addresses this time frame in its normal
planning process. In addition, this period is covered in the long-term
challenge technology roadmaps, at least for the research efforts associated
with those six challenges.

Program and Resource The Air Force satisfied the top-level review
requirements in the act. (See

table 3 for the checklist of provisions.) The act required the secretary
ofAssessment Complies the Air Force to conduct a timely review of the
science and technologywith Act programs and to assess the budgetary
resources needed to address the

long-and short-term needs. The secretary delegated this responsibility to

the deputy assistant secretary for Science, Technology and Engineering.

The deputy complied with the requirement to conduct a review of the long-and
short-term science and technology programs within the 1 year time limit
specified in the act. On October 25, 2001, the deputy briefed the secretary
on the final results and received his approval.

Table 3: Resource Assessment Checklist

a Not applicable: No course of action was needed because the secretary found
that these programs did address the long-or short-term objectives.

The act also required the secretary to assess the fiscal year 2001 budget
resources used and needed to adequately address science and technology
needs. After consultation with representatives from the House and Senate
Armed Services Committees, however, the deputy changed the budget baseline
to fiscal year 2002. This was done to reflect the science and technology
budget realignment occurring in fiscal year 2002. The deputy assessed the
2002 budget resources planned for science and technology programs and
determined that they were adequately funded. The deputy noted, however, that
the current level of funding would enable the programs to pursue the minimum
level of scientific research. Additional funding would be required to pursue
other projects. The deputy also

complied with the provision to evaluate whether the ongoing and projected
science and technology programs addressed the long- and short-term science
and technology needs. He determined that the programs did address these
needs, thus obviating the requirement to develop a course of action for
science and technology programs that do not address the long-term challenges
or short-term objectives.

Finally, the act required the secretary to review the long-term challenges
and short-term objectives and to identify additional work that should be
undertaken to meet the challenges and objectives. The deputy complied with
both provisions. Not only did he review the results of the long- and
short-term planning efforts and identify additional work, but he also
directed that the additional work be incorporated into the laboratory's
future planning, programming, and budget decisions. The deputy was in a
unique position to address these requirements. He served not only as the
overall review director for the science and technology planning process, but
also as the chairman of the short-term objective task force. As a result,
the deputy had many opportunities to review the work of both the long-term
challenge and particularly the short-term objective planning teams.

Recommendations  Because the Air  Force complied with the  provisions of the
act, we are not making any recommendations in this report.

Agency  Comments  The  Department of  Defense has  reviewed this  report and
concurs with its contents.

   We conducted our work from May 2001 to January 2002 in compliance with
 generally accepted auditing standards. Additional information on our scope

and methodology is located in appendix I. If you have any questions about
the information contained in this letter, please call me at (202) 512-4530.
Major contributors to this work included Robert Murphy, Rae Ann Sapp,
and Kristin Pamperin.

Sincerely yours,

James F. Wiggins
Director
Acquisition and Sourcing Management

Appendix I

Scope and Methodology

To document the extent to which the Air Force complied with the long-term
planning process specified in the National Defense Authorization Act for
Fiscal Year 2001, we obtained appointment letters, membership rosters,
initial guidance and work plans, meeting schedules, biographies of each
technical coordinator, and a comprehensive listing of the initial long-term
challenge ideas. We also obtained minutes from team meetings, weekly
activity reports, E-mail communications, interim and final briefing reports,
associated studies, workshop agendas and results, current and projected
budget spreadsheets, capability lists, and promising research areas. To
discuss how each team addressed the act's provisions, we met with each
long-term challenge technical coordinator and management coordinator. We
also met with officials from the Air Force Research Laboratory's
headquarters and the Office of the Deputy Assistant Secretary for Science,
Technology, and Engineering. Finally, we physically observed the proceedings
of one long-term challenge workshop over the course of 2 days. To determine
whether each provision was addressed, we prepared summary checklists for
each long-term challenge and keyed the data back to a specific provision of
the act.

To document the extent to which the Air Force complied with the short-term
objective planning process specified in the act, we obtained appointment
letters, membership rosters, initial guidance and work plans, meeting
schedules, and a comprehensive listing of the initial short-term objective
ideas. We also obtained weekly activity reports, short-term objective
descriptive summaries, meeting minutes, E-mail communications, interim and
final briefing reports, current and projected budget spreadsheets, and
prioritized listings of enabling capabilities. To discuss how each team
addressed the act's provisions, we met with each short-term objective
director. We also met with officials from the Air Force Research
Laboratory's headquarters and the Office of the Deputy Assistant Secretary
for Science, Technology, and Engineering. Finally, we physically observed
the proceedings of one short-term objective workshop. To evaluate whether
each provision was addressed, we prepared summary checklists for each
short-term objective and keyed the data back to a specific provision of the
act.

To document the extent to which the Air Force complied with the program and
budgetary resource assessment process specified in the act, we obtained the
final weekly activity reports, internal correspondence, review schedule, and
overview briefing. To evaluate whether each provision was addressed, we
prepared a summary checklist and obtained a written summary of the Air
Force's actions to comply with the provisions. Finally,

Appendix I
Scope and Methodology

we discussed the Air Force's actions with representatives from the Office of
the Deputy Assistant Secretary for Science, Technology, and Engineering.

GAO's Mission The General Accounting Office, the investigative arm of
Congress, exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability of
the federal government for the American people. GAO examines the use of
public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's commitment to good
government is reflected in its core values of accountability, integrity, and
reliability.

Obtaining Copies of GAO Reports and Testimony

The fastest and easiest way to obtain copies of GAO documents is through the
Internet. GAO's Web site (www.gao.gov) contains abstracts and full-text
files of current reports and testimony and an expanding archive of older
products. The Web site features a search engine to help you locate documents
using key words and phrases. You can print these documents in their
entirety, including charts and other graphics.

Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as "Today's Reports," on its Web
site daily. The list contains links to the full-text document files. To have
GAO E-mail this list to you every afternoon, go to www.gao.gov and select
"Subscribe to daily e-mail alert for newly released products" under the GAO
Reports heading.

Order by Mail or Phone The first copy of each printed report is free.
Additional copies are $2 each. A check or money order should be made out to
the Superintendent of Documents. GAO also accepts VISA and Mastercard.
Orders for 100 or more copies mailed to a single address are discounted 25
percent. Orders should be sent to:

U.S. General Accounting Office 441 G Street NW, Room LM Washington, D.C.
20548

To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061

Visit GAO's Document GAO Building

Room 1100, 700 4th  Street, NW (corner of 4th and G Streets, NW)Distribution
Center Washington, D.C. 20013

To Report Fraud, Contact:

Web site: www.gao.gov/fraudnet/fraudnet.htm,Waste, and Abuse in E-mail:
[email protected], orFederal Programs 1-800-424-5454 or (202) 512-7470
(automated answering system).

Public Affairs Jeff Nelligan, Managing Director, [email protected] (202)
512-4800 U.S. General Accounting Office, 441 G. Street NW, Room 7149,
Washington, D.C. 20548

                             Presorted Standard
                             Postage & Fees Paid
                                     GAO
                               Permit No. GI00

United States
General Accounting Office
Washington, D.C. 20548-0001

Official Business
Penalty for Private Use $300

Address Correction Requested
*** End of document. ***