Defense Pilot Programs: DOD Needs to Improve Implementation	 
Process for Pilot Programs (28-JUL-03, GAO-03-861).		 
                                                                 
In fiscal years 1999, 2000, and 2003, the Congress authorized	 
pilot programs to help the Department of Defense (DOD)		 
laboratories and test centers explore innovative business	 
partnerships and human capital strategies. Congressional concerns
about DOD's implementation of the pilot programs have been	 
growing. The Congress mandated that GAO review pilot program	 
implementation. GAO (1) identified the pilot initiatives proposed
and their current status, (2) examined factors that affected	 
implementation, and (3) assessed implementation challenges the	 
2003 pilot program faces.					 
-------------------------Indexing Terms------------------------- 
REPORTNUM:   GAO-03-861 					        
    ACCNO:   A07723						        
  TITLE:     Defense Pilot Programs: DOD Needs to Improve	      
Implementation Process for Pilot Programs			 
     DATE:   07/28/2003 
  SUBJECT:   Laboratories					 
	     Military research and development			 
	     Program evaluation 				 
	     Research program management			 
	     Test facilities					 

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO Product.                                                 **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
******************************************************************
GAO-03-861

Report to the Committee on Armed Services, U. S. Senate

United States General Accounting Office

GAO

July 2003 DEFENSE PILOT PROGRAMS

DOD Needs to Improve Implementation Process for Pilot Programs

GAO- 03- 861

The 1999 and 2000 pilot programs have not worked as intended. Since their
inception, 178 initiatives have been proposed by the participating
laboratories and test centers but only 4* or 2 percent* were implemented
under the pilot programs, as shown below. Participants proposed
initiatives covering a variety of areas, including business- like
practices, partnerships, and human capital innovations.

The pilot programs were not effective because DOD lacked an effective
implementation process and proposed human capital initiatives were not
consistent with statutory provisions. First, DOD did not provide
standardized guidance on proposal requirements, coordinate proposals, or
clarify decision- making authority for proposal review and approval.
Furthermore, DOD did not designate a strong focal point to provide
assistance and advice to participants and advocate process improvements.
The lack of a strong focal point exacerbated other process gaps. Second,

DOD attorneys advised that the pilot programs did not provide authority to
make most of the proposed human capital changes.

Implementation of the new 2003 pilot program faces several challenges.
First, DOD has not addressed implementation problems. For example, clear
guidance is still lacking and decision- making authority is still unclear.
Second, the 2003 pilot program provides no change in authority concerning
human capital initiatives. Finally, laboratories and test centers may be
reluctant to participate. Many participants in the earlier pilots told us
they were discouraged by their experience and consequently unwilling to
repeat it. Status of Proposed Initiatives

In fiscal years 1999, 2000, and 2003, the Congress authorized pilot
programs to help the Department of Defense (DOD) laboratories and test
centers explore innovative business partnerships and human

capital strategies. Congressional concerns about DOD*s implementation of
the pilot programs have been growing. The Congress mandated that GAO
review pilot program implementation. GAO (1) identified the pilot
initiatives

proposed and their current status, (2) examined factors that affected
implementation, and (3) assessed implementation challenges the 2003 pilot
program faces.

GAO recommends that by March 31, 2004, the Secretary of Defense (1) inform
the Congress of DOD*s objectives regarding human capital and business
operations in the laboratories and test centers; (2) develop a process for
proposing, evaluating, and implementing human capital and business
operations initiatives, regardless whether by the pilot authority or by
some other vehicle; and (3) designate a strong focal point to coordinate
and facilitate this process. DOD did not concur with GAO*s
recommendations.

www. gao. gov/ cgi- bin/ getrpt? GAO- 03- 861. To view the full product,
including the scope and methodology, click on the link above. For more
information, contact Catherine Baltzell at (202) 512- 8001 or baltzelld@
gao. gov. Highlights of GAO- 03- 861, a report to

the Committee on Armed Services, U. S. Senate

July 2003

DEFENSE PILOT PROGRAMS

DOD Needs to Improve Implementation Process for Pilot Programs

Page i GAO- 03- 861 Defense Pilot Programs Letter 1 Results in Brief 1
Background 3 Many Initiatives Were Proposed but Few Were Implemented under

Pilot Programs 5 The Pilot Programs Were Not Effective for Two Primary
Reasons 8 The 2003 Pilot Program Faces Implementation Challenges 12
Conclusions 12 Recommendations for Executive Action 13 Agency Comments and
Our Evaluation 13 Scope and Methodology 15 Appendix I Fiscal Year 1999 and
2000 Pilot Program Participants 17

Appendix II Comments from the Department of Defense 18

Table

Table 1: Laboratory and Test Center Pilot Program Proposals 5 Figure

Figure 1: Status of Proposed Initiatives 7 Contents

Page ii GAO- 03- 861 Defense Pilot Programs Abbreviations

DOD Department of Defense DDR& E Directorate of Defense Research and
Engineering NSPS National Security Personnel System OSD Office of the
Secretary of Defense P& R Personnel and Readiness

This is a work of the U. S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.

Page 1 GAO- 03- 861 Defense Pilot Programs July 28, 2003 The Honorable
John Warner Chairman

The Honorable Carl Levin Ranking Minority Member Committee on Armed
Services United States Senate

Over the last decade a number of studies have raised concerns that
dwindling budgets and an aging workforce have contributed to serious
shortfalls in the infrastructure and capabilities of Department of Defense
(DOD) laboratories and test centers. In fiscal years 1999 and 2000, the
Congress enacted legislation aimed at helping DOD laboratories and test
centers address problems by undertaking pilot programs to explore
innovative partnerships and human capital strategies. 1 In fiscal year
2003, the Congress extended the 1999 and 2000 pilot programs until 2005
and

enacted a new pilot program that runs until 2006. However, congressional
concerns about pilot program implementation have been growing.
Consequently, the Senate Committee on Armed Services directed us to review
the implementation of the 1999 and 2000 pilot programs. 2 In response,
this report (1) identifies initiatives proposed to date and determines
their current status, (2) examines factors that affected implementation of
proposed initiatives, and (3) assesses implementation challenges the new
2003 pilot program faces.

The 1999 and 2000 pilot programs have not worked as intended. Since their
inception, 178 initiatives have been proposed by the participating
laboratories and test centers but only 4* or 2 percent* have been
implemented using the pilot program authorities. Twelve times as many* 24
percent* were implemented using other authorities than those provided by
the pilot programs. Participating laboratories and test centers proposed
initiatives covering a variety of areas, including business- like

1 Public Law 105- 261, Strom Thurmond National Defense Authorization Act
for Fiscal Year 1999, section 246. Public Law 106- 65, National Defense
Authorization Act for Fiscal Year 2000, section 245.

2 Senate Report 107- 151.

United States General Accounting Office Washington, DC 20548

Results in Brief

Page 2 GAO- 03- 861 Defense Pilot Programs practices, partnerships with
industry and academia, and human capital innovations. In general,
laboratories tended to propose initiatives dealing with human capital
innovations and test centers focused on business- like

practices and partnerships. The pilot programs were not effective because
DOD lacked an effective implementation process and proposed human capital
initiatives were not consistent with statutory provisions. First, DOD did
not provide standardized guidance on proposal requirements, coordinate
proposals, or clarify decision- making authority for proposal review and
approval. Furthermore, DOD did not designate a strong focal point to
provide assistance and advice to participants and advocate process
improvements. The lack of a strong focal point exacerbated other process
gaps. Second, DOD attorneys advised participants that the 1999 and 2000
pilot

programs did not provide authority to make most of the proposed human
capital changes.

Implementation of the new 2003 pilot program faces several challenges.
First, DOD has not addressed implementation problems. For example, clear
guidance is still lacking and decision- making authority has not been
clarified. Second, the 2003 pilot program provides no change in authority
concerning human capital initiatives. DOD officials believe that the human
capital management legislation the department recently proposed to the
Congress will provide flexibility throughout DOD to make necessary human
capital changes, thereby eliminating the need for the pilot programs in
this area. 3 However, this legislation, if enacted, would still require an
implementation process. Finally, laboratories and test centers may be
reluctant to participate in the new pilot program. Many participants in
the earlier pilots told us they were discouraged by their experience and
consequently unwilling to repeat it.

We are making recommendations aimed at clarifying how DOD plans to address
concerns about the laboratories and test centers and improving the
implementation of initiatives proposed for that purpose. In written
comments on a draft of this report, DOD stated that it did not concur with
our recommendations.

3 Defense Transformation for the 21st Century Act of 2003, as transmitted
by letter, dated April 10, 2003, from the DOD General Counsel to the
Speaker of the House and the President of the Senate.

Page 3 GAO- 03- 861 Defense Pilot Programs The United States has a long
history of military research and development. To help conduct and manage
this research, DOD has a diverse network

of 80 in- house laboratories and 26 test centers. Their missions range
from basic scientific research to direct technical support to operational
commands. The management, operations, and funding for these disparate
laboratories and test centers also vary among the services.

Over the past decade, several organizations, panels, and commissions have
identified significant personnel and resource problems facing the
laboratories and test centers. For example, several studies found that the
laboratories needed more flexibility in personnel rules governing the
scientific workforce in order to attract and retain staff. 4 Similarly,
several recent studies identified problems with declines in investment and

infrastructure, resulting in outdated facilities and technical equipment.
5 To help the laboratories and test centers with these problems, the
Congress enacted legislation in fiscal years 1999 and 2000 establishing
pilot programs for laboratories and test centers to propose innovative
partnerships, business- like practices, and human capital initiatives. 6
The 1999 pilot program focused on partnerships and business- like
practices, while the 2000 program focused more on human capital
initiatives. Together, the two pilot programs authorized the Secretary of
Defense to provide one laboratory and one test center in each service the
authority to  explore innovative methods for partnering with universities
and private

sector entities to conduct defense research and development;  attract a
workforce balance between permanent and temporary personnel and with an
appropriate skill and experience level;  develop or expand innovative
methods of operation that provide more

defense research for the dollar; and  waive any restrictions on these
methods that are not required by law.

4 Blue Ribbon Panel on Management Options for Air Force Laboratories,
January 1994;

Improving Federal Laboratories to Meet the Challenges of the 21st Century,
National Science and Technology Council, July 1999; Science and Technology
Community in Crisis, Naval Research Advisory Committee, May 2002.

5 Defense Science and Technology Base for the 21st Century, Defense
Science Board, June 1998; House Report 105- 532, House National Security
Committee, May 1998; Science and Technology Community in Crisis, Naval
Research Advisory Committee, May 2002.

6 Public Law 105- 261, Strom Thurmond National Defense Authorization Act
for Fiscal Year 1999, section 246; Public Law 106- 65, National Defense
Authorization Act for Fiscal Year 2000, section 245. Background

Page 4 GAO- 03- 861 Defense Pilot Programs A total of 10 laboratories and
test centers from all 3 services participated in the pilot programs. They
are listed in appendix I.

Both programs were authorized for 3 years. The 1999 pilot expired in March
2002; the 2000 pilot, in March 2003. For both programs, DOD was required
to submit preliminary and final reports to the Congress on program
activities. The preliminary report for the 1999 program was submitted in
July 1999. However, as of the date of this report, the three other reports
have not been submitted.

In fiscal year 2003, the Congress authorized another 3- year pilot program
and extended the 1999 and 2000 pilot programs until 2005. 7 Under the new
2003 pilot program, the Secretary of Defense is to provide one laboratory
and one test center in each service the authority to  use innovative
personnel management methods to ensure that the

participants can employ and retain an appropriately balanced workforce,
and effectively shape the workforce to fulfill the organization mission; 
develop or expand innovative methods of using cooperative agreements

with private sector and educational organizations to promote the
technological industrial base for critical defense technologies and
facilitate the training of a future scientific and technical workforce;
and  waive any restrictions not required by law.

As of May 2003, DOD had not identified any participants for the 2003 pilot
program.

The 2003 legislation also requires DOD to issue three reports, including a
January 2003 report on its experience with the 1999 and 2000 pilot
programs, barriers to implementation of these programs, and proposed
solutions to overcome these barriers. 8 According to DOD officials, this
report has been drafted, but as of May 2003, it had not been submitted to
the Congress.

7 Public Law 107- 314, Bob Stump National Defense Authorization Act for
Fiscal Year 2003, section 241. 8 The other two reports are a September
2003 report on all three pilot programs and a final report on the 2003
pilot at its conclusion.

Page 5 GAO- 03- 861 Defense Pilot Programs Since the inception of the
pilot programs in 1999, 178 initiatives have been proposed, but only 4* or
2 percent* have been implemented

under the pilot programs. Participating laboratories and test centers
proposed initiatives covering a variety of areas, including business- like
practices, partnerships with industry and academia, and human capital
innovations. We found that laboratories focused many of their proposals

on human capital innovations, while test centers tended to concentrate on
business- like practices and partnerships. Over the course of the 1999 and
2000 pilot programs, the laboratories and test centers proposed 178 human
capital, business, and partnership initiatives. As shown in table 1,
slightly over half of the initiatives dealt with human capital and the
remainder dealt with business- like practices and partnerships.

Table 1: Laboratory and Test Center Pilot Program Proposals Business/
partnerships Human capital Total

Number Percent Number Percent Number Percent

Laboratories 49 33 98 67 147 100

Test centers 27 87 4 13 31 100 Total 76 43 102 57 178 100

Source: GAO. Overall, the laboratories proposed substantially more
initiatives than did the test centers. Furthermore, the laboratories and
test centers focused on different types of initiatives. The laboratories
more often proposed human capital initiatives, while the test centers
overwhelmingly focused on business and partnership initiatives. Laboratory
officials told us that they are especially concerned about attracting top-
quality scientists to replace a retiring workforce. Test center officials
told us that they are focused on modernizing their infrastructure and
developing new methods of sharing the cost of operations.

Proposals for business- like practices included many initiatives to
streamline or improve local operations. Some initiatives focused on
expanding the use of innovative techniques such as other transactions Many
Initiatives Were

Proposed but Few Were Implemented under Pilot Programs

Range and Volume of Proposed Initiatives

Page 6 GAO- 03- 861 Defense Pilot Programs or cooperative agreements. 9
Several other proposals sought the authority to reinvest fees or revenues
into facilities revitalization. For example, one

Navy laboratory proposed imposing a surcharge for its services and using
that revenue to fund capital investments, and an Air Force laboratory
proposed using facility construction as a valid in- kind contribution
under cooperative agreements.

Partnership proposals included initiatives such as collaborative research
agreements with Arnold Engineering Development Center and the University
of Tennessee Space Institute to create a formal business bond to pursue
research in laser- induced surface improvement technology and university
flight research.

The Army*s Aberdeen Test Center proposed a limited liability company.
Under this concept, industry, academia, and government would form a
profit- making company to conduct research and testing at the
installation. The test center proposed using its share of the profits to
reinvest in the infrastructure at Aberdeen.

Several human capital initiatives focused on recruiting and retention
flexibilities as well as additional voluntary separation incentives. These
proposals included initiatives to streamline hiring of experts and
consultants; accelerate promotions for scientists and engineers; provide
retention bonuses for key scientists; and hire students directly after
graduation. Several participants submitted proposals for direct hire
authority to allow faster hiring of scientists, and several submitted
proposals for voluntary retirement incentives as a mechanism for reshaping
the workforce.

Almost none of the 178 proposed initiatives were approved and implemented
using the pilot programs* authorities. As figure 1 shows, only 2 percent*
or 4 proposals* were implemented under the pilot programs. In contrast, 74
percent were blocked or dropped during the review process or remain on
hold awaiting resolution.

9 *Other transactions* is a term commonly used to refer to 10 U. S. C.
2371 authority to enter into agreements that are not generally covered by
federal laws and regulations applicable to standard procurement contracts.
Consequently, the arrangements include broader latitude to negotiate terms
and conditions than standard procurement contracts under the

Federal Acquisition Regulations. Few Proposals Were Implemented under
Pilot

Program Authorities

Page 7 GAO- 03- 861 Defense Pilot Programs Figure 1: Status of Proposed
Initiatives

The four implemented initiatives were  donating laboratory equipment
directly to local schools,  waiving top- level certification of certain
service agreements with

private industry,  streamlining cooperative agreements to facilitate
collaborative work

agreements with outside activities, 10 and  granting temporary relief
from some mandatory personnel placement

reviews. 11 Officials at the laboratories that proposed these initiatives
told us that they were considered minor changes with little impact on the
larger problems facing the laboratories.

10 This initiative included several closely related but separate waivers
that were grouped together by the service of the laboratories that
proposed them. 11 This waiver expired in August 2002. It was extended
until February 2003 as part of a separate pilot program sponsored by the
DOD Business Initiative Council.

Page 8 GAO- 03- 861 Defense Pilot Programs Twelve times as many
initiatives* 24 percent* were implemented using different authorities than
the pilot programs. For example, several laboratories requested the
authority to appoint retired military members

to civilian positions without having to wait the required 180 days. This
requirement was waived using a different authority than the pilot
programs. Another human capital initiative* to appoint senior scientists
from private industry* was authorized by subsequent legislation. 12 In the
business/ partnership category, the 46th Test Group at Holloman Air Force
Base used other authorities to negotiate a complex leasing arrangement
with industry to install a radar test facility at White Sands Missile
Range. This effort took several years and overcame many contractual and
regulatory barriers. In addition, a Navy laboratory streamlined foreign
license applications using another authority. 13 The low level of
implementation of the proposed initiatives occurred for

two primary reasons. First, DOD did not develop an effective process for
implementing the pilot programs. Second, DOD determined that proposed
human capital initiatives* for example, requests for the authority to hire
directly or offer voluntary retirement incentives* were in conflict with
statutory provisions.

DOD did not provide standardized guidance on proposal requirements or
feedback for improving proposals; coordinate or prioritize proposals; or
clarify decision- making authority for proposal review and approval. DOD
also did not designate a strong focal point to coordinate the pilot
programs, advocate process improvements, and provide assistance and advice
to participants. The lack of a strong focal point exacerbated other
process gaps.

According to officials at DOD laboratories, test centers, and
headquarters, DOD did not provide standardized guidance on proposal
requirements or feedback for improving proposals (or, in many cases,
information on the status of proposals submitted for approval). Proposals
often lacked

12 Public Law 106- 398, National Defense Authorization Act for Fiscal Year
2001, section 1113. 13 The Office of the Secretary of Defense (OSD) and
the Navy disagree on what authority was used to implement this proposal.
OSD believes that it was implemented using pilot program authority. The
Pilot Programs

Were Not Effective for Two Primary Reasons

Lack of an Effective Implementation Process

Lack of Guidance and Coordination

Page 9 GAO- 03- 861 Defense Pilot Programs specificity and detail. Many
were broadly conceptual or generic in nature and lacked a detailed
business case that linked their contribution to overall objectives for the
pilot programs. For example, a proposal to

permit scientists to serve in a leadership role in professional societies
failed to include details of the problems encountered, and the potential
to improve operations. Similarly, several proposals for direct hire
authority

failed to include a business case to explain what specific needs this
authority would address or how it would address them. Lack of specificity
and business case detail led to the failure of many initiatives to win
approval. DOD attorneys told us that many proposals were so vague that it
was impossible to determine whether or not the proposed initiatives could
meet legal requirements.

At a department level, DOD also did not coordinate or prioritize
proposals, thereby precluding decisions on how best to pursue common
interests and issues such as direct hiring authority or forming
partnerships with universities. Instead, each participant submitted
proposals individually, and thus multiple independent proposals were often
submitted for the same or similar issues. DOD attorneys pointed out that
it would have been more effective to group proposals by common theme and
prioritize them. They believed a unified approach and prioritized
proposals with clearly written, specific plans for solving well- defined
problems would have enabled them to more effectively assist participants
with resolving legal issues. DOD did not clarify decision- making
authority for proposal review and

approval. Many organizations and individuals were stakeholders in proposal
review and approval, and they often had differing management structures,
concerns, and interests. Stakeholders included military and civilian
leaders, attorneys, and human capital and personnel staff at several
levels: the local installation where participating laboratories and test
centers were housed; the individual service; and OSD. The roles and
decision- making authority of the various stakeholders were never
negotiated and clarified. As a result, many players at multiple
organizational levels had* and took* an opportunity to say *no* to a
particular proposal, but it remained unclear who had the authority to say
*yes.*

For example, some participants believed that the pilot program legislation
gave the director of a participating laboratory or test center the
authority to approve a proposed initiative. OSD officials, however,
believed that the proposed initiatives had to be approved at higher
levels. The role of the services was also unclear. Some laboratory and
test center directors Unclear Decision- Making

Authority

Page 10 GAO- 03- 861 Defense Pilot Programs initially sent proposals
directly to OSD*s Directorate of Defense Research and Engineering (DDR&
E), bypassing their service headquarters. Others

sent proposals to their service headquarters for approval before
submitting the proposals to DDR& E. Eventually, however, each of the
service headquarters decided to become more heavily involved in the
approval process and provide service- level responses to proposals. These
service- level responses often came into play after proposals had been
sent directly to DDR& E for approval, further complicating the approval
process.

Within OSD, both DDR& E and Personnel and Readiness (P& R) had substantial
stakes in the human capital proposals* DDR& E because it is charged with
oversight and management of defense laboratories and P& R because it has
the authority within DOD for human capital issues. 14 However, DDR& E and
P& R never agreed on a process for approving

proposals. In addition, for the past year P& R*s attention has been
focused primarily on developing DOD*s proposed new civilian human capital
management system, the National Security Personnel System (NSPS),

which the Secretary of Defense recently submitted to the Congress. DOD
officials believe that, if enacted, NSPS will provide flexibility to make
necessary human capital changes. 15 The Undersecretary of Defense P& R
directed that implementation of new personnel initiatives be placed on
hold during the development of NSPS so that the existing system could be
studied to identify needs and best practices. Consequently, P& R officials

believed it would be premature for DOD to implement new personnel
initiatives during this time.

DOD did not designate a strong focal point to coordinate the pilot
programs, advocate process improvements, and provide assistance and advice
to participants. This exacerbated the other process gaps. Without such a
focal point, participants found their own individual ways to develop
proposals and get them reviewed. Several officials agreed that a strong
focal point would be helpful. For example, DOD attorneys stated that the
laboratories or someone acting as their focal point needed to define the

14 Unlike the laboratories, the test centers are not overseen centrally
but by the individual service to which they belong. 15 The Comptroller
General recently testified on NSPS. U. S. General Accounting Office.

Human Capital: DOD*s Civilian Personnel Strategic Management and the
Proposed National Security Personnel System. GAO- 03- 493T (Washington, D.
C.; May 12, 2003).

Defense Transformation: DOD*s Proposed Civilian Personnel System and
Governmentwide Human Capital Reform. GAO- 03- 741T (Washington, D. C.; May
1, 2003). No Strong Focal Point

Page 11 GAO- 03- 861 Defense Pilot Programs issues they wanted to resolve.
The attorneys noted that a focal point could have more successfully drawn
upon their expertise and experience

with addressing legal challenges in other innovative programs (e. g.,
demonstration projects). Some pilot program participants also agreed a
strong focal point was needed, but they had some concerns regarding the
amount of influence and authority he or she should have.

According to officials at DOD laboratories, test centers, and
headquarters, human capital initiatives were generally in conflict with
title 5 of the United States Code. Title 5 provides the framework for
standard and equitable personnel practices across the federal government
and is the current foundation for management of the DOD civilian
workforce. Over time, the Office of Personnel Management has added
implementing rules

and regulations to the framework. Proposed human capital initiatives often
sought relief from these provisions, for example, requests for the
authority to hire directly or offer voluntary retirement incentives.

However, after reviewing the legislation, the DOD Office of General
Counsel advised that the 1999 and 2000 legislation did not provide the
authority to waive personnel rules based on title 5 provisions. Rather,
the office advised that the pilot programs* authorities allow only for
changes that could already be accomplished under existing DOD regulations.
In other words, the pilot programs did not provide any new or additional
authority to waive existing personnel rules and regulations grounded in
title 5. Consequently, absent statutory authority beyond that provided by
the pilot programs, human capital proposals in conflict with title 5 and
its implementing rules and regulations could not be implemented. 16 Many
initiatives fell into this category.

16 Our attorneys reviewed the pilot program legislation and concurred with
the DOD General Counsel*s view. Human Capital Proposals

Were in Conflict with Existing Statutory Provisions

Page 12 GAO- 03- 861 Defense Pilot Programs The 2003 pilot program faces
several implementation challenges. First, as of May 2003, DOD had not
addressed implementation problems. Thus,

proposals made via the 2003 pilot program will face the same obstacles as
previous proposals.

Second, human capital initiatives will continue to face title 5
challenges. Like the earlier legislation, the 2003 legislation does not
provide DOD any new authority. Hence, initiatives proposed under the 2003
pilot program will encounter the same statutory restrictions as previous
initiatives. P& R officials believe that, if implemented, NSPS will
provide the flexibility to make necessary human capital changes, thereby
eliminating the need for the pilot programs in this area. However, NSPS
has not yet been enacted, and if enacted, it will still require an
implementation process.

Finally, laboratories and test centers may be reluctant to participate in
the new pilot program. Many participants in the earlier pilots told us
they were discouraged by their experience and consequently unwilling to
repeat it. Some expressed frustration with the lack of guidance and
feedback on their proposals; others questioned whether management was
really committed to the pilot program. Even those few participants that
had proposals approved were wary of expending additional resources on
another pilot program.

While DOD appears to recognize a need to address human capital and
business operations issues specific to laboratories and test centers, it
has not effectively managed the pilot programs. If DOD intends to use the

pilot programs to address laboratory and test center issues, it will have
to address the factors* both process and statutory* that blunted previous
proposals made through the pilot programs. The small volume of approved
proposals, coupled with DOD*s not providing status reports required by the
Congress, has left the Congress uninformed about what objectives DOD would
like to achieve with the laboratories and test centers, how it plans to
achieve those objectives, and what vehicles it plans to use. This
information will be important to the success of any future actions. The
2003 Pilot

Program Faces Implementation Challenges

Conclusions

Page 13 GAO- 03- 861 Defense Pilot Programs We recommend that by March 31,
2004, the Secretary of Defense inform the Congress of DOD*s objectives
regarding human capital and business operations in the laboratories and
test centers, how it plans to meet these

objectives, and what vehicles it will use to meet them. We also recommend
that by March 31, 2004, the Secretary of Defense develop a process for
proposing, evaluating, and implementing human capital, business, and
partnership initiatives for the laboratories and test centers, regardless
whether by the pilot authority or by some other vehicle. Such a process
should include

 clear decision- making authority,  instructions for proposal
requirements such as linking to overall goals and

measurable objectives and the need for a business case, and 
specification of procedures for proposal submission and review and
providing feedback on proposal quality and scope. Finally, we recommend
that the Secretary of Defense designate a strong focal point to  receive,
evaluate, and prioritize all proposals and

 work with laboratory and test center directors, legal counsel, personnel
and other specialists to develop sound and well- developed business cases
and strategies to obtain needed changes.

In written comments on a draft of this report, DOD states that it does not
concur with our recommendations because it has already taken actions that
in effect implement them. 17 While the actions DOD cites that it has taken
are important to implementing our recommendations, they are not
sufficiently specific to address the problems identified in our report.

DOD*s written comments are contained in appendix II. Regarding our first
recommendation* that DOD inform the Congress of its human capital and
business objectives for the laboratories and test centers and the
strategies it will employ to meet them* DOD did not concur. DOD discusses
various high- level, agencywide initiatives it has taken to address human
capital and business issues in general and stated that the Congress

17 In its letter, DOD refers to the *1999 and 2000 demonstration
programs.* We confirmed with DDR& E that these demonstration programs were
indeed the 1999 and 2000 pilot programs as described in this report.
Recommendations for Executive Action

Agency Comments and Our Evaluation

Page 14 GAO- 03- 861 Defense Pilot Programs has been made aware of these
initiatives, obviating the need for additional reporting. We continue to
believe that additional reporting is necessary.

We recognize that the general initiatives DOD discusses may provide ways
of helping the laboratories and test centers; however, to be effective,
they must be made specific, that is, developed into targeted strategies
and plans

that address the particular problems the laboratories and test centers
face. DOD has not provided the Congress sufficient details on how the
general initiatives will be used to address laboratories* and test
centers* objectives and problems.

Regarding our second recommendation* that DOD develop a process for
proposing, evaluating, and implementing human capital and business- like
practices initiatives for the laboratories and test centers* DOD did not
concur. DOD states that it has already introduced new agencywide
management processes* the Business Initiative Council and the submission
of the NSPS proposal to the Congress* to address human capital and
business issues in general. However, DOD has not detailed how these
general initiatives will apply to the laboratories and test centers or
address our process concerns. For example, while the Business

Initiative Council may have an effective process for proposing,
evaluating, and implementing laboratory and test center business- like
practices initiatives, DOD has not provided sufficient information for us
to make such a determination. We also recognize that NSPS may address some
of the human capital problems faced by the laboratories and test centers,
but this system is still under consideration by the Congress. Until it
becomes law, we believe it is premature to cite it as an effective
management tool. With regard to our third recommendation* that DOD
designate a strong

focal point to work with the laboratories and test centers to develop,
evaluate, prioritize, and coordinate proposed initiatives* DOD did not
concur. DOD states that the recently created position of Undersecretary
for Laboratories and Basic Sciences has oversight responsibility for all
laboratory initiatives and that it is establishing a new Defense Test
Resources Management Center that will oversee the test centers. 18 DOD
asserts that these two organizations will perform as focal points.
However, DOD has not detailed how these organizations will fulfill this
role and work with the laboratories and test centers to overcome the many
barriers noted in our report.

18 This new center is not yet operational.

Page 15 GAO- 03- 861 Defense Pilot Programs During our review, we met with
officials from the following organizations in the Office of the Secretary
of Defense: the Director, Defense Research

and Engineering; the Director, Operational Test and Evaluation; the
General Counsel, and the Deputy Undersecretary of Defense for Personnel
and Readiness. We also met with officials from the Army Research
Laboratory, Aberdeen Test Center, Army Medical Research and Materiel
Command, Naval Research Laboratory, Naval Undersea Warfare Center,

Air Force Research Laboratory, Air Force Research Laboratory*s Space
Vehicles Directorate, and 46th Test Wing. We also discussed pilot program
issues with each participating laboratory or center.

To determine the initiatives proposed to date and their status, we
obtained records from OSD and service officials. From these records and
from discussions with each participant, we compiled a listing of
initiatives

proposed by each participating laboratory and test center. We verified the
listing and the current status of each initiative with cognizant service
officials.

To determine what obstacles inhibited DOD*s implementation of the pilot
programs, we obtained documentation and data from pilot program
participants as well as from OSD officials. We also discussed statutory
obstacles with the officials from DOD*s Office of General Counsel and
Undersecretary of Defense for Personnel and Readiness. We discussed
management and procedural obstacles with officials from the Director,
Operational Test and Evaluation and Defense Research and Engineering. In
addition, we discussed all obstacles with the participating laboratories

and test centers. The problems facing the laboratories and test centers
have been documented by many organizations, panels, and commissions. We
did not independently verify these problems or the findings and
conclusions of these entities. We conducted our review from July 2002 to
April 2003 in accordance with generally accepted government auditing
standards.

We are sending copies of this report to the Secretary of Defense; the
Secretaries of the Army, Navy, and Air Force; and interested congressional
committees. We will also make copies available to others upon request. In
addition, the report will be available at no charge on the GAO Web site at

http:// www. gao. gov. Scope and

Methodology

Page 16 GAO- 03- 861 Defense Pilot Programs Major contributors to this
report were Catherine Baltzell, Arthur Cobb, Christopher Durbin, Rae Ann
Sapp, Sylvia Schatz, and Katrina Taylor.

If you have any questions regarding this report, please call me at (202)
512- 4841.

Paul L. Francis Director Acquisition and Sourcing Management

Appendix I: Fiscal Year 1999 and 2000 Pilot Program Participants

Page 17 GAO- 03- 861 Defense Pilot Programs Laboratory/ test center 1999
pilot 2000 pilot

Army Research Laboratory X Army Medical Research and Materiel Command X
Aberdeen Test Center X X

Naval Research Laboratory X Naval Undersea Warfare Center X Naval Air
Warfare Center* Aircraft and Weapons Divisions X X Air Force Research
Laboratory* Information Directorate X Air Force Research Laboratory* Space
Vehicles Directorate X Air Armament Center, 46th Test Wing X

Arnold Engineering Development Center X Source: DOD.

Appendix I: Fiscal Year 1999 and 2000 Pilot Program Participants

Appendix II: Comments from the Department of Defense

Page 18 GAO- 03- 861 Defense Pilot Programs Appendix II: Comments from the
Department of Defense

Appendix II: Comments from the Department of Defense Page 19 GAO- 03- 861
Defense Pilot Programs

Appendix II: Comments from the Department of Defense Page 20 GAO- 03- 861
Defense Pilot Programs

Appendix II: Comments from the Department of Defense

Page 21 GAO- 03- 861 Defense Pilot Programs (120159)

The General Accounting Office, the audit, evaluation and investigative arm
of Congress, exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability of
the federal government for the American people. GAO examines the use of
public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO*s commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents at no cost
is through the Internet. GAO*s Web site (www. gao. gov) contains abstracts
and fulltext files of current reports and testimony and an expanding
archive of older products. The Web site features a search engine to help
you locate documents using key words and phrases. You can print these
documents in their entirety, including charts and other graphics.

Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as *Today*s Reports,* on its
Web site daily. The list contains links to the full- text document files.
To have GAO e- mail

this list to you every afternoon, go to www. gao. gov and select
*Subscribe to e- mail alerts* under the *Order GAO Products* heading.

The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent of
Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more
copies mailed to a single address are discounted 25 percent. Orders should
be sent to: U. S. General Accounting Office 441 G Street NW, Room LM
Washington, D. C. 20548 To order by Phone: Voice: (202) 512- 6000

TDD: (202) 512- 2537 Fax: (202) 512- 6061

Contact: Web site: www. gao. gov/ fraudnet/ fraudnet. htm E- mail:
fraudnet@ gao. gov Automated answering system: (800) 424- 5454 or (202)
512- 7470 Jeff Nelligan, Managing Director, NelliganJ@ gao. gov (202) 512-
4800

U. S. General Accounting Office, 441 G Street NW, Room 7149 Washington, D.
C. 20548 GAO*s Mission Obtaining Copies of

GAO Reports and Testimony

Order by Mail or Phone To Report Fraud, Waste, and Abuse in Federal
Programs Public Affairs
*** End of document. ***