DOD Business Systems Modernization: Limited Progress in 	 
Development of Business Enterprise Architecture and Oversight of 
Information Technology Investments (17-MAY-04, GAO-04-731R).	 
                                                                 
The Department of Defense's (DOD) long-standing business systems 
problems adversely affect the economy, effectiveness, and	 
efficiency of its business operations and have resulted in a lack
of adequate transparency and appropriate accountability across	 
all of its major business areas. In July 2001, DOD initiated a	 
program to, among other things, develop a DOD business enterprise
architecture (architecture). This effort is an essential part of 
the Secretary of Defense's broad initiative to "transform the way
the department works and what it works on." Because DOD is one of
the largest and most complex organizations in the world,	 
overhauling its business operations and supporting systems	 
represents a huge management challenge. In fiscal year 2003, DOD 
reported that its operations involved over $1 trillion in assets,
nearly $1.6 trillion in liabilities, approximately 3.3 million	 
military and civilian personnel, and disbursements of over $416  
billion. To support its business operations, DOD reported that it
relies on about 2,300 business systems, including accounting,	 
acquisition, logistics, and personnel systems. The department	 
requested about $19 billion--about $4.8 billion for business	 
systems modernization and about $14 billion for operation and	 
maintenance of these systems--in fiscal year 2004. Recognizing	 
the importance of DOD's efforts to transform its business	 
operations and systems through the use of an enterprise 	 
architecture, the Congress included provisions in the National	 
Defense Authorization Act for Fiscal Year 2003 that were aimed at
developing and effectively implementing a well-defined		 
architecture. Specifically, section 1004 of this act required	 
that DOD (1) develop, by May 1, 2003, a financial management	 
enterprise architecture and a transition plan for implementing	 
the architecture that meets certain requirements and (2) review  
financial system improvements with proposed obligations of funds 
in amounts exceeding $1 million to determine if those system	 
improvements meet specific conditions that are called for in the 
act. The act also directed us to assess actions that DOD has	 
taken to comply with these requirements. In July and September	 
2003, we reported on DOD's actions and made a number of 	 
recommendations to assist DOD in its efforts to effectively	 
develop and implement an architecture and to guide and constrain 
its business systems investments. The act further requires that  
the Secretary of Defense submit an annual report not later than  
March 15 of each year from 2004 through 2007 to congressional	 
defense committees on its progress in implementing the		 
architecture, including the transition plan. Additionally, the	 
act directs us to submit to congressional defense committees,	 
within 60 days of DOD's report submission, an assessment of DOD's
actions taken to comply with these requirements. DOD submitted	 
its first annual report on March 15, 2004. This report is our	 
assessment of DOD's March 15, 2004 report. We determined (1) the 
actions DOD has taken to address our previous recommendations	 
regarding the development and implementation of the architecture 
and (2) the actions DOD is taking to ensure its ongoing and	 
planned investments will be consistent with its evolving	 
architecture.							 
-------------------------Indexing Terms------------------------- 
REPORTNUM:   GAO-04-731R					        
    ACCNO:   A10112						        
  TITLE:     DOD Business Systems Modernization: Limited Progress in  
Development of Business Enterprise Architecture and Oversight of 
Information Technology Investments				 
     DATE:   05/17/2004 
  SUBJECT:   Defense cost control				 
	     Federal agency accounting systems			 
	     Financial management systems			 
	     Information resources management			 
	     Information technology				 
	     Management information systems			 
	     Strategic information systems planning		 
	     Systems conversions				 
	     Systems design					 
	     Systems management 				 
	     Best practices					 
	     Investment planning				 
	     Performance measures				 
	     Accountability					 
	     Enterprise architecture				 
	     DOD Business Management Modernization		 
	     Program						 
                                                                 

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO Product.                                                 **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
******************************************************************
GAO-04-731R

United States General Accounting Office Washington, DC 20548

May 17, 2004

Congressional Defense Committees

Subject: DOD Business Systems Modernization: Limited Progress in
Development of Business Enterprise Architecture and Oversight of
Information Technology Investments

The Department of Defense's (DOD) long-standing business systems problems
adversely affect the economy, effectiveness, and efficiency of its
business operations and have resulted in a lack of adequate transparency
and appropriate accountability across all of its major business areas. To
help the department transform its operations, we recommended1 in 2001 that
DOD develop an enterprise architecture to guide and constrain its almost
$20 billion annual investment in business systems and that it establish
the investment controls needed to implement this architecture. In July
2001, DOD initiated a program2 to, among other things, develop a DOD
business enterprise architecture (architecture). This effort is an
essential part of the Secretary of Defense's broad initiative to
"transform the way the department works and what it works on."

Because DOD is one of the largest and most complex organizations in the
world, overhauling its business operations and supporting systems
represents a huge management challenge. In fiscal year 2003, DOD reported
that its operations involved over $1 trillion in assets, nearly $1.6
trillion in liabilities, approximately 3.3 million military and civilian
personnel, and disbursements of over $416 billion. To support its business
operations, DOD reported that it relies on about 2,300 business systems,
including accounting, acquisition, logistics, and personnel systems. The

3

department requested about $19 billion-about $4.8 billion for business
systems

1U.S. General Accounting Office, Information Technology: Architecture
Needed to Guide Modernization of DOD's Financial Operations, GAO-01-525
(Washington, D.C.: May 17, 2001).

2The Business Management Modernization Program is the department's
business transformation initiative; it encompasses defense policies,
processes, people, and systems that guide, perform, or support all aspects
of business management-including development and implementation of the
business enterprise architecture. The Under Secretary of Defense
(Comptroller) established a DODwide program management office called
Business Modernization and Systems Integration (BMSI), to oversee and
manage the program.

3Business systems include financial and nonfinancial systems, such as
civilian personnel, finance, health, logistics, military personnel,
procurement, and transportation, with the common element being the
generation or use of financial data to support DOD's business operations.

modernization and about $14 billion for operation and maintenance of these
systems-in fiscal year 2004.

Recognizing the importance of DOD's efforts to transform its business
operations and systems through the use of an enterprise architecture, the
Congress included provisions in the National Defense Authorization Act for
Fiscal Year 20034 that were aimed at developing and effectively
implementing a well-defined architecture. Specifically, section 1004 of
this act required that DOD (1) develop, by May 1, 2003, a financial
management enterprise architecture5 and a transition plan for implementing
the architecture that meets certain requirements and (2) review financial
system improvements with proposed obligations of funds in amounts
exceeding $1 million to determine if those system improvements meet
specific conditions that are called for in the act. The act also directed
us to assess actions that DOD has taken to comply with these requirements.
In July6 and September 2003,7 we reported on DOD's actions and made a
number of recommendations to assist DOD in its efforts to effectively
develop and implement an architecture and to guide and constrain its
business systems investments.

The act further requires that the Secretary of Defense submit an annual
report not later than March 15 of each year from 2004 through 2007 to
congressional defense committees on its progress in implementing the
architecture, including the transition plan. Additionally, the act directs
us to submit to congressional defense committees, within 60 days of DOD's
report submission, an assessment of DOD's actions taken to comply with
these requirements. (See enc. I for a copy of section 1004 of the act.)
DOD submitted its first annual report on March 15, 2004. This report is
our assessment of DOD's March 15, 2004 report. As agreed with your
offices, we determined (1) the actions DOD has taken to address our
previous recommendations regarding the development and implementation of
the architecture and (2) the actions DOD is taking to ensure its ongoing
and planned investments will be consistent with its evolving architecture.

4Bob Stump National Defense Authorization Act for Fiscal Year 2003, Pub.
L. No. 107-314, S: 1004, 116 Stat. 2458, 2629 (Dec. 2, 2002).

5In May 2003, the DOD Comptroller changed the architecture name from the
Financial Management Enterprise Architecture to the Business Enterprise
Architecture to reflect the transformation of departmentwide business
operations and supporting systems, including accounting and finance,
budget formulation, acquisition, inventory management, logistics,
personnel, and property management systems.

6U.S. General Accounting Office, DOD Business Systems Modernization:
Summary of GAO's Assessment of the Department of Defense's Initial
Business Enterprise Architecture, GAO-03-877R (Washington, D.C.: July 7,
2003).

7U.S. General Accounting Office, DOD Business Systems Modernization:
Important Progress Made to Develop Business Enterprise Architecture, but
Much Work Remains, GAO-03-1018 (Washington, D.C.: Sept. 19, 2003).

We performed our work from December 2003 through April 2004 in accordance
with U.S. generally accepted government auditing standards. Details on our
scope and methodology are in enclosure II.

Results in Brief

Since our last review-and after 3 years of effort and over $203 million in
obligations-we have not seen any significant change in the content of
DOD's architecture or in DOD's approach to investing billions of dollars
annually in existing and new systems. Few actions have been taken to
address the recommendations we made in our September 20038 report, which
were aimed at improving DOD's plans for developing the next version of the
architecture and implementing the institutional means for selecting and
controlling both planned and ongoing business systems investments. To
DOD's credit, it has established, for example, a group under the Business
Management Modernization Program (program) steering committee9 to
facilitate communication and coordination across the domains10 for
modernization program activities, including extending and evolving the
architecture. However, DOD has not yet adopted key architecture management
best practices11 and has not added the scope and detail to its
architecture that we previously identified as missing. Further, DOD has
not yet implemented an effective management structure and processes to
provide adequate control and accountability over its $5 billion annual
investment in business systems modernization. Additionally, the department
does not have reasonable assurance that it is in compliance with the
National Defense Authorization Act for Fiscal Year 2003, which requires
DOD's Comptroller to review all system improvements with obligations
exceeding $1 million.

Regarding architecture management best practices, DOD has not yet
implemented key elements, such as assigning accountability and
responsibility for directing, overseeing, and approving the architecture.
In addition, it has not explicitly defined performance metrics to evaluate
the architecture's quality, content, and utility. With

8GAO-03-1018.

9The steering committee, made up of senior leaders from across the
department, is advisory in nature and is not accountable for directing,
overseeing, and approving the architecture.

10DOD has six departmental domains, which are (1) accounting and finance,
(2) acquisition, (3) human resources management, (4) installations and
environment, (5) logistics, and (6) strategic planning and budgeting. It
also has one enterprise information environment mission area. The domains
and the mission area, comprised of the Under Secretaries of Defense and
the Assistant Secretary of Defense for Networks and Information
Integration/DOD Chief Information Officer, have authority, responsibility,
and accountability within their domains for business transformation,
implementation of the architecture, development and execution of the
transition plan, portfolio management, and establishment of a structure to
ensure representation of the DOD components and the appropriate federal
agencies.

11U.S. General Accounting Office, Information Technology: A Framework for
Assessing and Improving Enterprise Architecture Management (Version 1.1),
GAO-03-584G (Washington, D.C.: April 2003).

respect to the architectural content, DOD's latest version of the
architecture does not include many of the key elements of a well-defined
"As Is," "To Be," and transition plan that we previously reported as not
being satisfied.12 For example, the "To Be" environment does not provide
sufficient descriptive content related to future business operations and
supporting technology to permit effective acquisition and implementation
of system solutions and associated operational change. Similarly, DOD's
verification and validation contractor has concluded that this latest
version of the architecture retains most of the limitations that the
initial version had. Moreover, DOD has not yet defined specific plans,
including milestones, detailing how it intends to extend and evolve the
architecture to incorporate this missing content. For example, the
program's first objective for increment one-to enable asset
accountability, total force visibility, and an unqualified audit opinion
on DOD's fiscal year 2007 consolidated financial statements-is not
supported by a DOD-wide plan of action, and the individual component plans
for accomplishing this objective may not be linked to the program's
activities, including the architecture. According to DOD, it will not have
the plans on how the capabilities for increment one will be achieved until
August 2004.

DOD has also made limited progress in addressing our recommendations aimed
at establishing and implementing effective investment management processes
and, therefore, continues to lack effective management oversight and
control over ongoing business systems modernization investments. While DOD
has recently issued a policy that assigns investment management
responsibilities to the domains, the policy has not yet been implemented,
and DOD has not clearly defined the roles and responsibilities of the
domains, established common investment criteria, and conducted a
comprehensive review of its existing business systems to ensure that they
are consistent with the architecture. Further, each of the DOD components
continues to receive its own funding and make its own parochial investment
decisions. Moreover, DOD has not yet established and implemented an
effective process for ensuring that system improvements with obligations
exceeding $1 million are submitted to DOD's Comptroller for review and to
determine whether they are consistent with the architecture, as required
by the National Defense Authorization Act for Fiscal Year 2003. Based upon
a comparison of limited information provided by the military services and
defense agencies, we identified a total of $863 million in obligations for
improvements that exceeded $1 million each but had not been submitted to
DOD's Comptroller for review and determination.

The department acknowledges that it still has much more to do, including
developing the architecture to a necessary level of detail, defining
specific performance metrics, and clarifying the roles and
responsibilities associated with managing the domains' portfolios of
business systems and ensuring that these systems comply with the
architecture. The limited progress that DOD has made is due, in part, to
the lack of clearly assigned, accountable, and sustained program
leadership and to changes in the program direction and priorities. For
example, from May 2003 to February 2004, there was no program manager to
identify, direct, and execute program activities.

12GAO-03-1018.

Our experience in reviewing other challenged architecture efforts shows
that these efforts have suffered from limited senior management
understanding of and commitment to an architecture and from cultural
resistance to having and using one. Cultural resistance to change,
organization component parochialism, and stovepiped operations are all
evident at DOD.

Because many of our prior recommendations-for managing architecture
development, maintenance, and implementation and for controlling ongoing
and planned investments in business systems-remain open, we are not making
any new recommendations in this report, but we are reiterating the 22 open
recommendations that we made in our May 2001,13 February 2003,14 and
September 200315 reports. (Enc. III contains details on the status of all
of our prior recommendations, including our assessment of DOD's actions.)
It is imperative that DOD act swiftly to implement these recommendations.
If it does not, the prognosis for this program is bleak, which in turn
puts the department's business transformation efforts in jeopardy.

In written comments, which are reprinted in enclosure IV, DOD agreed with
our assessment that the department's long-standing business systems
problems have resulted in a lack of adequate transparency and appropriate
accountability across all major business areas and that development and
use of an architecture is necessary to transforming its business
operations and supporting systems. In response to our characterization
that progress has been limited, DOD stated that, over the past 3 years,
progress has been slower than either GAO or DOD would prefer but that it
has been significant, despite appearances to the contrary. In support of
its view, DOD provided an extensive package of information. We considered
this information, most of which we had previously evaluated. The
willingness of DOD's leadership to tackle the department's decades-old
problems with its business operations and supporting systems represents a
major step forward. However, we remain convinced that it is fair to
characterize DOD's progress in developing a well-defined architecture and
implementing effective management oversight and control over its business
systems as limited. As a result, we did not make any changes to this
report.

13GAO-01-525.

14U.S. General Accounting Office, DOD Business Systems Modernization:
Improvements to Enterprise Architecture Development and Implementation
Efforts Needed, GAO-03-458 (Washington, D.C.: Feb. 28, 2003).

15GAO-03-1018.

Background

Prior Reviews of DOD's Architecture Efforts Have Identified Challenges and

Weaknesses

Over the last 3 years, we have reported16 that one of the key elements to
successfully meeting DOD's financial and related business management
challenges is establishing and implementing an enterprise architecture, or
modernization blueprint. An enterprise architecture provides a clear and
comprehensive picture of an entity, whether it is an organization (e.g.,
federal department or agency) or a functional or mission area that cuts
across more than one organization (e.g., financial management). This
picture consists of snapshots of both the enterprise's current or "As Is"
operational and technological environment and its target or "To Be"
environment, as well as a capital investment road map for transitioning
from the current to the target environment. These snapshots further
consist of "views," which are basically one or more architecture products
that provide conceptual or logical representations of the enterprise. We
have made numerous recommendations to assist DOD in successfully
developing the architecture and using it to gain control over its ongoing
business systems investments. Enclosure III contains details on the status
of all of our prior recommendations, including our assessment of DOD's
actions.

In May 200117, we reported that the department did not have an
architecture for its financial and financial-related business operations,
nor the management structures, processes, and controls in place to
effectively develop and implement one. We reported that if the department
continued to spend billions of dollars on new and modified systems,
independently from one another and outside the context of an architecture,
this would result in more processes and systems that are duplicative, not
interoperable, and unnecessarily costly to maintain and interface. We made
eight recommendations aimed at providing the means for effectively
developing and implementing an architecture. The Secretary of Defense
established a program in July 2001 to develop and implement an
architecture. In April 2002, DOD entered into an agreement with
International Business Machines (IBM) pursuant to a governmentwide General
Services Administration contract, under which DOD issued task orders for
services to begin developing the architecture.

During the first year of DOD's architecture development, in 2002, we
reviewed the department's efforts and recognized that it was undertaking a
challenging and

16GAO-01-525; U.S. General Accounting Office, DOD Financial Management:
Important Steps Underway But Reform Will Require a Long-term Commitment,
GAO-02-784T (Washington, D.C.: June 4, 2002); and U.S. General Accounting
Office, Department of Defense: Further Actions Needed to Establish and
Implement a Framework for Successful Business Transformation, GAO-04-626T
(Washington, D.C.: Mar. 31, 2004).

17GAO-01-525.

ambitious task and following some architecture best practices and
information technology (IT) investment management processes and controls.
We also identified challenges and weaknesses in DOD's architecture
efforts. For example, in a February 2003 report,18 we pointed out that DOD
had not yet (1) established a governance structure and process controls
needed to ensure ownership of and accountability for the architecture
across the department, (2) clearly communicated to intended stakeholders
its purpose, scope, and approach for developing the architecture, and (3)
defined and implemented an independent quality assurance process. We also
reported that DOD had yet to establish the necessary departmental
investment governance structure and process controls needed to adequately
align ongoing investments with its architectural goals and direction. We
made six recommendations aimed at enhancing DOD's ability to further
develop its architecture and guide and constrain its business systems
modernization investments.

Our March 200319 report noted that the draft version of the architecture
did not include a number of items recommended by relevant architectural
guidance and that DOD's plans would not fully satisfy the requirements of
the National Defense Authorization Act for Fiscal Year 2003. For example,
the draft architecture did not include a "To Be" security view, which
defines the security requirements, including relevant standards to be
applied in implementing security policies, procedures, and controls. DOD
officials agreed with our preliminary assessment of the architecture and
stated that subsequent versions of the architecture would provide these
missing details.

In July and September 2003,20 we reported that although DOD had expended
tremendous effort and resources in complying with statutory requirements
for developing and implementing a well-defined architecture, the initial
version of its architecture, including the transition plan, did not
adequately address the statutory requirements and other relevant
architectural requirements. For example, the "As Is" environment did not
include descriptions of the current business operations in terms of
entities and people who perform the functions, processes, and activities
and the locations where these are performed. The "To Be" environment did
not include descriptions of actual systems to be developed or acquired to
support future business operations and the physical infrastructure that
would be needed to support the business systems. The transition plan did
not include time frames for phasing out existing systems within DOD's
reported current inventory of about 2,300 business systems. Overall, the
department's initial version of the architecture did not contain
sufficient scope and detail either to satisfy the act's requirements or to
effectively guide and constrain the departmentwide business transformation
and systems

18GAO-03-458.

19U.S. General Accounting Office, Information Technology: Observations on
Department of Defense's Draft Enterprise Architecture, GAO-03-571R
(Washington, D.C.: Mar. 28, 2003).

20GAO-03-877R and GAO-03-1018.

modernization. In our September 2003 report21, we reiterated the open
recommendations that we had made in our May 200122 and February 200323
reports, and we made 10 new recommendations aimed at improving DOD's plans
for developing the next version of the architecture and implementing the
institutional means for selecting and controlling both planned and ongoing
business systems investments. To date, DOD has yet to address 22 of our
recommendations.

Funding Status of Architecture Program

As of March 12, 2004, about $258 million had been appropriated to support
the program, of which the department reported having obligated over $203
million and having made disbursements of $111 million since the program
began in 2002. Table 1 shows the reported status of program funding by
appropriation and fiscal year.

21 GAO-03-1018.

22GAO-01-525.

23GAO-03-458.

Table 1: Reported Funding Status of DOD's Business Management
Modernization
Program as of March 12, 2004
(Dollars in millions)

Appropriation Appropriated Obligated Unobligateda Disbursed Unliquidatedb

Fiscal year 2002/2003
Research,
Development, Test,
and Evaluation
(RDT&E) -
Defensewide (DW)c $94.5 $94.5 $0.0 $83.0 $11.5

        Fiscal year 2003/2004                                        
                   RDT&E -DWd   67.2      67.2       0.0      11.9       55.3 
        Fiscal year 2004/2005                                        
                  RDT&E - DWe   45.1       8.5      36.6      0.0    

Fiscal year 2003
Operations and
Maintenance (O&M) -
DWf 24.9 24.9 0.0 16.1

Fiscal year 2004 O&M-
DWg 26.1 8.3 17.8 0.0

                     Total $257.8 $203.4 $54.4 $111.0 $92.4

Source: Unaudited funding information contained in DOD's March 15, 2004
report to congressional defense committees. We did not validate the
accuracy and reliability of the funding information reported.

a

Unobligated balances are the differences between funds appropriated and
obligated; they represent funds that have not been obligated for
expenditure. GAO's calculations are based on DOD's reported data.

b Unliquidated balances are the differences between the funds obligated
and disbursed; they represent funds that have been obligated but have not
yet been paid. GAO's calculations are based on DOD's reported data.

c The fiscal year 2002/2003 RDT&E funds supported the initial delivery of
the architecture, transition plan, and change management and
communications initiatives.

d The fiscal year 2003/2004 RDT&E funds were used to refine the
architecture through business process modeling/reengineering for the
program's first increment and to fund test and evaluation activities,
verification and validation efforts, and engineering support.

e The fiscal year 2004/2005 RDT&E funds will be used to complete the
program's first increment and begin work on the second increment and to
fund the integration of the architecture, engineering support, and test
and evaluation activities.

f The fiscal year 2003 O&M funds were used for salaries, facilities,
supplies, and program management support contracts.

g The fiscal year 2004 O&M funds are being used for salaries, facilities,
supplies, and program management support contracts.

DOD Has Taken Few Actions to Address Our Previous Recommendations

DOD has taken some steps since our last review, but it has not yet (1)
implemented key architecture management best practices,24 such as
assigning accountability and responsibility for directing, overseeing, and
approving the architecture to a committee or group comprised of
representatives from across DOD's major organization components, (2)
revised the architecture products to include the missing scope and detail
needed to guide and constrain the implementation of system solutions, and
(3) clearly defined near-term and long-term plans for guiding its
transformation activities. Until the department addresses these key
elements to fully satisfy relevant architectural guidance, it will be
challenged in its ability to produce an architecture that is sufficient to
guide and constrain its business operations and systems modernization
efforts. In response to our recommendations, DOD has taken some actions,
including establishing a group to facilitate communication and
coordination across the domains for program activities; beginning to
establish a configuration change management process; recently issuing a
policy governing the development, maintenance, and implementation of the
architecture; and updating the initial version of the architecture.
However, these actions are not sufficient and do not fully address our
concerns.

DOD Has Not Yet Implemented Key Architecture Management Best Practices

DOD's actions have not been adequate to address architecture management
best practices or our previous recommendations. Since our last report, the
department has taken some actions in response to our recommendations
concerning the need to implement an effective architecture management
program. First, in September 2003, it formally established the Domain
Owners Integration Team (DO/IT), which reports to the steering committee.
The DO/IT is comprised of the various senior executives from each domain
and the Business Modernization and Systems Integration office. The DO/IT
is responsible for facilitating communication and coordination across the
domains for program activities, including extending and evolving the
architecture. According to program officials, the DO/IT's role is
continuing to evolve. In particular, the specific actions or tasks it
needs to perform to carry out this responsibility have not been defined in
detail.

Second, DOD has taken steps to establish and implement a configuration
management process to ensure that all changes to the architecture products
are justified and are accounted for in a manner that maintains
documentation integrity. Specifically, the department has established a
configuration control board (CCB), developed a charter, and written
procedures governing this process. However, the CCB has been tasked with
reviewing changes to only some, but not all, of the architecture products.
For example, the CCB is not responsible for tracking changes that are
being made to the transition plan. In addition, both the charter and the
procedures governing this process are still in draft. According to DOD
officials, the

24GAO-03-584G.

charter is to be approved in June 2004, but no time frame has been
provided for final approval of the procedures.

Third, DOD has recently issued an IT portfolio management policy governing
the development, maintenance, and implementation of the architecture. This
policy addresses, among other things, the roles, responsibilities, and
relationships of key players and program participants, the value of an
architecture, and the scope of the architecture. However, the policy does
not address accountability for and approval of updates to the
architecture, as called for by best practices, nor does it address the
issuance of waivers in those instances when exceptions to the architecture
are justified on the basis of documented analysis.

Finally, DOD has developed high-level performance measures that are to be
used to develop the more specific results-oriented performance metrics
needed to enable it to evaluate program progress and benefits. This means
that DOD has yet to establish measurable, results-oriented goals to
evaluate and track, on an ongoing basis, specific program progress,
outcomes, and results. For example, it has not explicitly defined
performance measures to evaluate the quality, content, and utility of
subsequent major updates to its initial architecture. Given that DOD has
reported obligations of over $203 million since architecture development
efforts began 3 years ago, this is a serious performance management
weakness. It is critical that the department establishes meaningful,
tangible, and measurable program goals and objectives- short-term and
long-term-to enable the department to determine what value it is receiving
for its investment.

In addition, the department has yet to address other prior recommendations
related to architecture management best practices. Two examples are
provided below.

o  	DOD has not assigned accountability and responsibility for directing,
overseeing, and approving the architecture to a committee or group
comprised of representatives from across the department. Although it
previously established an executive and a steering committee, these
committees are advisory in nature, and they are currently not accountable
or responsible for directing, overseeing, and approving the architecture.
According to the department, it expects to revise the committees' charters
in June 2004 to include these responsibilities.

o  	DOD does not have an independent verification and validation function
to review the architecture products and management processes. The
department's current verification and validation contractor is not
independent25 and is responsible for reviewing the architecture products
and only selected program deliverables. According to program officials,
the department is planning to acquire the services of a new contractor to
perform this work, who will be tasked with reviewing all

25Best practices recommend that the verification and validation function
be independent of the architecture program and report directly to the
steering committee.

the architecture products and all other program deliverables. However, DOD
officials also said that no decision has been made to change the reporting
structure of this function to make it independent or to have this
contractor review architecture management processes. In addition, no
milestones have been set for any of these planned actions.

Several factors have contributed to the department's limited progress in
implementing an effective architecture management program, such as changes
to and a lack of accountability in program leadership, direction, and
priorities. As we previously stated, DOD has not yet formalized
responsibility for directing, overseeing, and approving the architecture.
Further, from May 2003 to February 2004, there was no program manager to
identify, direct, and execute program activities. DOD hired a new program
manager in February 2004. Initially, DOD had planned to develop and
implement its architecture in 1 year, but later it adopted an incremental
approach to developing the architecture. About 1 year after adopting this
approach, DOD has assigned labels to these increments, but it has not yet
defined the purpose of these increments and plans of action to implement
them.

Further, our research of successful organizations and experience in
reviewing other challenged enterprise architecture efforts show that
senior management's understanding of and commitment to an enterprise
architecture and overcoming cultural resistance to having and using one
are critical success factors. Cultural resistance to change, military
service parochialism, and stovepiped operations have all contributed
significantly to the failure of previous attempts to implement broadbased
management reforms at DOD. Until such barriers are addressed, and
effective architecture management structures and processes are
established, it is unlikely that DOD will be able to produce and maintain
a complete and enforceable architecture or implement modernized systems in
a way that minimizes overlap and duplication and maximizes integration and
mission support.

DOD's Architecture Products Remain Incomplete, with Minimal Content Change

Since our last review, DOD has not made significant changes to the content
of its architecture. The department has an updated version of the
architecture (version 2.0), which is currently being reviewed, but it has
not been approved. According to DOD, this version of the architecture
differs from the initial architecture (version 1.0) in that it
incorporates integrated "baseline reference business process models,"
which, according to DOD, constitute a high-level process framework
consisting of processes that provide a business perspective on the
department's operations (e.g., execute budget and performance plans) that
cut across functions and organizations. DOD also stated that version 2.0
includes the previously excluded requirements (e.g., revenue requirements)
for the Joint Financial Management Improvement Program (JFMIP)26 and
partially addresses 21 of the 62 missing architecture content elements

26JFMIP requirements arise from various public laws, regulations,
bulletins, circulars, federal accounting standards, and leading practices
and are applicable governmentwide.

that we recommended be added. According to DOD, the domains are currently
validating all of the requirements-including the JFMIP requirements-that
are contained in the architecture and, as a result, requirements may be
added or deleted.

Further, with regard to the 21 missing elements, documentation provided by
DOD shows that it is either just beginning to initiate activities or was
planning to develop plans to address these missing elements. This
documentation also did not specify either how or when any of the 21
elements would be fully addressed. Instead, it showed that the department,
to date, has made only cosmetic changes to the architecture, such as
correcting typographical (i.e., misspelled words) and grammatical errors,
as well as deleting unnecessary text from various architecture products.
DOD has yet to incorporate the missing scope and detail that we previously
identified. For example, the "To Be" environment does not provide
sufficient descriptive content related to future business operations and
supporting technology to permit effective acquisition and implementation
of system solutions and the associated operational change.

Program officials also stated that version 2.0 addresses some of the
recommendations made by its verification and validation contractor.
However, DOD did not provide documentation showing that it had addressed
any of these recommendations. For example, the verification and validation
contractor had previously reported that the "As Is" information was
insufficient to support realistic transition planning. In April 2004, this
contractor further reported that, while there have been some improvements
in both the overall completeness of definitions and the structure of the
architecture products, version 2.027 retains most of the critical problems
previously cited for version 1.0. Some examples are below.

o  	The utility of the architecture to stakeholders and the ability of the
architecture to support acquisition and portfolio investment management
decisions remain unclear.

o  	Supporting documentation that describes the analysis and rationale for
architecture choices represented within many of the architecture products
remains incomplete in many areas and, when available, is poorly linked or
referenced.

o  	The "As Is" environment, including business processes and existing
business application systems and supporting technology, is inadequate,
making it difficult for DOD to perform a gap analysis to support
development of a transition plan.

o  	The information assurance (security) representation remains spotty and
difficult to find even at the enterprise level, and it has some
unnecessary deviations from the department's accepted practices.

27The focus of this review was limited to the "As Is" and "To Be"
architecture products, because the transition plan has not been updated.

In addition, the verification and validation contractor stated that the
reference business process models included in version 2.0, which DOD cited
as the major difference from version 1.0, lacked the underlying data and
linkage to make it useful in reengineering existing processes. The
contractor also stated that these models would likely need to be reworked
based on the results of efforts that the domains currently have under way.

According to DOD and contractor officials responsible for updating the
architecture products, the changes made to date have primarily affected
architecture management processes and not the architecture's content; and
this is the result of the change in the program's focus and priorities, as
discussed above. However, these officials also stated that the
Architecture Integration Teams (AIT), comprised of representatives from
the domains, are currently focused on developing needed architectural
content-primarily business processes, business rules, and regulations for
increment one.

Until the architecture is sufficiently complete and includes the missing
scope and detail, the department remains at risk for not achieving its
intended business transformation goals and of not having an architecture
that can be used to guide and constrain ongoing and planned business
systems investments to prevent duplicative and noninteroperable systems.

DOD Has Yet To Explicitly Define Near-term and Long-term Plans for Guiding
Its Transformation Activities

DOD has not yet developed either near-term or long-term plans for
developing the architecture that explicitly identify and establish a
baseline for the actions to be taken, milestones to be achieved, cost
estimates to be met, and targeted outcomes to be achieved. As we
previously stated, DOD has adopted an incremental approach to developing
the architecture, including the transition plan, and plans to refine and
extend the architecture in three increments. The increments, as defined by
DOD, are:

o  	Increment one. To enable asset accountability, total force visibility,
and an unqualified audit opinion of DOD's consolidated fiscal year 2007
financial statements.

o  	Increment two. To focus on reducing acquisition cycle time and
streamlining the Planning, Programming, Budgeting, and Execution System
process between fiscal years 2004 and 2009.

o  	Increment three. To focus on providing total asset visibility and
total force management between fiscal years 2004 and 2010.

However, it is unclear what the increments individually or collectively
mean, and what they will provide or allow DOD to achieve in the near-term
and long-term, because DOD does not have detailed plans that include
performance measures for

the quality, content, and utility of the architecture. Although the three
increments were identified in November 2003, program officials do not
expect to have a plan for increment one until the next version of the
transition plan is completed in August 2004. According to program
officials, the goals and scope for the second and third increments were
only recently approved by the steering committee and, therefore, detailed
plans of action and milestones for developing these plans do not yet
exist.

Currently, DOD has three initiatives under way to support increment one.
First, the program office is developing a plan of action for increment one
and intends to complete the plan by August 2004. Second, the accounting
and finance domain is conducting workshops to develop needed business
rules and requirements for extending and evolving version 2.0 of the
architecture. This domain also has two ongoing pilot initiatives to
acquire and implement production accounting systems by the end of 2005 at
an estimated cost of $135 million, which are intended to provide
JFMIP-compliant financial management systems to support the department's
general and working capital fund activities. Last, DOD components are
developing individual plans detailing their respective efforts for
supporting increment one. However, there is no evidence that the program
office is coordinating with the components and that the components are
coordinating amongst themselves. According to a Defense Logistics Agency
(DLA) official, there is currently no direct link between DLA's plan and
modernization program activities. Because there are not yet detailed plans
guiding the program's activities, it is unclear whether and how these
activities support each other and whether they support the department's
goal of achieving an unqualified audit opinion in 2007.

DOD recognizes that it needs to develop detailed plans and establish
performance metrics to measure and track program progress in order to
determine (1) what it wants to accomplish by a certain point in time, (2)
what it has actually accomplished by that point in time, and (3) how much
it has spent. Changes in the direction of the program and a lack of
sustained leadership have hindered DOD's ability to do so. In its March
15, 2004 progress report, DOD reported that it plans to establish an
initial approved program metrics baseline to evaluate the cost, schedule,
and performance of the program and that, beginning with the fourth quarter
of fiscal year 2004, it plans to begin formal tracking and reporting of
specific program goals, objectives, and measures.

The lack of explicitly defined program plans also has made it difficult to
define measurable tasks that are or will be assigned to DOD employees and
the 289 program contractor employees who are involved in architecture
development, domain support, and program support functions. In reviewing
the contractor's work statements for architecture development, we found
that many of the tasks lacked the specificity necessary to use them
effectively to monitor the contractor's progress. For example, tasks
included such broad statements as "maintain, extend, and integrate" the
architecture. Since DOD is acquiring services on the basis of direct labor
hours at specified fixed hourly rates and cost of materials, it is
essential that the department use efficient methods and effective cost
controls to measure the work

being performed.28 It is also essential because the proposed work
statement that DOD plans to approve, at an estimated cost of $43 million,
explicitly states that work is considered complete when the hours allotted
have been expended, rather than defining completion as the contractor's
delivery of products that meet predefined quality standards. According to
DOD, to enable it to better monitor the contractor's progress, it plans to
reduce future tasks to 15 staff-day increments (from 6 week increments)
and to require the contractor to provide more frequent, informal products
to program officials so they can monitor and track progress more
rigorously.

Without an explicitly defined program baseline, detailed plans, and
performance measures, it is difficult to validate or justify the $122
million that DOD has requested for fiscal year 2005 and the $494 million
the department estimates it will need for fiscal years 2006 through 2009.
In fact, DOD has been unable to show measurable progress and meaningful
utility of the architecture products to DOD stakeholders for the $203.4
million that had been obligated for this program as of March 2004.

DOD Has Taken Few Actions to Control Ongoing and Planned Investments in
Business Systems

DOD continues to lack effective investment management oversight and
control over its numerous investments in business systems. While the
domains have been designated to oversee business systems investments, the
actual funding continues to be spread among DOD's components, which
continue to make their own parochial decisions regarding those
investments, including obligations in excess of $1 million for system
improvements, without having received the scrutiny of DOD's Comptroller as
required by the National Defense Authorization Act for Fiscal Year 2003.29
Because DOD lacks a departmentwide focus and effective management
oversight and control of its business systems investments, it continues to
invest billions of dollars in systems that potentially will fail to
deliver the integrated business systems outcomes

28We also found that in issuing task orders to International Business
Machines (IBM) pursuant to the General Services Administration contract,
DOD did not execute a Determination and Finding to support its use of
Time-and-Materials task orders, as required by Federal Acquisition
Regulation 16.601, 48 C.F.R. S: 16.601 (2003). This Determination and
Finding requirement guides agencies in evaluating the cost and performance
risks assumed by the government in awarding Time-and-Materials contracts
compared to other types of contracts that may provide increased incentives
for the contractor to control costs and efficiency.

29 Subsection 1004(d) of the Bob Stump National Defense Authorization Act
for Fiscal Year 2003, Pub.

L. No. 107-314, 116 Stat. 2630, (Dec. 2, 2002), provides that any amount
in excess of $1 million may be obligated for Defense financial system
improvements before approval of its enterprise architecture and a
supporting transition plan only if the DOD Comptroller makes a
determination that the improvement is necessary for (1) critical national
security capability or critical safety and security requirements or (2)
prevention of significant adverse effect on a project that is needed to
achieve an essential capability. The act further provides that after the
architecture and transition plan are approved, the DOD Comptroller must
determine, before making obligations that exceed $1 million for system
improvements, that such improvements are consistent with the enterprise
architecture and the transition plan.

that are needed to provide DOD management with timely and reliable
financial information.

We previously recommended30 that DOD establish investment review boards to
better control its business systems investments (with each board comprised
of representatives from across the department) and that the boards,
consistent with recognized best practices, use a standard set of
investment review and decisionmaking criteria to ensure compliance and
consistency with the architecture. DOD agreed with our recommendations
and, in response, it issued in March 2004 an IT portfolio management
policy that assigns the domains responsibility for IT portfolio
management. However, the procedures to be followed to implement the policy
are still under development and no time frames for completion have been
provided. According to the IT portfolio management policy, the department
has 180 days (until mid-September 2004) to develop these procedures. In
addition, the department has yet to formalize specific roles and
responsibilities of the domains, develop standard criteria for performing
the system reviews, and assign explicit authority for fulfilling roles and
responsibilities. DOD recognizes the need to clarify the roles and
responsibilities associated with managing the domains' portfolios of
business systems and ensure compliance with the architecture. However, it
has yet to establish time frames for completing these activities.

DOD has not yet implemented an effective investment management structure
and processes for controlling ongoing and planned business systems
investments, including one that meets the act's requirements for ensuring
that system improvements with obligations in excess of $1 million are
consistent with the architecture. In an attempt to substantiate that the
obligations for business systems modernization were in accordance with the
National Defense Authorization Act for Fiscal Year 2003, we requested that
DOD agencies provide us with a list of obligations greater than $1 million
for fiscal year 200331 and fiscal year 2004, as of December 2003. To
ascertain whether DOD's Comptroller had made the determination required by
the act, we compared a list of systems approvals provided by the program
office with the obligational data (by system) provided by the DOD
activities. As we previously testified,32 we identified $479 million in
obligations for business systems modernization reported by the DOD
military services that were not submitted to DOD's Comptroller for review.
Subsequently, we requested information from the defense agencies and found
$384 million in reported obligations for business systems modernization
that had not been submitted to DOD's Comptroller for review. Examples of
DOD system improvements with obligations in excess of $1 million that were
not submitted include the following:

30GAO-01-525 and GAO-03-458.

31We requested the obligational data for fiscal year 2003 for the period
December 2, 2002, the date of enactment of the act, through September
2003.

32GAO-04-626T.

o  	The Defense Finance and Accounting Service (DFAS) obligated about $19
million in fiscal year 2003 for the DFAS Corporate Database/DFAS Corporate
Warehouse. We previously reported33 that DOD had yet to provide economic
justification that its investment in this system would result in tangible
improvements to DOD's financial management operations. As of April 2004,
an economic analysis has yet to be approved but, according to DOD
officials, about $129 million had been spent on the program through
January 2004.

o  	The Defense Information Systems Agency obligated about $7 million in
fiscal year 2003 and about $2 million in fiscal year 2004 for the Wide
Area Workflow.34

o  	DFAS obligated about $7 million in fiscal year 2003 for the Defense
Standard Disbursing System before it was terminated in December 2003 after
approximately 7 years of effort and a reported investment of about $53
million. We previously reported35 that continued investment in this system
had not been justified, because an economic analysis had not been updated
to reflect significant schedule delays. DFAS noted that this system was
terminated because a valid business case for continuing the effort could
not be made.

o  	DLA obligated about $5 million in fiscal year 2003 and about $2
million in fiscal year 2004 for the Defense Medical Logistics Standard
Support Program.

These and other examples demonstrate that DOD does not have reasonable
assurance that it is in compliance with the National Defense Authorization
Act for Fiscal Year 2003, which provides that obligations in excess of $1
million for system improvements may not be made unless DOD Comptroller
makes a determination that the improvement is in accordance with the
criteria specified in the act. The act places limitations on the legal
authority of individual program and government contracting officials to
obligate funds in support of the systems for which they are responsible,
but DOD has yet to proactively manage its investments to avoid violations
of the limitations and to review and approve investments in any meaningful
way in order to comply with these statutory limitations.

DOD acknowledges that the department does not have a systematic means to
identify and determine which system improvements should be submitted to
DOD's Comptroller for review and, in essence, is dependent upon system
owners coming forward to the domains and requesting approval. Also, as we
have testified,36 DOD components continue to receive direct funding for
their business systems and continue to make their own parochial decisions
regarding investments without the

33U.S. General Accounting Office, DOD Business Systems Modernization:
Continued Investment in Key Accounting Systems Needs to be Justified,
GAO-03-465 (Washington, D.C.: Mar. 28, 2003).

34Wide Area Workflow is a secure web-based system for electronic
invoicing, receipt, and acceptance.

35GAO-03-465.

36GAO-04-626T.

scrutiny of DOD's Comptroller that is required by the act. The current
funding process has contributed to the proliferation of duplicative,
nonintegrated, and stovepiped business systems that are highly prone to
error and are unable to provide DOD management with timely, reliable
financial information. In March 2004,37 we offered a suggestion for
legislative action to address this issue, consistent with our open
recommendations to DOD that funds for its business systems be appropriated
directly to the domains in order to provide for accountability,
transparency, and the ability to prevent the continued parochial approach
to systems investments that exists today. The legislation we
envisioned-for which we will make a legislative proposal in a related
report to be issued soon-would define the scope of the domains and
establish functional responsibility for managing the portfolio of business
systems to the domains. The domains would establish business systems
investment review boards with DOD-wide representation, including the
military services and defense agencies.

Until DOD strengthens its process for selecting and controlling business
systems investments and sufficiently develops a well-defined architecture
that can be used to guide and constrain investment decisions, it will
likely continue to spend billions of dollars on duplicative,
nonintegrated, stovepiped systems that do not optimize mission performance
and accountability and, therefore, do not support the department's
transformation goals.

Conclusions

Having and effectively using a well-defined architecture is essential for
guiding and constraining DOD's business transformation efforts and moving
the department away from nonintegrated business systems development
efforts. DOD's efforts to date to address our prior recommendations aimed
at accomplishing this have not been sufficient. Specifically, despite 3
years of effort and over $203 million in reported obligations, DOD's
architecture remains insufficiently defined, and the way in which the
department makes business systems investments decisions remains largely
unchanged. As a result, billions of dollars continue to be at risk of
being spent on more systems that are duplicative, are not interoperable,
cost more to maintain than necessary, and do not optimize mission
performance and accountability.

The future of DOD's architecture development and implementation activities
is at risk; this in turn places the department's business transformation
effort in jeopardy of failing as other efforts by the department have in
the past. Therefore, it is imperative that DOD move swiftly to implement
our 22 open recommendations, which are aimed at strengthening architecture
management activities, adding missing content to architecture products,
and implementing investment oversight and control measures.

37U.S. General Accounting Office, Department of Defense: Further Actions
Needed to Establish and Implement a Framework for Successful Financial and
Business Management Transformation,

GAO-04-551T (Washington, D.C.: Mar. 23, 2004) and GAO-04-626T.

Agency Comments and Our Evaluation

In commenting on a draft of this report (reprinted in enc. IV), DOD agreed
with our assessment that its long-standing business systems problems have
resulted in a lack of adequate transparency and appropriate accountability
across all major business areas. It confirmed that development and use of
an architecture is necessary to transforming its business operations and
supporting systems. However, in response to our characterization that
progress has been limited, DOD stated that, over the past 3 years,
progress has been slower than either GAO or DOD would prefer but that it
has been significant, despite appearances to the contrary. The willingness
of DOD's leadership to tackle the department's decades-old problems with
its business operations and supporting systems represents a major step
forward. However, we continue to believe that our characterization of
DOD's progress in developing a welldefined architecture and implementing
effective management oversight and control over its business systems as
limited is fair. Therefore, we are not making any changes to our report.

DOD stated that we did not adequately consider or incorporate in our
report important completed and ongoing activities, plans, and
documentation, and provided an extensive package of information. We
disagree that we did not adequately consider or incorporate in our report
important completed and ongoing activities, plans, and documentation.
While our assessment focused primarily on the department's completed,
ongoing, and planned activities that it reported to Congress on March 15,
2004, we also considered and incorporated various activities, plans, and
documentation through April 27, 2004, which included the vast majority of
the information in the package.

In addition, we acknowledged in this report most, if not all, of the
actions DOD represents in their comments as having been completed such as
issuing the IT portfolio management policy and establishing the DO/IT. We
also recognized many of DOD's ongoing activities, such as the workshops
being held by the accounting and finance domain and the department's
establishment of the AIT, which are currently focused on developing needed
architectural content for increment one. To gain an understanding of the
specific AIT activities, which began in January 2004, we met with the
domains during our review and discussed various collaborative efforts,
such as their participation in validating business rules and requirements
to support the development of end-to-end business processes. Regarding
planned activities, we recognized DOD's intent to revise the executive and
steering committees' charters to include responsibilities for directing,
overseeing, and approving the architecture, and to develop the procedures
for implementing the IT portfolio management policy. In its response, DOD
stated that it plans to complete these actions in June and December 2004,
respectively.

Because we received information about certain activities after we
completed our fieldwork-or not at all-these activities are not included in
our evaluation. For example, DOD stated that the department plans to
release a management initiative during May 2004; establish an architecture
and program management maturity plan

and supporting metrics to address the quality, content, and utility of the
architecture; and establish a balanced scorecard and supporting metrics
focused on the organization's internal management processes. While we
inquired about the management initiative during our review, program
officials stated that they were unable to provide us a copy because DOD
does not allow the release of such draft documents outside the department.
In addition, program officials did not provide a characterization of the
focus or scope of the management initiative; therefore we were unable to
include it in our report. We made inquiries about other initiatives during
the course of our review, but were not advised of the maturity plan and
balanced scorecard. However, it is important to note that carrying out
activities does not necessarily guarantee results, and thus progress,
unless these activities are based on an approved program plan that can be
used to measure progress.

In addition, we disagree with DOD's other comments. Specifically, DOD
stated that the Comptroller is accountable for approval of updates to the
architecture as called for by best practices. As we previously reported38
and reiterated in this report, best practices recommend that the
accountability and responsibility for directing, overseeing, and approving
the architecture be assigned to a committee or group comprised of
representatives responsible for the core business areas across the
department-not a single individual. The purpose of assigning the
accountability and responsibility to such a committee is to obtain and
ensure continued enterprisewide support of the architecture effort,
thereby, increasing the department's chances for success.

In addition, DOD stated that its verification and validation function is
independent because the contractor responsible for these activities
currently reports to the Deputy Director, Program Support, who is not
directly involved with developing the architecture. DOD also stated that
to ensure this independence, reports prepared by this contractor are
provided directly to GAO and the steering committee. We disagree that this
function is sufficiently independent. According to best practices, the
contractor performing this role should report the results of its work
directly to the steering committee, as well as the program office. Since
the Deputy Director, Program Support reports directly to the program
management office and not the steering committee, the function is not
independent. In addition, the verification and validation contractor
agreed with us that its reports are not provided directly to GAO or the
steering committee. The contractor also concurred with our statement that,
in the one instance in which it briefed the steering committee, the
program office modified the content of the presentation.

DOD disagreed with our prior position that funds for DOD business systems
be appropriated directly to the domains. We are reaffirming that position
as stated by the Comptroller General in his March 2004 testimony39 before
the Subcommittee on Readiness and Management Support, Senate Armed
Services Committee. This matter

38 GAO-03-458.

39 GAO-04-551T.

for congressional consideration and related changes in responsibility will
be discussed in additional detail in a report to be issued shortly.

We are sending copies of this report to interested congressional
committees; the Director, Office of Management and Budget; the Secretary
of Defense; the Under Secretary of Defense (Comptroller); the Assistant
Secretary of Defense (Networks and Information Integration)/DOD Chief
Information Officer; the Under Secretary of Defense (Acquisition,
Technology, and Logistics); the Under Secretary of Defense (Personnel and
Readiness); and the Director, Defense Finance and Accounting Service. This
report will also be available at no charge on our Web site at
http://www.gao.gov.

If you have any questions concerning this information, please contact
Gregory Kutz at (202) 512-9095 or [email protected] or Randolph Hite at (202)
512-3439 or [email protected]. GAO contacts and key contributors to this
report are listed in enclosure V.

Gregory D. Kutz
Director, Financial Management and Assurance

Randolph C. Hite
Director, Information Technology Architecture

and Systems Issues

Enclosures

List of Committees

The Honorable John W. Warner
Chairman
The Honorable Carl Levin
Ranking Minority Member
Committee on Armed Services
United States Senate

The Honorable Ted Stevens
Chairman
The Honorable Daniel K. Inouye
Ranking Minority Member
Subcommittee on Defense
Committee on Appropriations
United States Senate

The Honorable Duncan Hunter
Chairman
The Honorable Ike Skelton
Ranking Minority Member
Committee on Armed Services
House of Representatives

The Honorable Jerry Lewis
Chairman
The Honorable John P. Murtha
Ranking Minority Member
Subcommittee on Defense
Committee on Appropriations
House of Representatives

Enclosure I

 SEC. 1004. [of Public Law 107-314] DEVELOPMENT AND IMPLEMENTATION OF FINANCIAL
                       MANAGEMENT ENTERPRISE ARCHITECTURE

(a) REQUIREMENT FOR ENTERPRISE ARCHITECTURE AND FOR TRANSITION
PLAN-
Not later than May 1, 2003, the Secretary of Defense shall develop-
(1) a financial management enterprise architecture for all budgetary,
accounting,
finance, enterprise resource planning, and mixed information systems of
the
Department of Defense; and
(2) a transition plan for implementing that financial management
enterprise
architecture.
(b) COMPOSITION OF ENTERPRISE ARCHITECTURE-
(1) The financial management enterprise architecture developed under
subsection (a)(1) shall describe an information infrastructure that, at a
minimum,
would enable the Department of Defense to-

(A) comply with all Federal accounting, financial management, and
reporting
requirements;
(B) routinely produce timely, accurate, and reliable financial information
for
management purposes;
(C) integrate budget, accounting, and program information and systems; and
(D) provide for the systematic measurement of performance, including the
ability
to produce timely, relevant, and reliable cost information.

(2) That enterprise architecture shall also include policies, procedures,
data
standards, and system interface requirements that are to apply uniformly
throughout
the Department of Defense.
(c) COMPOSITION OF TRANSITION PLAN-The transition plan developed under
subsection (a)(2) shall include the following:
(1) The acquisition strategy for the enterprise architecture, including
specific time
phased milestones, performance metrics, and financial and nonfinancial
resource
needs.
(2) A listing of the mission critical or mission essential operational and
developmental financial and nonfinancial management systems of the
Department of
Defense, as defined by the Under Secretary of Defense (Comptroller),
consistent with
budget justification documentation, together with-

(A) the costs to operate and maintain each of those systems during fiscal
year
2002; and
(B) the estimated cost to operate and maintain each of those systems
during
fiscal year 2003.

(3) A listing of the operational and developmental financial management
systems of the Department of Defense as of the date of the enactment of
this Act (known as `legacy systems') that will not be part of the
objective financial and nonfinancial management system, together with the
schedule for terminating those legacy systems that provides for reducing
the use of those legacy systems in phases.

(d) CONDITIONS FOR OBLIGATION OF SIGNIFICANT AMOUNTS FOR FINANCIAL SYSTEM
IMPROVEMENTS-An amount in excess of $1,000,000 may be obligated for a
defense financial system improvement only if the Under Secretary of
Defense (Comptroller) makes a determination regarding that improvement as
follows: (1) Before the date of an approval specified in paragraph (2), a
determination that the defense financial system improvement is necessary
for either of the following reasons:

(A) To achieve a critical national security capability or address a
critical
requirement in an area such as safety or security.
(B) To prevent a significant adverse effect (in terms of a technical
matter, cost, or
schedule) on a project that is needed to achieve an essential capability,
taking into
consideration in the determination the alternative solutions for
preventing the
adverse effect.

(2) On and after the date of any approval by the Secretary of Defense of a
financial
management enterprise architecture and a transition plan that satisfy the
requirements of this section, a determination that the defense financial
system
improvement is consistent with both the enterprise architecture and the
transition
plan.
(e) CONGRESSIONAL REPORTS-Not later than March 15 of each year from 2004
through 2007, the Secretary of Defense shall submit to the congressional
defense
committees a report on the progress of the Department of Defense in
implementing
the enterprise architecture and transition plan required by this section.
Each report
shall include, at a minimum-
(1) a description of the actions taken during the preceding fiscal year to
implement
the enterprise architecture and transition plan (together with the
estimated costs of
such actions);
(2) an explanation of any action planned in the enterprise architecture
and transition
plan to be taken during the preceding fiscal year that was not taken
during that fiscal
year;
(3) a description of the actions taken and planned to be taken during the
current
fiscal year to implement the enterprise architecture and transition plan
(together with
the estimated costs of such actions); and
(4) a description of the actions taken and planned to be taken during the
next fiscal
year to implement the enterprise architecture and transition plan
(together with the
estimated costs of such actions).
(f) COMPTROLLER GENERAL REVIEW-Not later than 60 days after the approval
of
an enterprise architecture and transition plan in accordance with the
requirements of
subsection (a), and not later than 60 days after the submission of an
annual report
required by subsection (e), the Comptroller General shall submit to the
congressional
defense committees an assessment of the extent to which the actions taken
by the
Department comply with the requirements of this section.
(g) DEFINITIONS-In this section:
(1) The term `defense financial system improvement' means the acquisition
of a new
budgetary, accounting, finance, enterprise resource planning, or mixed
information
system for the Department of Defense or a modification of an existing
budgetary,
accounting, finance, enterprise resource planning, or mixed information
system of

the Department of Defense. Such term does not include routine maintenance
and
operation of any such system.
(2) The term `mixed information system' means an information system that
supports
financial and non-financial functions of the Federal Government as defined
in Office
of Management and Budget Circular A-127 (Financial Management Systems).
(h) REPEAL-(1) Section 2222 of title 10, United States Code, is repealed.
The table
of sections at the beginning of chapter 131 of such title is amended by
striking the
item relating to such section.
(2) Section 185(d) of such title is amended by striking `has the meaning
given that
term in section 2222(c)(2) of this title' and inserting `means an
automated or manual
system from which information is derived for a financial management system
or an
accounting system'.

Enclosure II

Scope and Methodology

Consistent with the act and as agreed with congressional defense
committees' staffs, our review focused on (1) the actions the Department
of Defense (DOD) has taken to address our previous recommendations
regarding the development and implementation of the architecture and (2)
the actions DOD is taking to ensure its ongoing and planned investments
will be consistent with its evolving architecture.

To determine what actions DOD has taken to address our previous
recommendations that relate to the development and implementation of the
architecture, we met with DOD and contractor officials to obtain an update
on the status of our prior recommendations. We used our Enterprise
Architecture Management Maturity Framework,40 which describes the stages
of management maturity, to update the status of key elements of
architecture management best practices that DOD had not adopted. To make
this determination, we reviewed program documentation, such as program
policies and procedures, communications plan, and charters for the
governance bodies; and we compared them to the elements in the framework.
We reviewed program documentation, such as requests documenting proposed
and actual changes to the architecture and external41 requirements
extracted from the updated architecture. We also reviewed contractor task
orders to determine the work performed and planned by the prime contractor
associated with extending and evolving the architecture.

To determine what actions DOD is taking to ensure its ongoing and planned
business systems investments are consistent with its evolving
architecture, we met with DOD officials to obtain an update on the status
of our prior recommendations. We met with appropriate officials from the
offices of the Under Secretary of Defense (Comptroller) and the Assistant
Secretary of Defense (Networks and Information Integration)/DOD Chief
Information Officer, and with representatives from the domains to discuss
the status of various draft policies and guidance that are aimed at
improving the department's control of and accountability for business
systems investments. We also reviewed and analyzed DOD's budget request
for fiscal year 2004 to identify the business systems investments that
could be subject to the requirements of the National Defense Authorization
Act for Fiscal Year 2003, which requires that all financial system
improvements with obligations in excess of $1 million be reviewed by DOD's
Comptroller, who must make a determination on whether the system
improvements are in accordance with criteria specified in the act. To
assess DOD's compliance with the act, we also obtained and reviewed
departmental guidance, memorandums, DOD Comptroller review decisions, and

40U.S. General Accounting Office, Information Technology: A Framework for
Assessing and Improving Enterprise Architecture Management (Version 1.1),
GAO-03-584G (Washington, D.C.: April 2003).

41External requirements are those that are obtained from authoritative
sources, such as the Joint Financial Management Improvement Program, and
constrain various aspects of the architecture.

other documentation provided by the program office. Additionally, we
requested that DOD provide us with data on obligations in excess of $1
million for business systems modernization for fiscal years 2003 and 2004,
as of December 2003. We also received obligational data from some of the
defense agencies. We then compared the obligational data with the
information we had received from the program office to ascertain if the
systems modernization had been reviewed.

To augment our document reviews and analyses, we interviewed officials
from various DOD organizations and contractors, including the Office of
the Under Secretary of Defense (Comptroller); Office of the Under
Secretary of Defense (Acquisition, Technology, and Logistics); Office of
the Under Secretary of Defense (Personnel and Readiness); International
Business Machines; and MITRE Corporation.

We did not validate the accuracy and reliability of the funding
information contained in DOD's March 15, 2004, report. Also, the
unliquidated and unobligated calculations that we made were based on DOD
reported data. Further, we did not validate the accuracy and reliability
of the obligational data provided by the military services and the defense
agencies for systems modernization.

We requested comments on a draft of this report from the Secretary of
Defense or his designee. We received written comments from the Acting
Under Secretary of Defense (Comptroller), which we printed in enclosure
IV. We considered, but did not reprint the voluminous attachments that DOD
provided with its written comments.

We conducted our work primarily at DOD headquarters offices in Washington,
D.C., and Arlington, Virginia, from December 2003 through April 2004, in
accordance with U.S. generally accepted government auditing standards.

Enclosure III

Status of Prior Recommendations on DOD's Business Enterprise Architecture
(BEA)

                                    Page 29
                 GAO-04-731R DOD Business Systems Modernization
                                                                                                                                                                     The         
                                                                                                                                  DOD recently                       department  
                                                                                                                                  issued its                         had         
                                                                                                                                  Information                        previously  
                                                                                                                                  Technology (IT)                    established 
                                                                                                                                  Portfolio                          the         
                                                                                                                                  Management                         executive   
                                                                                           (1) The                                policy. This                       and         
                                                                                           Secretary of                           policy, in                         steering    
                                                                                           Defense                                conjunction with                   committees, 
                                                                                           immediately                            the overarching                    which we    
                                                                                           designate DOD      (2) The             Global                             reported    
                                                                                           financial          Secretary           Information                        were        
                                                                                           management         immediately         Grida policy,                      advisory in 
                                                                                           modernization      issue a DOD         assigns                            nature. The 
                                                                                           a                  policy that         responsibilities (3) The           department  
                                                                                           departmental       directs the         for the          Secretary         recently    
                 Status of               Implemented                                       priority and  X    development,     X  development,     immediately    X  established 
               recommendation                                                              accordingly        implementation,     implementation,  modify the        the Domain  
                                                                                           direct the         and maintenance     and maintenance  Senior            Owners      
                                                                                           Deputy             of a financial      of the BEA.b     Financial         Integration 
                                                                                           Secretary of       management          However, it does Management        Teamc and   
                                                                                           Defense to         enterprise          not provide for  Oversight         stated that 
                                                                                           lead an            architecture.       having           Council's         these three 
                                                                                           integrated                             accountability   charter to  o     bodies are  
                                                                               GAO-01-525: program                                for and approval  designate        responsible 
                                                                               Information across the                             of updates to    the Deputy        for         
                                                                               Technology: department                             the BEA,         Secretary of      governing   
                                                                              Architecture for                                    processes for    Defense as        the         
                                                                                 Needed to modernizing                            BEA oversight    the Council       program.    
                                                                                     Guide and                                    and control, and Chair and the     However,    
                                                                             Modernization optimizing                             review and       Under             these three 
                                   DOD's                                          of DOD's financial                              validation of    Secretary of      groups have 
                                comments                                         Financial management                             the BEA.         Defense           not been    
                                 and our             Partially   Not           Operations. operations                                              (Comptroller)     assigned    
Recommendation                assessment             implemented implemented May 17, 2001. and systems.                                            as                the         

                                    Page 30
                 GAO-04-731R DOD Business Systems Modernization
                                                                             the Council                         
                                                                             vice-Chair;  o    accountability    
                                                                              empower the      and               
                                                                             Council to        responsibility    
                                                                             serve as DOD's    for directing,    
                                                                             financial         overseeing, and   
                                                                             management        approving the     
                                                                             enterprise        BEA. According to 
                                                                             architecture      DOD, it expects   
                                                                             steering          to revise and     
                                                                             committee,        finalize the      
                                                                             giving it the     executive and     
                                                                             responsibility    steering          
                                                                             and authority     committees'       
                                                                             to ensure that    charters to       
                                                                             a DOD             include these     
                                                                             financial         responsibilities  
                                                                             management        in June 2004. DOD 
                                                                             enterprise        issued an IT      
                                                                             architecture      portfolio         
                                                                             is developed      management policy 
                                                                             and maintained    (March 2004) that 
                                                                             in accordance     assigns the       
                                                                             with the DOD      domains d         
                                                                             C4ISR             responsibility    
                                                                             Architecture      for IT portfolio  
                                                                             Framework;  o     management.       
                                                                             empower the       However, the      
                                                                             Council to        procedures to be  
                                                                             serve as DOD's    followed to       
                                                                             financial         implement the     
                                                                             management        policy are still  
                                                                             investment        under development 
                 Status of                                                   review board,     and no time       
               recommendation            Implemented                         giving it the     frames for        
                                                                             responsibility    completion have   
                                                                             and authority     been provided.    
                                                                             to (1) select     According to the  
                                                                             and control       IT portfolio      
                                                                             all DOD           management        
                                                                             financial         policy, the       
                                                                             management        department has    
                                                                             investments       180 days (until   
                                                                             and (2) ensure    mid-September     
                                                                             that its          2004) to develop  
                                                                             investment        these procedures. 
                                                                             decisions         In addition, the  
                                                                             treat             department has    
                                                                             compliance        yet to formalize  
                                                                             with the          specific roles    
                                                                             financial         and               
                                                                             management        responsibilities  
                                                                             enterprise        of the domains,   
                                                                             architecture      develop standard  
                                                                             as an explicit    criteria for      
                                                                             condition for     performing the    
                                                                             investment        system reviews,   
                                                                             approval that     and assign        
                                                                             can be waived     explicit          
                                                                             only if           authority for     
                                                                             justified by a    fulfilling roles  
                                                                             compelling        and               
                                                                             written           responsibilities. 
                                                                             analysis; and     DOD has yet to    
                                                                             o  expand the     establish time    
                                   DOD's                                     role of the       frames for        
                                comments                                     Council's         completing these  
                                 and our             Partially   Not         System            activities.
Recommendation                assessment             implemented implemented Compliance        

                                    Page 31
                 GAO-04-731R DOD Business Systems Modernization
                                                                                              (4) The                              
                                                                                              Secretary                            
                                                                                              immediately                          
                                                                                              make the                             
                                                                                              Assistant                            
                                                                                              Secretary of                         
                                                                                              Defense                              
                                                                                              (Command,                            
                                                                                              Control,            The Assistant    
                                                                                              Communications      Secretary of     
                                                                                              &                   Defense          
                                                                                              Intelligence),      (Networks and    
                                                                                              in                  Information      
                                                                                              collaboration       Integration)/DOD 
                                                                                              with the Under      Chief            
                                                                                              Secretary of        Information      
                                                                                              Defense             Officer is a     
                                                                                              (Comptroller),      member of the    
                                                                                              accountable to      executive and    
                                                                                              the Senior          steering         
                                                                                              Financial           committees;      
                                                                                              Management          however, as      
                                                                                              Oversight           discussed        
                                                                                              Council for         previously,      
                                                                                              developing and      members' roles   
                                                                                              maintaining a       and              
                                                                                              DOD financial       responsibilities 
                                                                                              management          are advisory in  
                                                                                              enterprise          nature. DOD      
                 Status of                                                                    architecture.       established a    
               recommendation            Implemented                                          In fulfilling    X  Financial        
                                                                                              this                Management       
                                                                                              responsibility,     Modernization    
                                                                                              the Assistant       Program Office   
                                                                                              Secretary           in July 2001.    
                                                                                              appoint a Chief     DOD has also     
                                                                                              Architect for       appointed a      
                                                                                              DOD financial       chief architect  
                                                                                  Working     management          and, according   
                                                                                 Group to     modernization       to the           
                                                                                  include     and establish       department, it   
                                                                               supporting     and adequately      has adequate     
                                                                              the Council     staff and fund      program funding  
                                                                                       in     an enterprise       and staff for    
                                                                              determining     architecture        developing and   
                                                                                      the     program office      maintaining its
                                                                               compliance     that is             BEA. However,
                                                                                  of each     responsible for     DOD has not yet
                                                                                   system     developing and      defined the
                                                                               investment     maintaining a       roles and
                                                                                 with the     DODwide             responsibilities
                                                                               enterprise     financial           for the Chief
                                                                             architecture     management          Architect.
                                                                                   at key     enterprise          
                                                                                 decision     architecture in     
                                                                                points in     a manner that       
                                                                             the system's     is consistent       
                                   DOD's                                      development     with the            
                                comments                                               or     framework           
                                 and our             Partially   Not          acquisition     defined in the      
Recommendation                assessment             implemented implemented  life cycle.     CIO Council's       

                                    Page 32
                 GAO-04-731R DOD Business Systems Modernization
                                                                             published                                                                      
                                                                             guide for                                                                      
                                                                             managing                                                                       
                                                                             enterprise                                                                     
                                                                             architectures.                                                       
                                                                             In particular,                                                       
                                                                             the Assistant                                                        
                                                                             Secretary                                                            
                                                                             should take                                                          
                                                                             appropriate                           The Assistant                  
                                                                             steps to                              Secretary of                   
                                                                             ensure that        (5) The            Defense                        
                                                                             the Chief          Assistant          (Networks and                  
                                                                             Architect  o       Secretary of       Information                    
                                                                             obtains            Defense            Integration)/DOD               
                                                                             executive          (Command,          Chief                          
                                                                             buy-in and         Control,           Information                    
                                                                             support;  o        Communications     Officer is a                   
                                                                             establishes        &                  member of the                  
                                                                             architecture       Intelligence)      steering                       
                                                                             management         reports at         committee, which               
                                                                             structure and      least              is briefed                     
                 Status of                                                   controls;  o       quarterly to       monthly by the                 
               recommendation            Implemented                         defines the        the Senior      X  program office              X  
                                                                             architecture       Financial          on various                     
                                                                             process and        Management         program                        
                                                                             approach;  o       Oversight          activities; but                
                                                                             develops the       Council on the     not necessarily                
                                                                             baseline           Chief              on the                         
                                                                             architecture,      Architect's        development and                
                                                                             the target         progress in        maintenance of                 
                                                                             architecture,      developing a       the BEA's                      
                                                                             and the            financial          content. The                   
                                                                             sequencing         management         Chief Architect                
                                                                             plan;  o           enterprise         is a member of                 
                                                                             facilitates        architecture,      the program                    
                                                                             the use of the     including the      office.                        
                                                                             architecture       Chief                                             
                                                                             to guide           Architect's                                       
                                                                             financial          adherence to                                      
                                                                             management         enterprise                                        
                                                                             modernization      architecture                                      
                                                                             projects and       policy and                                        
                                   DOD's                                     investments;       guidance from                                           The
                                comments                                     and  o             OMB, the CIO                        (6) The          Deputy
                                 and our             Partially   Not         maintains the      Council, and                        Senior            Chief
Recommendation                assessment             implemented implemented architecture.      DOD.                                Financial     Financial

                                    Page 33
                 GAO-04-731R DOD Business Systems Modernization
                                                                                                                                                              DOD has not yet   
                                                                                                                                                              defined and       
                                                                                                                                                              implemented an    
                                                                                                                                                              approach for      
                                                                                                                                                              selecting and     
                                                                                                                                                              controlling       
                                                                                                                                                              business systems  
                                                                                                                                                              investments. In   
                                                                                                                                                              March 2004, the   
                                                                                                                                                              Deputy Secretary  
                                                                                                                                                              of Defense signed 
                                                                                                                                                              the IT portfolio  
                                                                                                                                                              management policy 
                                                                                                                                                              and assigned      
                                                                                                                                                              responsibility to 
                                                                                                                                         (8) Until a          the domains for   
                                                                                                                                         financial            IT portfolio      
                                                                                                                                         management           management.       
                                                                                                                                         enterprise           However, the      
                                                                                                                                         architecture is      procedures to be  
                                                                                                           (7) The                       developed and        followed to       
                                                                                                           Secretary                     the Council is       implement the     
                                                                                                           reports every                 positioned to        policy are still  
                                                                                              Officer      6 months to       Senate      serve as DOD's       under development 
                                                                                              briefed the  the               Report      financial            and no time       
                                                                                              Secretary of congressional     107-213     management           frames for        
                                                                                              Defense in   defense           directs     investment           completion have   
                                                                                              November     authorizing       that the    review board as      been provided.    
                 Status of                                                                    2003 on      and               department  recommended          According to the  
               recommendation            Implemented                                          behalf of    appropriating  X  report      above, the         X IT portfolio      
                                                                                              DOD's        committees on     every 6     Secretary of         management        
                                                                                              Comptroller, progress in       months on   Defense limit        policy, the       
                                                                                              who chairs   developing        the status  DOD components'      department has    
                                                                                              the          and               of the BEA  financial            180 days (until   
                                                                                              executive    implementing      effort. DOD management           mid-September     
                                                                                              committee.   a financial       submitted   investments to       2004) to develop  
                                                                                                           management        status      o  deployment of     these procedures. 
                                                                                                           enterprise        reports on  systems that         In addition, the  
                                                                                                           architecture.     January 31  have already         department has    
                                                                                                                             and July    been fully           yet to formalize  
                                                                                                                             31, 2003    tested and           specific roles    
                                                                                                                             and on      involve no           and               
                                                                                                                             January 31, additional           responsibilities  
                                                                                                                             2004.       development or       of the domains,   
                                                                             Management                                      However,    acquisition          develop standard  
                                                                             Oversight                                       these       cost;  o             criteria for      
                                                                             Council                                         reports     stay-in-business     performing the    
                                                                             report to the                                   were        maintenance          system reviews,   
                                                                             Secretary of                                    submitted   needed to keep       and assign        
                                                                             Defense every                                   by DOD's    existing systems     explicit          
                                                                             6 months on                                     Comptroller operational;  o      authority for     
                                                                             progress in                                     and were    management           fulfilling roles  
                                                                             developing                                      not signed  controls needed      and               
                                                                             and                                             by the      to effectively       responsibilities. 
                                                                             implementing                                    members of  invest in            DOD has yet to    
                                   DOD's                                     a financial                                     the         modernized           establish time    
                                comments                                     management                                      executive   systems; and  o      frames for        
                                 and our             Partially   Not         enterprise                                      or steering new systems or       completing these  
Recommendation                assessment             implemented implemented architecture.                                   committees. existing system      activities.       

                                    Page 34
                 GAO-04-731R DOD Business Systems Modernization
                                                                                                                                  The department                                                                  
                                                                                                                                  had previously                                                                  
                                                                                                                                  established the                                                                 
                                                                                                                                  executive and                                                                   
                                                                                                                                  steering                                                                        
                                                                                                                                  committees,                                                                     
                                                                                                                                  which we                                                                        
                                                                                                                                  reported were                                                                   
                                                                                                                (1) The           advisory in                                                                     
                                                                                                                Secretary of      nature. The                                                        According to 
                                                                                                                Defense           department                                                         DOD, the     
                                                                                                                ensure that       recently                                                           quality      
                                                                                                                the               established the                                                    assurance    
                                                                                                                enterprise        Domain Owners                                                      function     
                                                                                                                architecture      Integration Team                   DOD has a                       does not yet 
                                                                                changes that                    executive         and stated that                    strategic                       address      
                                                                                         are                    committee         these three                        communications                  process      
                                                                             congressionally                    members are       bodies are                         plan; however,                  standards    
                                                                             directed or are                    singularly        responsible for                    the plan has                    and program  
                                                                                  relatively                    and               governing the                      not yet been                    performance, 
                 Status of                                                       small, cost                    collectively      program.                           executed.                       nor is it    
               recommendation            Implemented                          effective, and                    made            X However, these                  X  According to   (3) The       X  yet an       
                                                                                low risk and                    explicitly        groups have not                    DOD, the plan  Secretary of     independent  
                                                                                      can be                    accountable       been assigned                      is scheduled   Defense          function.    
                                                                              delivered in a                    to the            the                                to be executed ensure that      Further, DOD 
                                                                                  relatively                    Secretary for     accountability                     between June   the quality      subject
                                                                                  short time                    the delivery      and                                and December   assurance        matter
                                                                                      frame.                    of the            responsibility                     2004.          function  o      experts
                                                                                                                enterprise        for directing,                                    includes the     continue to
                                                                                                 GAO-03-458:    architecture,     overseeing, and                                   review of        be involved
                                                                                                 DOD Business   including         approving the    (2) The                          adherence to     in the
                                                                                                 Systems        approval of       BEA. According   Secretary of                     process          quality
                                                                                                 Modernization: each version      to DOD, it       Defense                          standards        assurance
                                                                                                 Improvements   of the            expects to       ensure that                      and              function.
                                                                                                 to Enterprise  architecture.     revise and       the                              reliability      
                                                                                                 Architecture                     finalize the     enterprise                       of reported      
                                                                                                 Development                      executive and    architecture                     program          
                                                                                                 and                              steering         program is                       performance,     
                                                                                                 Implementation                   committees'      supported by                      o  is made      
                                   DOD's                                                         Efforts                          charters to      a proactive                      independent      
                                comments                                                         Needed.                          include these    marketing and                    of the           
                                 and our             Partially   Not                             February 28,                     responsibilities communication                    program          
Recommendation                assessment             implemented implemented                     2003.                            in June 2004.    program.                         management       

                                    Page 35
                 GAO-04-731R DOD Business Systems Modernization
                                                                                                                   DOD has not yet                                   
                                                                                                                   defined and                                       
                                                                                                                   implemented an                                    
                                                                                                                   approach for                                      
                                                                                                                   selecting and                                     
                                                                                                                   controlling                                       
                                                                                                                   business systems                                  
                                                                                                                   investments. In                                   
                                                                                                                   March 2004, the                                   
                                                                                                                   Deputy Secretary                                  
                                                                                                                   of Defense signed                                 
                                                                                                                   the IT portfolio                                  
                                                                                                                   management policy                                 
                                                                                                                   and assigned                                      
                                                                                                                   responsibility to                                 
                                                                                                                   the domains for                                   
                                                                                              (4) The              IT portfolio                          
                                                                                              Secretary gain       management.                           
                                                                                              control over         However, the                          
                                                                                              ongoing IT           procedures to be                      
                                                                                              investments by       followed to                           
                                                                                              establishing a       implement the                         DOD has not
                                                                                              hierarchy of         policy are still                      developed
                                                                                              investment           under development                     standard
                                                                                              review boards,       and no time                           criteria
                                                                                              each responsible     frames for                            for
                                                                                              and accountable      completion have                       evaluating
                                                                                              for selecting        been provided.                        business
                 Status of                                                                    and controlling      According to the                      systems
               recommendation            Implemented                                          investments that   X IT portfolio                        X investments
                                                                                              meet defined         management                            and has not
                                                                                              threshold            policy, the                           established
                                                                                              criteria, and        department has                        time frames
                                                                                              each composed of     180 days (until                       for
                                                                                              the appropriate      mid-September     (5) The             completing
                                                                                              level of             2004) to develop  Secretary gain      this
                                                                                              executive            these procedures. control over        activity.
                                                                                              representatives,     In addition, the  ongoing IT          
                                                                                              depending on the     department has    investments by      
                                                                                              threshold            yet to formalize  establishing a      
                                                                                              criteria, from       specific roles    standard set of     
                                                                                              across the           and               criteria to         
                                                                                              department.          responsibilities  include (a)         
                                                                                                                   of the domains,   alignment and       
                                                                                                                   develop standard  consistency         
                                                                                                                   criteria for      with the DOD        
                                                                             function,                             performing the    enterprise          
                                                                             and  o  is                            system reviews,   architecture        
                                                                             not                                   and assign        and (b) our         
                                                                             performed by                          explicit          open                
                                                                             subject                               authority for     recommendations     
                                                                             matter                                fulfilling roles  governing           
                                                                             experts                               and               limitations in      
                                                                             involved in                           responsibilities. business            
                                                                             the                                   DOD has yet to    systems             
                                   DOD's                                     development                           establish time    investments         
                                comments                                     of key                                frames for        pending             
                                 and our             Partially   Not         architecture                          completing these  development of      
Recommendation                assessment             implemented implemented products.                             activities.       the                 

                                    Page 36
                 GAO-04-731R DOD Business Systems Modernization
                                                                                                                In March 2004,                                                 
                                                                                                                the Deputy                                                     
                                                                                                                Secretary of                                                   
                                                                                                                Defense signed                                                 
                                                                                                                the IT portfolio                                               
                                                                                                                management policy                                              
                                                                                                                and assigned                                                   
                                                                                                                responsibility to                                              
                                                                                                                the domains for                                                
                                                                                                                IT portfolio                                                   
                                                                                                                management.                                                    
                                                                                                                However, the                                                   
                                                                                                                procedures to be                                               
                                                                                                                followed to                                                    
                                                                                                                implement the                                                  
                                                                                                                policy are still                                               
                                                                                               (6) The          under development                                              
                                                                                               Secretary        and no time                                                    
                                                                                               gain control     frames for                                                     
                                                                                               over ongoing     completion have                                   As discussed 
                                                                                               IT               been provided.                                    in this      
                                                                                               investments      According to the                                  report, DOD  
                                                                                               by directing     IT portfolio                                      has not yet  
                                                                                               these boards     management                                        defined and  
                                                                                               to               policy, the                                       implemented  
                                                                                               immediately      department has                                    an effective 
                                                                                               apply these      180 days (until                                   investment   
                                                                                               criteria in      mid-September                                     management   
                                                                                               completing       2004) to develop                                  process to   
                                                                                               reviews of       these procedures.                                 proactively  
                 Status of                                                                     all ongoing      In addition, the                 (1) The          identify and 
               recommendation            Implemented                         architecture.     IT             X department has                   Secretary of   X control      
                                                                                               investments,     yet to formalize                 Defense or       system       
                                                                                               and to not       specific roles                   his              improvements 
                                                                                               fund             and                              appropriate      exceeding $1 
                                                                                               investments      responsibilities                 designee         million in   
                                                                                               that do not      of the domains,                  define and       obligations. 
                                                                                               meet these       develop standard                 implement an     DOD          
                                                                                               criteria         criteria for                     effective        officials    
                                                                                               unless they      performing the                   investment       have         
                                                                                               are              system reviews,                  management       acknowledged 
                                                                                               otherwise        and assign                       process to       that the     
                                                                                               justified by     explicit                         proactively      department   
                                                                                               explicit         authority for                    identify,        does not     
                                                                                               criteria         fulfilling roles                 control, and     have a
                                                                                               waivers.         and                              obtain DOD       systematic
                                                                                                                responsibilities.                Comptroller      means to
                                                                                                                DOD has yet to                   review and       identify and
                                                                                                                establish time                   approval of      determine
                                                                                                                frames for          GAO-03-1018: expenditures     which system
                                                                                                                completing these    DOD Business for new and      improvements
                                                                                                                activities.              Systems ongoing          should be
                                                                                                                Further, the      Modernization: business         submitted to
                                                                                                                policy also does       Important systems          the
                                                                                                                not address the    Progress Made investments      Comptroller
                                                                                                                issuance of           to Develop exceeding $1     for review
                                                                                                                waivers in those        Business million          and, are
                                                                                                                instances when        Enterprise while the        dependent
                                                                                                                exceptions to the  Architecture, architecture     upon system
                                   DOD's                                                                        BEA are justified  but Much Work is being         owners
                                comments                                                                        on the basis of         Remains. developed        coming
                                 and our             Partially   Not                                            documented         September 19, and after it     forward to
Recommendation                assessment             implemented implemented                                    analysis.                  2003. is               the

                                    Page 37
                 GAO-04-731R DOD Business Systems Modernization
                                                                                                                             DOD has taken some  
                                                                                                                             actions, but these  
                                                                                                                             actions are not     
                                                                                                                             sufficient to fully 
                                                                                                                             address our         
                                                                                                                             previous concerns.  
                                                                                                                             For example, DOD    
                                                                                                                             has  o  established 
                                                                                                                             the Domain Owners   
                                                                                                                             Integration Team;   
                                                                                                                             o  begun to         
                                                                                                                             establish a         
                                                                                                                             configuration       
                                                                                                            (2) The          management process; 
                                                                                                            Secretary of      o  recently issued 
                                                                                                            Defense or       an IT portfolio     
                                                                                                            his              management policy   
                                                                                                            appropriate      governing the       
                                                                                                            designee         development,        
                                                                                                            implement        maintenance, and    
                                                                                                            the core         implementation of   
                                                                                                            elements in      the BEA; and  o     
                                                                                                            our              developed           
                                                                                                            Enterprise       high-level          
                                                                                                            Architecture     performance         
                                                                                                            Framework        measures that are   
                                                                                                            for              to be used to       
                                                                                                            Assessing        develop the more    
                                                                                                            and              specific            
                                                                             completed, and                 Improving        resultsoriented     
                                                                             which includes                 Enterprise       performance metrics 
                                                                             clearly defined                Architecture     that are needed to  
                                                                             domain owners'         domains Management       enable it to        
                 Status of                                                   roles and                  and that we          evaluate program    
               recommendation            Implemented                         responsibilities    requesting identify in   X  progress and        
                                                                             for selecting        approval. this report      benefits. However,  
                                                                             and controlling                as not           as discussed        
                                                                             ongoing and                    satisfied,       previously,         
                                                                             planned system                 including        accountability and  
                                                                             investments.                   ensuring         responsibility have 
                                                                                                            that minutes     not been assigned   
                                                                                                            of the           to a committee or   
                                                                                                            meetings of      group comprised of  
                                                                                                            the              representatives     
                                                                                                            executive        from across the DOD 
                                                                                                            body charged     component           
                                                                                                            with             organizations-which 
                                                                                                            directing,       would be            
                                                                                                            overseeing,      responsible for     
                                                                                                            and              directing,          
                                                                                                            approving        overseeing, and     
                                                                                                            the              approving the BEA.  
                                                                                                            architecture     DOD plans to update 
                                                                                                            are prepared     the executive and   
                                                                                                            and              steering            
                                                                                                            maintained.      committees'         
                                                                                                                             charters to include 
                                                                                                                             these               
                                                                                                                             responsibilities in 
                                                                                                                             June 2004. Also,    
                                                                                                                             the configuration   
                                                                                                                             control board has   
                                                                                                                             been tasked with    
                                                                                                                             reviewing changes   
                                                                                                                             to only some, not   
                                   DOD's                                                                                     all BEA products    
                                comments                                                                                     (e.g., the board is 
                                 and our             Partially   Not                                                         not responsible for 
Recommendation                assessment             implemented implemented                                                 tracking            

                                    Page 38
                 GAO-04-731R DOD Business Systems Modernization
                                                                                 changes that                                                    
                                                                                 are being made                                                  
                                                                                 to the                                                          
                                                                                 transition                                                      
                                                                                 plan as part                                                    
                                                                                 of the BEA                                                      
                                                                                 effort). In                                                     
                                                                                 addition, both                                                  
                                                                                 the charter                                                     
                                                                                 and the                                                         
                                                                                 procedures                                                      
                                                                                 governing the                                                   
                                                                                 configuration                                                   
                                                                                 management                                                      
                                                                                 process are                                                     
                                                                                 still in                                                        
                                                                                 draft.                                                          
                                                                                 According to                                                    
                                                                                 DOD, the                                                        
                                                                                 charter is to                                                   
                                                                                 be approved in                                                  
                                                                                 June 2004, but      (3) The                                     
                                                                                 no time frame  Secretary of      (4) The                        
                                                                                 has been         Defense or      Secretary of                   
                                                                                 provided for            his      Defense or                     
                                                                                 final approval  appropriate      his                            
                                                                                 of the             designee      appropriate                    
                                                                                 procedures.          update      designee                       
                                                                                 The portfolio   version 1.0      update                         
                                                                                 management           of the      version 1.0                    
                                                                                 policy does    architecture      of the                         
                                                                                 not address      to include      architecture                   
                 Status of                                                       accountability      the 340      to include                     
               recommendation            Implemented                             for and               Joint X    the 29 key      X Of the 29    
                                                                                 approval of       Financial      elements          elements,
                                                                                 updates to the   Management      governing the     DOD stated
                                                                                 BEA-as called   Improvement      "As Is"           that it
                                                                                 for by best         Program      architectural     plans to
                                                                                 practices. In  requirements      content that      address 26,
                                                                                 addition, the      that our      our report        but it did
                                                                                 policy does          report      identified as     not have
                                                                                 not address      identified      not being         detailed
                                                                                 the issuance     as omitted      fully             plans on how
                                                                                 of waivers in  or not fully      satisfied.        and when
                                                                                 those            addressed.                        they would
                                                                                 instances when                                     be fully
                                                                                 exceptions to                                      addressed.
                                                                                 the BEA are                                        According to
                                                                                 justified on                                       DOD, it is
                                                                                 the basis of                                       currently
                                                                                 documented                                         addressing 2
                                                                                 analysis. The                                      of the
                                                                                 department's                                       26-system
                                                                                 current                                            and process
                                                                                 verification                                       inventories-
                                                                                 and validation                                     which will
                                                                                 contractor is                                      be part of
                                                                                 not                                                the
                                                                                 independent                                        transition
                                                                                 and is                                             plan. With
                                                                                 responsible                                        respect to
                                                                                 for reviewing                                      the system
                                                                                 the BEA                                            inventory,
                                   DOD's                                         products and                                       DOD's
                                comments                                         only selected                                      Comptroller
                                 and our             Partially   Not             program                                            recently
Recommendation                assessment             implemented implemented     deliverables.                                      testified

                                    Page 39
                 GAO-04-731R DOD Business Systems Modernization
                                                                                                                 According                                                                         
                                                                                                                 to DOD, it                                                                        
                                                                                                                 has                                                                               
                                                                                 that the                        partially                                                                         
                                                                                 actual number                   addressed                                                                         
                                                                                 of DOD                          21                                                                                
                                                                                 systems could                   elements,                                                                         
                                                                                 be twice as                     but it                                                                            
                                                                                 many as is    (5) The           plans to                                                                          
                                                                                 currently     Secretary of      address all                                                                       
                                                                                 reported;     Defense or        30.                                                                               
                                                                                 2,274. For    his               However,                                                                          
                                                                                 the 3         appropriate       DOD did not                                                                       
                                                                                 elements that designee          provide any                                                      According to     
                                                                                 DOD did not   update            detailed                                                         DOD, the next    
                                                                                 consider to   version 1.0       plans                           DOD did not                      transition plan
                                                                                 be relevant   of the            showing how (6) The             have                             will be
                                                                                 to the        business          and when it Secretary of        information     (7) The          available in
                                                                                 program, it   enterprise        would fully Defense or his      readily         Secretary of     August 2004.
                 Status of                                                       did not       architecture      address     appropriate         available to    Defense or       However, DOD did
               recommendation            Implemented                             provide any   to include      X these       designee update   X show that it    his            X not have
                                                                                 explicit      the 30 key        elements.   version 1.0 to      had addressed   appropriate      information
                                                                                 analysis as   elements          In          ensure that "To     inconsistencies designee         readily
                                                                                 to why the 3  governing the     addition,   Be"                 identified in   update           available to
                                                                                 would be      "To Be"           the         architecture        our report.     version 1.0      show how or when
                                                                                 irrelevant.   architectural     information artifacts are                       of the           it would fully
                                                                                 Until DOD     content that      provided    internally                          architecture     address our
                                                                                 provides such our report        for the 21  consistent, to                      to include       recommendations.
                                                                                 an analysis,  identified as     elements    include                             (a) the 3        
                                                                                 we consider   not being         shows that  addressing the                      key elements     
                                                                                 all 29        fully             it had not  inconsistencies                     governing        
                                                                                 elements to   satisfied.        addressed   described in                        the              
                                                                                 be critical                     them but    this report, as                     transition       
                                                                                 for the "As                     was         well as                             plan content     
                                                                                 Is"                             planning to including user                      that our         
                                                                                 architectural                   address     instructions or                     report           
                                                                                 content.                        them or had guidance for                        identified       
                                   DOD's                                                                         just begun  easier                              as not being     
                                comments                                                                         to take     architecture                        fully            
                                 and our             Partially   Not                                             needed      navigation and                      satisfied        
Recommendation                assessment             implemented implemented                                     actions.    use.                                and (b)          

                                    Page 40
                 GAO-04-731R DOD Business Systems Modernization
                                                                                                                According to                                                               
                                                                                                                DOD, it has                                                                
                                                                                                                addressed                                                                  
                                                                                                                some of its                                                                
                                                                                                                verification                                                               
                                                                                                                and                                                                        
                                                                                                                validation                                                                 
                                                                                                                contractor's                                                               
                                                                                                                comments and                                                               
                                                                                                                plans to                                                                   
                                                                                                                address the                                                                
                                                                                                                majority of                        As                          The         
                                                                                                                the others.   (9) The              discussed                   accounting  
                                                                                               (8) The          However, DOD  Secretary of         in this                     and finance 
                                                                                               Secretary of     did not have  Defense or his       report, DOD    (10) The     domain has  
                                                                                               Defense or       documentation appropriate          has not       Secretary     two ongoing 
                                                                                               his              readily       designee develop     developed    of Defense     pilot       
                                                                                               appropriate      available to  a well-defined       welldefined      or his     initiatives 
                                                                                               designee         show that it  near-term plan       near-term   appropriate     to acquire  
                                                                                               update           had addressed for extending        or             designee     and         
                 Status of                                                                     version 1.0      some of the   and evolving the     long-term     limit the     implement   
               recommendation            Implemented                                           of the         X contractor's  architecture and   X plans to be       pilot   X production
                                                                                               architecture     comments, and ensure that this     used to     projects to     accounting
                                                                                               to address       was unable to plan includes        guide            small,     systems by
                                                                                               comments         provide plans addressing our       dayto-day     low-cost,     the end of
                                                                                               made by the      detailing how recommendations,     program         lowrisk     2005-at an
                                                                                               verification     and when it   defining roles       activities    prototype     estimated
                                                                                               and              would fully   and                  to enable   investments     cost of
                                                                                               validation       address the   responsibilities     it to          that are     $135
                                                                                               contractor.      remaining     of all               evaluate                    million.
                                                                                                                comments.     stakeholders         its                         This domain
                                                                              those system                      According to  involved in          progress.                   is
                                                                               investments                      DOD, the      extending and                                    
                                                                             that will not                      verification  evolving the                                     
                                                                               become part                      and           architecture,                                    
                                                                                of the "To                      validation    explaining                                       
                                                                                       Be"                      contractor's  dependencies                                     
                                                                             architecture,                      comments      among planned                                    
                                                                                 including                      would be      activities, and                                  
                                   DOD's                                       time frames                      addressed in  defining                                         
                                comments                                       for phasing                      BEA updates   measures of                                      
                                 and our             Partially   Not             out those                      after version activity                                         
Recommendation                assessment             implemented implemented      systems.                      2.0.          progress.                                        

                          Status of recommendation                            
                     Implemented Partially   Not           DOD's comments and 
      Recommendation             implemented implemented       our assessment
                                                         also currently       
                                                         conducting workshops 
                                                         to develop needed    
                                                         business rules and   
                                                         requirements for     
                                                         extending and        
         intended to                                     evolving the BEA in  
provide knowledge                                     support of DOD's     
    needed to extend                                     achieving objective  
      and evolve the                                     of increment one,    
architecture, and                                     which includes       
          are not to                                     achieving an         
         acquire and                                     unqualified audit    
           implement                                     opinion on DOD's     
          production                                     fiscal year 2007     
      version system                                     consolidated         
     solutions or to                                     financial            
           deploy an                                     statements. However, 
         operational                                     the department did   
              system                                     not provide detailed 
         capability.                                     plans that showed    
                                                         how or if the        
                                                         workshop results     
                                                         would serve as the   
                                                         basis for acquiring  
                                                         these production     
                                                         systems.             

Source: GAO analysis of DOD data.

a DOD defines the Global Information Grid as the globally interconnected,
end-to-end set of information, capabilities, associated processes, and
personnel for collecting, processing, storing, disseminating, and managing
information on demand to warfighters, policymakers, and support personnel.

b In May 2003, the DOD Comptroller changed the architecture name from the
Financial Management Enterprise Architecture to the Business Enterprise
Architecture to reflect the transformation of departmentwide business
operations and supporting systems, including accounting and finance,
budget formulation, acquisition, inventory management, logistics,
personnel, and property management systems.

c In September 2003, DOD established the Domain Owners Integration Team,
comprised of domain representatives, to facilitate communication and
coordination across the domains.

d DOD has six departmental domains, which are (1) accounting and finance,
(2) acquisition, (3) human resources management, (4) installations and
environment, (5) logistics, and (6) strategic planning and budgeting. It
also has one enterprise information environment mission area. The domains
and the mission area, comprised of the Under Secretaries of Defense and
the Assistant Secretary of Defense for Networks and Information
Integration/DOD Chief Information Officer, have authority, responsibility,
and accountability within their domains for business transformation,
implementation of the BEA, development and execution of the transition
plan, portfolio management, and establishment of a structure to ensure
representation of the DOD components and the appropriate federal agencies.

Enclosure IV

Comments from the Department of Defense

                                 See comment 2.

                                 See comment 1.

                         See comment 1. See comment 1.

                  See comment 1. See comment 3. See comment 1.

                                 See comment 4.

                         See comment 5. See comment 3.

                                 See comment 1.

                                 See comment 6.

                  See comment 7. See comment 8. See comment 1.

                                 See comment 9.

                                 See comment 1.

                                 See comment 1.

                         See comment 1. See comment 10.

                                See comment 11.

                        See comment 12. See comment 13.

                        See comment 13. See comment 14.

The following are GAO's comments on the Department of Defense's (DOD)
letter dated May 10, 2004.

GAO Comments

1. See the "Agency Comments and Our Evaluation" section of this report.

2. 	We have reviewed multiple versions of the architecture development
methodology, including the latest version, which is currently draft. Based
on our review of the latest draft, we concluded that this methodology does
not provide a documented approach for performing activities in a coherent,
consistent, accountable, and repeatable manner. Key DOD stakeholders also
expressed concerns about the methodology. For example, these stakeholders
stated that it is an "after-the-fact" description of work products rather
than a prescriptive document that details the approach to be followed to
develop the architecture.

3. 	The descriptions of increments 2 and 3 in DOD's comment letter are
different than the descriptions the program manager provided to us on
April 27, 2004, as reflected in this report. Because of the
inconsistencies in the responses, it remains unclear what the department's
focus is for increments 2 and 3. DOD's response also stated that the
executive committee approved all three increments; however, DOD did not
provide evidence of this approval.

4. 	As stated in our report, DOD's objective for achieving an unqualified
audit opinion is not supported by a DOD-wide plan of action nor are the
individual component plans linked to program activities.

5. 	As stated in our report, DOD does not expect to complete the next
version of its transition plan until August 2004. In addition, as we
previously reported,42 its initial transition plan was basically a plan to
develop a transition plan and did not possess the attributes needed to
guide its transformation efforts.

6. 	Attachment 11 does not list the deliverables the verification and
validation contractor has reviewed since April 2003.

7. 	The matrix does not show the anticipated disposition (e.g., date and
architecture version) of the verification and validation contractor's
comments.

8. This information was provided after fieldwork was completed.

9. 	These briefings were presentations given by each domain describing its
approach for conducting portfolio management reviews. The steering

                                42 GAO-03-1018.

committee did not review actual systems investments nor was there detailed
discussion about the domains' approaches.

10. The scope of our review did not include an evaluation of the
realignment of funds made by the logistics domain to determine an
effective demonstration of portfolio management.

11. DOD's characterization of its current process for oversight and
control of modernization investments is inaccurate. We along with the DOD
Inspector General continue to report on significant weaknesses in the
department's requirements, acquisition, and budgeting processes for
business systems modernization projects. For example, we reported43 that
estimated costs for the Defense Procurement Payment System had increased
by $274 million and the schedule had slipped by almost 4 years after
having already invested 7 years and over $126 million. DOD Comptroller's
noted that the project was being terminated due to poor program
performance and increased costs.

12. We look forward to receiving DOD's forthcoming guidance to contracting
and program officials that describes the process by which these officials
obtain DOD Comptroller's statutorily required determination before making
obligations exceeding $1 million for a financial system improvement.
However, given the obligation data provided to us by DOD and included in
our report, we are concerned that DOD plans no action to review previous
obligations for system improvements to ensure that there have been no such
violations since the enactment of the limitation.

13. We continue to believe that contractor tasks have not been clearly
defined and that, because DOD is acquiring services on the basis of time
and materials, it is essential that the department use efficient methods
and effective cost controls to measure the work being performed. While we
did not take issue with DOD on its use of a time and materials contract,
we did state that DOD's failure to execute the required Determination and
Finding to evaluate the cost and performance risks assumed by the
government illustrates an overall lack of clear architecture development
plans.

14. According to a program official, DOD did not begin requiring
acceptance criteria prior to the contractor beginning work until August
2003. Further, DOD has paid the contractor under the time and materials
contract even when the contractor did not meet all of the criteria for the
deliverable. For example, based on reports provided by DOD, the department
paid the contractor for delivery of version 2.0 of the architecture even
though this version did not meet 50 percent of the acceptance criteria.

                                 43 GAO-03-465.

Enclosure V

GAO Contacts and Staff Acknowledgments

GAO Contacts 	Jenniffer Wilson, (202) 512-9192 Cynthia Jackson, (202)
512-5086

Acknowledgments 	In addition to the individuals named above, key
contributors to this report included Beatrice Alff, Francine DelVecchio,
Abe Dymond, Joanne Fiorino, Neelaxi Lakhmani, J. Christopher Martin, Mai
Nguyen, Michael Peacock, David Plocher, Katherine Schirano, Darby Smith,
and Bernard Trescavage.

(192114)

                                 GAO's Mission

Obtaining Copies of GAO Reports and Testimony

The General Accounting Office, the audit, evaluation and investigative arm
of Congress, exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability of
the federal government for the American people. GAO examines the use of
public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents at no cost
is through the Internet. GAO's Web site (www.gao.gov) contains abstracts
and fulltext files of current reports and testimony and an expanding
archive of older products. The Web site features a search engine to help
you locate documents using key words and phrases. You can print these
documents in their entirety, including charts and other graphics.

Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as "Today's Reports," on its
Web site daily. The list contains links to the full-text document files.
To have GAO e-mail this list to you every afternoon, go to www.gao.gov and
select "Subscribe to e-mail alerts" under the "Order GAO Products"
heading.

Order by Mail or Phone 	The first copy of each printed report is free.
Additional copies are $2 each. A check or money order should be made out
to the Superintendent of Documents. GAO also accepts VISA and Mastercard.
Orders for 100 or more copies mailed to a single address are discounted 25
percent. Orders should be sent to:

U.S. General Accounting Office 441 G Street NW, Room LM Washington, D.C.
20548

To order by Phone: 	Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061

To Report Fraud,	Contact: Web site: www.gao.gov/fraudnet/fraudnet.htm

                  Waste, and Abuse in E-mail: [email protected]

Federal Programs Automated answering system: (800) 424-5454 or (202)
512-7470

Jeff Nelligan, Managing Director, [email protected] (202) 512-4800

Public Affairs 	U.S. General Accounting Office, 441 G Street NW, Room 7149
Washington, D.C. 20548

This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
*** End of document. ***