District of Columbia: Software Acquisition Processes for a New Financial
Management System (Letter Report, 04/30/98, GAO/AIMD-98-88).

Pursuant to a congressional request, GAO reviewed whether the District
of Columbia had implemented disciplined software acquisition processes
for its new financial management system (FMS).

GAO noted that: (1) while the district has many strengths in its
acquisition processes for FMS, it also has many weaknesses; (2) when
compared to standards established by the Software Engineering Institute,
the District's processes for software acquisitions are not mature; (3)
of the six key process areas (KPA) evaluated for the repeatable level,
the District fully satisfied only one--solicitation; (4) severe
weaknesses were found in other critical key processes, including
requirements development and management and evaluation; (5) for example,
the District does not have a policy for establishing and managing
software-related requirements, does not at present have adequate
resources for requirements development, and has not formally designated
responsibility for requirements development and management; (6)
similarly, the District does not have an effective evaluation process,
and is currently unable to objectively determine if the acquired systems
will satisfy the contract requirement; (7) finally, the District has not
satisfied the one KPA evaluated for the defined level of maturity,
acquisition risk management; and (8) the FMS project does not have a
risk management plan and does not track project risk.

--------------------------- Indexing Terms -----------------------------

 REPORTNUM:  AIMD-98-88
     TITLE:  District of Columbia: Software Acquisition Processes for a 
             New Financial Management System
      DATE:  04/30/98
   SUBJECT:  Computer software
             Financial management systems
             Requirements definition
             Computer software verification and validation
             Strategic information systems planning
             Municipal governments
             Financial management systems
IDENTIFIER:  District of Columbia
             DC Financial Management System
             Software Capability Maturity Model
             CMM
             
******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO report.  Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved.  Major          **
** divisions and subdivisions of the text, such as Chapters,    **
** Sections, and Appendixes, are identified by double and       **
** single lines.  The numbers on the right end of these lines   **
** indicate the position of each of the subsections in the      **
** document outline.  These numbers do NOT correspond with the  **
** page numbers of the printed product.                         **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
** A printed copy of this report may be obtained from the GAO   **
** Document Distribution Center.  For further details, please   **
** send an e-mail message to:                                   **
**                                                              **
**                                            **
**                                                              **
** with the message 'info' in the body.                         **
******************************************************************


Cover
================================================================ COVER


Report to the Chairman, Subcommittee on the District of Columbia,
Committee on Appropriations, House of Representatives

April 1998

DISTRICT OF COLUMBIA - SOFTWARE
ACQUISITION PROCESSES FOR A NEW
FINANCIAL MANAGEMENT SYSTEM

GAO/AIMD-98-88

D.C.  Financial Management System

(916247)


Abbreviations
=============================================================== ABBREV

  FMS - financial management system
  KPA - key process area
  SA-CMM - Software Acquisition Capability Maturity Model
  SCE - Software Capability Evaluation
  SEI - Software Engineering Institute

Letter
=============================================================== LETTER


B-279266

April 30, 1998

The Honorable Charles H.  Taylor
Chairman, Subcommittee on the
 District of Columbia
Committee on Appropriations
House of Representatives

Dear Mr.  Chairman: 

This report responds to your request that we determine whether the
District of Columbia had implemented disciplined software acquisition
processes for its new financial management system (FMS).  We found
that the District's software acquisition processes for the FMS
acquisition, while having some strengths, are not mature when
compared to standards established by the Software Engineering
Institute (SEI).\1 Accordingly, we are making recommendations for
strengthening these processes as they relate to the FMS project and
to improve any future software acquisitions. 


--------------------
\1 Carnegie Mellon University's Software Engineering Institute (SEI),
recognized for its expertise in software processes, has developed
models and methods that define and determine organizations' software
process maturity. 


   BACKGROUND
------------------------------------------------------------ Letter :1

In September 1997, the District of Columbia Financial Responsibility
and Management Assistance Authority (Authority) awarded a contract to
acquire a new FMS.  The overall objective of the FMS project is to
improve the District's financial systems through faster, more
efficient, and accurate processing providing increased functionality,
flexibility, and reduced cost of operations.  According to the Chair
of the Authority, the new FMS is intended to (1) eliminate the
principal problems that exist with the current system and ensure that
all financial management guidelines are adhered to, (2) enable
managers to more effectively and efficiently monitor and control
financial resources, and (3) produce timely, accurate, and reliable
information, providing decisionmakers with the basic financial
information needed to make more informed decisions. 

The Authority awarded a contract for the new FMS in September 1997
and committed to an aggressive implementation schedule.  The schedule
anticipates (1) pilots in five agencies beginning in February 1998,
(2) the accounting system to be implemented by October 1998, and (3)
District-wide implementation by February 1999. 


   OBJECTIVE, SCOPE, AND
   METHODOLOGY
------------------------------------------------------------ Letter :2

We were asked to review the District's efforts to acquire a new
financial management system.  Our objective was to determine whether
the District had implemented disciplined software acquisition
processes for acquiring its new financial management system. 

To accomplish this, we applied the Software Engineering Institute's
Software Acquisition Capability Maturity Model (SA-CMM) and its
Software Capability Evaluation (SCE) method.  SEI's expertise in, and
methods for, software process assessment are recognized and accepted
throughout the industry.  Our evaluators were all SEI-trained
software specialists. 

SA-CMM ranks organizational maturity according to five levels (see
figure 1).  Maturity levels 2 through 5 require the verifiable
existence and use of certain software acquisition processes, known as
key process areas (KPA).  According to SEI, an agency that has these
acquisition processes in place is in a much better position to
successfully acquire software than an organization that does not have
these processes in place.  We evaluated the District's software
acquisition processes against six of the seven level 2 KPAs (the
transition to support KPA was not evaluated because the District does
not plan to support FMS in-house) and one level 3 KPA (see table 1). 
We selected level 2 because it is the minimum level at which any
assurance exists that software acquisition processes are mature
enough to consistently deliver promised software capabilities on time
and within budget.  We included one level 3 KPA--acquisition risk
management--because it is considered by software experts to be a very
important process area. 

   Figure 1:  SA-CMM Levels and
   Descriptions

   (See figure in printed
   edition.)



                                Table 1
                
                   SA-CMM KPAs Used to Assess the FMS
                 Project Software Acquisition Maturity

SA-CMM level 2 key
process areas           Description
----------------------  ----------------------------------------------
Software acquisition    Ensuring that reasonable planning for the
planning                software acquisition is conducted and that all
                        elements of the project are included.

Solicitation            Ensuring that award is made to the contractor
                        most capable of satisfying the specified
                        requirements.

Requirements            Establishing a common and unambiguous
development and         definition of software acquisition
management              requirements understood by the acquisition
                        team, system user, and the contractor.

Project management      Managing the activities of the project office
                        and supporting contractor(s) to ensure a
                        timely, efficient, and effective software
                        acquisition.

Contract tracking and   Ensuring that the software activities under
oversight               contract are being performed in accordance
                        with contract requirements, and that products
                        and services will satisfy contract
                        requirements.

Evaluation              Determining that the acquired software
                        products and services satisfy contract
                        requirements prior to acceptance and
                        transition to support.

SA-CMM level 3 key
process area            Description

Acquisition risk        Identifying risks as early as possible,
management              adjusting acquisition strategy to mitigate
                        those risks, and developing and implementing a
                        risk management process as an integral part of
                        the acquisition process.
----------------------------------------------------------------------
As established by the model, each KPA contains five common attributes
that indicate whether the implementation and institutionalization of
a KPA can be effective, repeatable, and lasting.  The five common
attributes are: 

Commitment to perform.  Commitment to perform describes the actions
that the organization must take to establish the process and ensure
that it can endure.  Commitment to perform typically involves
establishing organizational policies and sponsorship.  Key practices
under commitment to perform include having written organization
policy for the process area and designating and assigning
responsibility to individuals to perform the activities. 

Ability to perform.  Ability to perform describes the preconditions
that must exist in the project or organization to implement the
software acquisition process competently.  Ability to perform
typically involves resources, organizational structures, and
training.  Key practices under ability to perform include having
either experienced or trained personnel and providing adequate
resources to conduct the activities for the process area. 

Activities performed.  Activities performed describes the roles and
procedures necessary to implement a KPA.  Activities performed
typically involve establishing plans and procedures, performing the
work, tracking it, and taking appropriate management actions.  Key
practices under activities performed include performing these
activities in accordance with documented plans and overall management
of the project team and process area. 

Measurement and analysis.  Measurement and analysis describes
activities performed to measure the process and analyze the
measurements.  Measurement and analysis typically includes defining
the measurements to be taken and the analyses to be conducted to
determine the status and effectiveness of the activities performed. 

Verifying implementation.  Verifying implementation describes the
steps to ensure that the activities are performed in compliance with
the process that has been established.  There are two key practices
under verifying implementation, including reviews by various levels
of management. 

In accordance with SEI's methodology, for each KPA selected, we
evaluated the District's policies and practices.  This
project-specific comparison can result in one of four possible
outcomes:  (1) project strength--an effective implementation of the
key practice, (2) project weakness--ineffective implementation of a
key practice or failure to implement a key practice, (3) project
observation--key practice evaluated but evidence inconclusive and
cannot be characterized as either a strength or weakness, and (4) not
rated--key practice not currently relevant to project, therefore not
evaluated. 

As requested, we applied the SEI methodology to evaluate the
District's management of only one project--the acquisition of a new
financial management system.  As a result, we have no information
regarding the process strengths and weaknesses of other acquisitions
the District may have underway. 

We performed our work at District of Columbia offices in Washington,
D.C.  between September 16, 1997, and January 16, 1998, in accordance
with generally accepted government auditing standards. 


   RESULTS IN BRIEF
------------------------------------------------------------ Letter :3

While the District has many strengths in its acquisition processes
for FMS, it also has many weaknesses.  When compared to standards
established by the SEI, the District's processes for software
acquisition are not mature.  Of the six KPAs evaluated for the
repeatable level, the District fully satisfied only
one--solicitation.  Severe weaknesses were found in other critical
key processes, including requirements development and management and
evaluation.  For example, the District does not have a policy for
establishing and managing software-related requirements, does not, at
present, have adequate resources for requirements development, and
has not formally designated responsibility for requirements
development and management. 

Similarly, the District does not have an effective evaluation
process, and is currently unable to objectively determine if the
acquired system will satisfy the contract requirements.  Finally, the
District has not satisfied the one key process area evaluated for the
"defined" level of maturity, acquisition risk management.  The FMS
project does not have a risk management plan and does not track
project risk.  Figure 2 provides a comprehensive listing of the FMS
project's strengths, weaknesses, and observations for the seven KPAs. 
The items under features in figure 2 refer to the five common
attributes in each KPA and are explained in detail in the following
sections. 

   Figure 2:  Key Process Area
   Results

   (See figure in printed
   edition.)


   SOFTWARE ACQUISITION PLANNING
------------------------------------------------------------ Letter :4

The purpose of software acquisition planning is to ensure that
reasonable planning for the software acquisition is conducted and
that all aspects of the total software acquisition effort are
included in these plans at the proper level of detail.  The software
acquisition planning process, among other things, includes (1)
addressing software life-cycle support in acquisition plans, (2)
preparing life-cycle software cost estimates, (3) having a written
software acquisition policy, (4) measuring and reporting on the
status of software acquisition planning activities, and (5) having
guidance on software training and experience requirements for project
personnel. 


      FMS PROJECT PERFORMS SOME
      BUT NOT ALL SOFTWARE
      ACQUISITION PLANNING
      PRACTICES
---------------------------------------------------------- Letter :4.1

The FMS project had many strengths in this KPA.  The District
received pro bono assistance from several companies to help define
the acquisition strategy and conduct the activities for software
acquisition planning.  An acquisition strategy was developed and the
acquisition planning team was staffed with personnel with software
and systems experience.  The team developed a cost estimate and the
District management was briefed by the team on a periodic basis. 
This enabled the District management to be informed on the progress
of the acquisition planning and the various activities through the
solicitation phase. 

However, the FMS project also had many weaknesses in this KPA. 
Weaknesses observed included a lack of policy on acquisition planning
and no specific assignment of responsibility for acquisition
planning.  Furthermore, the FMS project did not always document
significant project decisions or update the planning document to
reflect these decisions.  For example, when the District decided to
not pursue a single contract to both acquire FMS and outsource data
center operations, the capability assessment (a software acquisition
planning document) was not updated to reflect this decision. 
Decisions should be documented and the planning documents updated to
ensure that large acquisitions such as FMS can be effectively
managed.  Table 2 shows the strengths and weaknesses for the software
acquisition planning KPA and the specific findings supporting these
ratings. 



                                     Table 2
                     
                      Software Acquisition Planning Findings

Feature       Key practice                 Finding                      Rating
------------  ---------------------------  ---------------------------  --------
Commitment 1  The acquisition              There is no written policy   Weakness
              organization has a written   for software acquisition
              policy for planning the      planning.
              software acquisition.

Commitment 2  Responsibility for software  Responsibility for software  Weakness
              acquisition planning         acquisition planning
              activities is designated.    activities was not
                                           designated.

Ability 1     The acquisition              The acquisition staff (10-   Strength
              organization has             15 Authority and D.C.
              experienced software         government staff, and 10-
              acquisition management       15 private sector employees
              personnel.                   working on a pro bono
                                           basis) had requisite
                                           experience.

Ability 2     Adequate resources are       Required resources for       Weakness
              provided for software        acquisition planning were
              acquisition planning         not determined.
              activities.

Activity 1    Software acquisition         The District government      Strength
              planning personnel are       recruited outside software
              involved in system           experts to assist with
              acquisition planning.        system acquisition
                                           activities.

Activity 2    The software acquisition     The software acquisition     Strength
              strategy for the project is  strategy is developed and
              developed and documented.    includes risk
                                           identification and
                                           schedules. It is documented
                                           in the capability
                                           assessment document.

Activity 3    The project's software       Program changes regarding    Weakness
              acquisition planning is      outsourcing of the data
              documented and the planning  center and upgrading the
              documentation is maintained  current system versus
              over the life of the         buying off the shelf were
              project.                     made, but the software
                                           acquisition planning
                                           documentation was not
                                           updated.

Activity 4    Life-cycle support of the    The software acquisition     Weakness
              software is included in      planning documentation does
              software acquisition         not address life cycle
              planning documentation.      support of the software.

Activity 5    Life-cycle cost and          Life-cycle cost and          Strength
              schedule estimates for the   schedule estimates were
              software products and        prepared by a consultant.
              services being acquired are  They were reviewed by the
              prepared and independently   District Chief Financial
              reviewed.                    Officer and Authority
                                           Executive Director.

Measurement   Measurements are made and    Measurements of software     Weakness
1             used to determine the        acquisition activities were
              status of the software       not taken.
              acquisition planning
              activities and resultant
              products.

Verification  Software acquisition         The Authority Executive      Strength
1             planning activities are      Director was briefed on
              reviewed by acquisition      planning activities on a
              organization management on   periodic basis.
              a periodic basis.

Verification  Software acquisition         Software acquisition         Strength
2             planning activities are      planning activities were
              reviewed by the project      reviewed by the project
              manager on both a periodic   manager.
              and event-driven basis.
--------------------------------------------------------------------------------

   SOLICITATION
------------------------------------------------------------ Letter :5

The purpose of solicitation is to prepare a request for proposal that
delineates a project's software-related requirements and select a
contractor that can most cost-effectively satisfy these requirements
while complying with relevant solicitation laws and regulations. 
Specific requirements for a solicitation process include, among other
things (1) having and following a solicitation plan, (2) assigning
responsibility and ensuring sufficient resources for coordinating and
conducting solicitation activities, (3) preparing and reviewing cost
and schedule estimates for the software products and services being
acquired, and (4) periodically measuring solicitation work completed
and effort and funds expended, comparing these measures to plans, and
reporting the results to management. 


      FMS PROJECT PERFORMING MOST
      SOLICITATION PRACTICES
---------------------------------------------------------- Letter :5.1

The FMS project exhibited many process strengths during the
solicitation.  The District has a policy on solicitation and the FMS
project followed this policy.  The project had experienced personnel
on the source selection team and these personnel briefed the team
members on the objectives of the solicitation.  However, the District
did not measure either time or funds expended to conduct the
solicitation.  Specifically, no evidence was provided to show that
the FMS project tracked personnel hours or costs during the conduct
of the solicitation.  Addressing this weakness would enable the
District to better estimate the resources needed to conduct similar
acquisitions in the future.  For example, if these data were
collected and made available to other projects, such as the tax
systems upgrade, the District would be in a better position to
understand its own capability to effectively conduct solicitation, to
estimate how long such a solicitation was likely to take, and to
eliminate problems that may have hampered the FMS solicitation. 
Table 3 shows the strengths, weaknesses, and observations for the
solicitation KPA and the specific findings supporting these ratings. 



                                     Table 3
                     
                              Solicitation Findings

Feature       Key practice                 Finding                      Rating
------------  ---------------------------  ---------------------------  --------
Commitment 1  The acquisition              Authority's procurement      Strength
              organization has a written   regulations provides
              policy for the conduct of    written policies for
              the software portion of the  solicitation.
              solicitation.

Commitment 2  Responsibility for the       Authority's Executive        Strength
              software portion of the      Director has designated
              solicitation is designated.  specific individuals to
                                           serve as members of a
                                           selection committee.

Commitment 3  A selection official has     Authority's Executive        Strength
              been designated to be        Director is designated as
              responsible for the          the source selection
              selection process and the    official.
              decision.

Ability 1     A group that is responsible  A solicitation organization  Strength
              for coordinating and         has been identified in the
              conducting solicitation      source selection plan.
              activities exists.

Ability 2     Adequate resources are       Adequate resources           Strength
              provided for solicitation    (including Authority staff
              activities.                  and a team of private
                                           sector experts) were
                                           provided for solicitation
                                           activities.

Ability 3     Individuals performing       Experienced individuals      Strength
              solicitation activities      from private contractor
              have experience or receive   staff and public sectors
              training.                    assisted in the
                                           solicitation activities.

Ability 4     The groups supporting the    Participants in source       Strength
              solicitation (e.g., end      selection process were
              user, systems engineering,   trained on the objectives
              software support             and procedures of FMS
              organization, and            solicitation.
              application domain experts)
              receive orientation on the
              solicitation's objectives
              and procedures.

Activity 1    The project team performs    Project team conducted its   Observat
              its activities in            activities according to the  ion
              accordance with its          source selection plan,
              documented solicitation      however, that plan does not
              plans.                       cover all solicitation
                                           tasks.

Activity 2    The project team performs    A number of subcommittees    Strength
              its activities in            evaluated proposals
              accordance with its          according to the process
              documented proposal          documented in the source
              evaluation plans.            selection plan.

Activity 3    Cost and schedule estimates  Capabilities assessment      Strength
              for the software products    report documents the cost
              and services being sought    and schedule estimates for
              are prepared.                the new financial
                                           management system. A
                                           preliminary economic
                                           analysis of FMS was also
                                           performed.

Activity 4    Software cost and schedule   Capabilities assessment      Strength
              estimates are independently  report and the economic
              reviewed for                 analysis of FMS were
              comprehensiveness and        reviewed by the D.C. Office
              realism.                     of the Inspector General
                                           and a report was issued.

Activity 5    The project team takes       The project team ensured     Strength
              action to ensure the mutual  mutual understanding of
              understanding of software    software requirements by
              requirements and plans       holding question and answer
              prior to contract award.     sessions with potential FMS
                                           vendors.

Measurement   Measurements are made and    No measurements were taken   Weakness
1             used to determine the        to determine the status of
              status of the solicitation   solicitation activities.
              activities and resultant
              products.

Verification  Solicitation activities are  Authority Executive          Strength
1             reviewed by the designated   Director and the District
              selection official or        Chief Financial Officer are
              acquisition organization     briefed on status of all
              management on a periodic     FMS activities (including
              basis.                       solicitation) on a periodic
                                           basis.

Verification  Solicitation activities are  During the solicitation,     Strength
2             reviewed by the project      the entire team reported to
              manager on both a periodic   the Executive Director who
              and event-driven basis.      served as the project
                                           manager. The Executive
                                           Director was briefed
                                           periodically and in
                                           response to significant
                                           events.
--------------------------------------------------------------------------------

   REQUIREMENTS DEVELOPMENT AND
   MANAGEMENT
------------------------------------------------------------ Letter :6

The purpose of requirements development and management is to
establish and maintain a common and unambiguous definition of
software requirements among the acquisition team, system users, and
software development contractor.  This KPA involves two subprocesses: 
(1) developing a baseline set of software-related contractual
requirements and (2) managing these requirements and changes to these
requirements for the duration of the acquisition. 

A number of requirements development and management practices are
necessary to satisfy this key process area.  These include (1) having
a written organizational policy for establishing and managing
requirements allocated to software, (2) documenting plans for the
development and management of requirements, (3) having documented
processes for requirements development, including elicitation,
analysis, and verification, (4) measuring and reporting on the status
of requirements development and management activities to management,
(5) appraising the impact on software of system-level requirements
changes and (6) having a mechanism to ensure that
contractor-delivered work products meet specified requirements. 


      FMS PROJECT PERFORMING SOME
      BUT NOT MOST REQUIREMENTS
      DEVELOPMENT AND MANAGEMENT
      PRACTICES
---------------------------------------------------------- Letter :6.1

The FMS project has some process strengths in the conduct of
requirements development and management.  The project team is
performing requirements management activities in accordance with its
documented plan and software-related contractual requirements have
been baselined.  In addition, District management periodically
reviews the status of requirements development and management
activities with the project team.  However, in acquiring FMS, the
District did not perform many of the requirements development and
management practices necessary to satisfy this KPA.  For example, the
District does not have an organizational policy for establishing and
managing software-related requirements, there is no clear assignment
of responsibility for requirements development and management and no
documented evidence exists to show either resource requirements or
resources expended for requirements development activities. 

Currently, the FMS project has begun to hold "requirements
confirmation meetings" with the users to validate the requirements
already specified in the FMS contract.  Although requirements should
be validated, this should have been done prior to releasing the
request for proposal to ensure that the proposal accurately reflects
the District's requirements.  Changing requirements after contract
award may adversely impact project cost, schedule, and/or
performance.  Table 4 shows the strengths, weaknesses, and
observations for the requirements development and management KPA and
the specific findings supporting these ratings. 



                                     Table 4
                     
                     Requirements Development and Management
                                     Findings

Feature       Key practice                 Finding                      Rating
------------  ---------------------------  ---------------------------  --------
Commitment 1  The acquisition              There is no organizational   Weakness
              organization has a written   policy for establishing and
              policy for establishing and  managing software-related
              managing the software-       requirements.
              related contractual
              requirements.

Commitment 2  Responsibility for           There is no formal           Weakness
              requirements development     documentation assigning
              and management is            anyone responsibility for
              designated.                  requirements development
                                           and management.

Ability 1     A group that is responsible  There is no documentation    Weakness
              for performing requirements  assigning any group
              development and management   responsibility for
              activities exists.           performing requirements
                                           development and management
                                           activities.

Ability 2     Adequate resources are       No documented evidence       Weakness
              provided for requirements    exists to show either
              development and management   resource requirements or
              activities.                  resources used for
                                           requirements development
                                           and management activities.

Ability 3     Individuals performing       Individuals currently        Observat
              requirements development     performing requirements      ion
              and management activities    management have appropriate
              have experience or receive   experience. However, no
              training.                    documented evidence was
                                           provided to show the
                                           experience of individuals
                                           who developed the
                                           requirements.

Activity 1    The project team performs    The requirements management  Strength
              its activities in            activities are incorporated
              accordance with its          in the project management
              documented requirements      plan, volumes I and II.
              development and management   This project team is
              plans.                       performing its activities
                                           in accordance with this
                                           plan.

Activity 2    The project team develops    The project team baselined   Strength
              and baselines the software-  software-related
              related contractual          contractual requirements in
              requirements and places      the statement of work prior
              them under change control    to the release of the
              early in the project, but    solicitation. The
              not later than release of    requirements are under
              the solicitation package.    change control.

Activity 3    The project team appraises   There have been no           Observat
              system requirements change   requirements changes to      ion
              requests for their impact    date to appraise. However,
              on the software being        a process to appraise
              acquired.                    changes exists and is
                                           documented in the program
                                           management plan.

Activity 4    The project team appraises   The scope change process     Observat
              all changes to the           documented in the project    ion
              software-related             management plan requires
              contractual requirements     the appraisal of cost and
              for their impact on          schedule impact. However,
              performance, architecture,   no evidence provided of any
              supportability, system       changes to the contract to
              resource utilization, and    date.
              contract schedule and cost.

Activity 5    Bi-directional traceability  There is no evidence to      Weakness
              between the software-        show traceability between
              related contractual          contractual requirements
              requirements and the         and the contractor's work
              contractor's software work   products.
              products and services is
              maintained throughout the
              effort.

Measurement   Measurements are made and    No measurements made to      Weakness
1             used to determine the        determine status of the
              status of the requirements   requirements development
              development and management   and management activities.
              activities and resultant
              products.

Verification  Requirements development     Project manager              Strength
1             and management activities    periodically reviews
              are reviewed by acquisition  requirements development
              organization management      and management activities.
              (and the contractor) on a
              periodic basis.

Verification  Requirements development     Periodic and event-driven    Strength
2             and management activities    meetings are held to
              are reviewed by the project  discuss requirements
              manager on both a periodic   development and management
              and event-driven basis.      activities.
--------------------------------------------------------------------------------

   PROJECT MANAGEMENT
------------------------------------------------------------ Letter :7

The purpose of project management is to manage the activities of the
project office and supporting contractors to ensure a timely,
efficient, and effective software acquisition.  Effective project
management requires, among other things, that project teams (1) be
organized to accomplish the project's objective, (2) have a written
policy for the management of the software project, (3) document their
plans for the activities of the project team, (4) have the authority
to alter either the project's performance, cost, or schedule baseline
while maintaining the other two, and (5) periodically brief
management on the status of project management activities. 


      FMS PROJECT PERFORMING MOST
      PROJECT MANAGEMENT PRACTICES
---------------------------------------------------------- Letter :7.1

The FMS project has many process strengths in project management. 
For example, a team was assigned responsibility for managing the
project and staffed with experienced individuals whose roles and
responsibilities were defined.  The program management plan was
written and a corrective action system to track issues and problems
was implemented.  However, the District has no written policy for the
execution of the software project.  As a result, the District has no
assurance that FMS or any other software acquisition project it
undertakes will be conducted in a disciplined manner.  Table 5 shows
the strengths, weaknesses, and observations for the project
management KPA and the specific findings supporting these ratings. 



                                     Table 5
                     
                           Project Management Findings

Feature       Key practice                 Finding                      Rating
------------  ---------------------------  ---------------------------  --------
Commitment 1  The acquisition              There is no documented       Weakness
              organization has a written   policy for the execution of
              policy for execution of the  the software project.
              software project.

Commitment 2  Responsibility for project   Responsibility for project   Strength
              management is designated.    management has been
                                           designated to the financial
                                           management system project
                                           team/program manager.

Ability 1     A team that is responsible   A team that is responsible   Strength
              for performing the           for performing software
              project's software           acquisition management
              acquisition management       activities exists.
              activities exists.

Ability 2     Adequate resources for the   Adequate personnel           Observat
              project team and matrix      resources for project        ion
              support persons are          management were provided.
              provided for the duration    At the time of the audit,
              of the software acquisition  there were no telephones,
              project.                     copiers, or supplies.

Ability 3     When project trade-offs are  The project manager is not   Weakness
              necessary, the project       permitted to independently
              manager is permitted to      alter either the
              alter either the             performance, cost, or
              performance, cost, or        schedule.
              schedule software
              acquisition baseline.

Ability 4     The project team and matrix  Project team and matrix      Strength
              support individual(s) have   support individuals have
              experience or receive        experience in acquisition
              training in project          management activities.
              software acquisition
              management activities.

Activity 1    The project team performs    Project team activities are  Strength
              its activities in            performed in accordance
              accordance with its          with program management
              documented software          plan.
              acquisition management
              plans.

Activity 2    The organization of the      The organization of the      Strength
              project provides for the     project provides for the
              management of all project    management of all project
              functions.                   functions.

Activity 3    The software acquisition     The roles and                Strength
              management activities of     responsibilities of team
              the project team are         members are defined and
              directed to accomplish the   serve to accomplish the
              project's objectives.        project's objectives.

Activity 4    The software acquisition     The software acquisition     Strength
              management activities of     management activities of
              the project team are         the team are controlled.
              controlled.

Activity 5    The project team implements  A corrective action system,  Strength
              a corrective action system   called the issue process,
              for the identification,      has been implemented.
              recording, tracking, and
              correction of problems
              discovered during the
              software acquisition.

Activity 6    The project team tracks      The project team tracks the  Strength
              project status, execution,   project's status and
              funding, and expenditures    execution. Funding and
              and takes action.            expenditures are tracked by
                                           the Office of the Chief
                                           Financial Officer.

Measurement   Measurements are made and    No evidence that             Weakness
1             used to determine the        measurements are taken to
              status of the project        determine the status of
              management activities and    project management
              resultant products.          activities.

Verification  Project management           Monthly meetings are held    Strength
1             activities are reviewed by   where all levels of
              acquisition organization     management are briefed on
              management on a periodic     the project status.
              basis.

Verification  Project management           Monthly meetings are held    Strength
2             activities are reviewed by   where all levels of
              the project manager on both  management are briefed on
              a periodic and event-        the project status.
              driven basis.
--------------------------------------------------------------------------------

   CONTRACT TRACKING AND OVERSIGHT
------------------------------------------------------------ Letter :8

The purpose of contract tracking and oversight is to ensure that (1)
the software development contractor performs according to the terms
of the contract, (2) needed contract changes are identified,
negotiated, and incorporated into the contract, and (3) contractor
performance issues are identified early, when they are easier and
less costly to address.  An effective contract tracking and oversight
process, among other things, includes (1) having a written
organizational policy for contract tracking and oversight, (2) having
a documented plan for contract tracking and oversight, (3) conducting
tracking and oversight activities in accordance with the plan, and
(4) ensuring that individuals performing contract tracking and
oversight are suitably experienced or trained. 


      FMS PROJECT PERFORMING MANY
      BUT NOT ALL CONTRACT
      TRACKING AND OVERSIGHT
      PRACTICES
---------------------------------------------------------- Letter :8.1

The FMS project had many strengths in this KPA.  The project has a
designated project manager, a group is responsible for managing
contract tracking and oversight activities, and the team is meeting
periodically with the contractor and tracking issues in a corrective
action system.  However, at the time of our review, there was no
contracting specialist supporting the team in the execution of the
contract.  In addition, the District has no documented policy for
contract tracking and oversight activities.  Table 6 shows the
strengths, weaknesses, and observations for the contract tracking and
oversight KPA and the specific findings supporting these ratings. 



                                     Table 6
                     
                     Contract Tracking and Oversight Findings

Feature       Key practice                 Finding                      Rating
------------  ---------------------------  ---------------------------  --------
Commitment 1  The acquisition              There is no written policy   Weakness
              organization has a written   for contract tracking and
              policy for contract          oversight activities for
              tracking and oversight of    the financial management
              the contracted software      system project.
              effort.

Commitment 2  Responsibility for contract  The project manager and the  Strength
              tracking and oversight       program management team are
              activities is designated.    responsible for contract
                                           tracking and oversight.

Commitment 3  The project team is          The project team is not      Weakness
              supported by contracting     supported by contracting
              specialists in the           specialists.
              execution of the contract.

Ability 1     A group that is responsible  Project manager and program  Strength
              for managing contract        management staff are
              tracking and oversight       responsible for managing
              activities exists.           contract tracking and
                                           oversight activities.

Ability 2     Adequate resources are       Staff and funding resources  Observat
              provided for contract        are adequate. However, at    ion
              tracking and oversight       the time of the audit, the
              activities.                  project team was lacking
                                           tools such as telephones,
                                           fax, copiers, and office
                                           supplies.

Ability 3     Individuals performing       Staff members have           Strength
              contract tracking and        experience in contract
              oversight activities have    tracking and oversight.
              experience or receive
              training.

Activity 1    The project team performs    The project team performs    Strength
              its activities in            its activities in
              accordance with its          accordance with the program
              documented contract          management plan.
              tracking and oversight
              plans.

Activity 2    The project team reviews     The project team did not     Weakness
              required contractor          review any of the
              software planning documents  contractor's planning
              which, when satisfactory,    documents (e.g., project
              are used to oversee the      management plan, software
              contractor's software        risk management plan,
              engineering effort.          software engineering plan,
                                           configuration management
                                           plan).

Activity 3    The project team conducts    The team conducts periodic   Strength
              periodic reviews and         reviews with the
              interchanges with the        contractor.
              contractor.

Activity 4    The project team reviews     The District does not plan   Not
              and tracks the development   to support FMS in-house.     rated
              of the software engineering
              environment required to
              provide life-cycle support
              for the acquired software.

Activity 5    Any problems or issues       The issue tracking system    Strength
              found by the project team    is used to track and
              during contract tracking     provide oversight of
              and oversight are recorded   problems or issues found
              in the appropriate           during the contract. Issues
              corrective action system     are tracked to closure.
              and tracked to closure.

Activity 6    The project team maintains   No one assumed               Weakness
              the integrity of the         responsibility for
              contract throughout the      maintaining the integrity
              contract performance         of the contract.
              period.

Measurement   Measurements are made and    No measurements are taken    Weakness
1             used to determine the        to determine the status of
              status of the contract       contract tracking and
              tracking and oversight       oversight activities.
              activities and resultant
              products.

Verification  Contract tracking and        Periodic and event-driven    Strength
1             oversight activities are     meetings are held with the
              reviewed by acquisition      Authority to review
              organization management on   contract tracking and
              a periodic basis.            oversight activities.

Verification  Contract tracking and        Periodic and event-driven    Strength
2             oversight activities are     meetings are held with the
              reviewed by the project      program manager to review
              manager on both a periodic   contract tracking and
              and event-driven basis.      oversight activities.
--------------------------------------------------------------------------------

   EVALUATION
------------------------------------------------------------ Letter :9

The purpose of evaluation (testing) is to determine that the acquired
software products and services satisfy contract requirements prior to
acceptance.  The evaluation process includes (1) documenting
evaluation plans and conducting evaluation activities in accordance
with the plan, (2) developing and managing evaluation requirements in
conjunction with developing software technical requirements, (3)
incorporating evaluation requirements into the solicitation and the
resulting contract, (4) tracking contractor performance of evaluation
activities for compliance with the contract, (5) ensuring that
adequate resources are provided for evaluation activities, and (6)
measuring and reporting on the status of evaluation activities to
management. 


      FMS PROJECT PERFORMING SOME
      BUT NOT MOST EVALUATION
      PRACTICES
---------------------------------------------------------- Letter :9.1

The FMS project has some process strengths in this KPA.  For example,
responsibility for evaluation activities has been designated to the
project manager, individuals designated to perform evaluation
activities have experience, and members of the evaluation team
received briefings on the objectives of the evaluation.  However,
there is no documented evaluation policy or plan, no evidence that
evaluation requirements have been developed, and neither the
Authority nor the project manager reviews the status of evaluation
activities.  Table 7 shows the strengths, weaknesses, and
observations for the evaluation KPA and the specific findings
supporting these ratings. 



                                     Table 7
                     
                               Evaluation Findings

Feature       Key practice                 Finding                      Rating
------------  ---------------------------  ---------------------------  --------
Commitment 1  The acquisition              No written policy exists     Weakness
              organization has a written   for managing the evaluation
              policy for managing the      of acquired software
              evaluation of the acquired   products and services.
              software products and
              services.

Commitment 2  Responsibility for           Responsibilities for         Strength
              evaluation activities is     evaluation activities has
              clearly designated.          been designated to the
                                           program manager.

Ability 1     A group that is responsible  The joint program            Strength
              for planning, managing, and  management and contractor
              performing evaluation        team, along with the users,
              activities for the project   is responsible for
              exists.                      planning, managing, and
                                           performing evaluation
                                           activities.

Ability 2     Adequate resources are       While personnel resource     Observat
              provided for evaluation      requirements for evaluation  ion
              activities.                  are identified, other
                                           resources such as
                                           traceability and data
                                           collection tools have not
                                           yet been identified.

Ability 3     Individuals performing       Individuals performing       Strength
              evaluation activities have   evaluation have experience
              experience or receive        in evaluation.
              training.

Ability 4     Members of the project team  Members of the project team  Strength
              and groups supporting the    receive orientation on the
              software acquisition         objectives of the
              receive orientation on the   evaluation.
              objectives of the
              evaluation approach.

Activity 1    The project team performs    There is no documented       Weakness
              its activities in            evaluation plan.
              accordance with its
              documented evaluation
              plans.

Activity 2    The project's evaluation     There is no evidence that    Weakness
              requirements are developed   evaluation requirements
              in conjunction with the      were developed in
              development of the system    conjunction with system
              or software technical        requirements.
              requirements.

Activity 3    The evaluation requirements  Requirements for the         Strength
              are incorporated into the    contractor to support the
              solicitation package and     evaluation are in the
              resulting contract.          solicitation and contract.

Activity 4    The project team assesses    The project team did not     Weakness
              the contractor's             assess the contractor's
              performance for compliance   performance for compliance
              with evaluation              with evaluation
              requirements.                requirements.

Activity 5    Planned evaluations are      Since no documented          Weakness
              performed on the acquired    evaluation plan exists,
              software products and        there is no plan to follow.
              services prior to
              acceptance for operational
              use.

Activity 6    Results of the evaluations   The FMS project has not yet  Observat
              are analyzed and compared    reached the stage where      ion
              to the contract's            evaluation activities are
              requirements to establish    analyzed and compared to
              an objective basis to        the contract.
              support the decision to
              accept the products and
              services or to take further
              action.

Measurement   Measurements are made and    No measurements are made to  Weakness
1             used to determine the        determine the status of
              status of the evaluation     evaluation activities.
              activities and resultant
              products.

Verification  Evaluation activities are    The Authority does not       Weakness
1             reviewed by acquisition      review the status of
              organization management on   evaluation activities.
              a periodic basis.

Verification  Evaluation activities are    The project manager does     Weakness
2             reviewed by the project      not review the status of
              manager on both a periodic   evaluation activities.
              and event-driven basis.
--------------------------------------------------------------------------------

   ACQUISITION RISK MANAGEMENT
----------------------------------------------------------- Letter :10

SEI defines risk as the possibility of suffering a loss.  The purpose
of acquisition risk management is to formally identify risks as early
as possible and adjust the acquisition to mitigate those risks.  An
effective risk management process, among other things, includes (1)
having a written policy on acquisition risk management, (2)
developing a software acquisition risk management plan, (3)
conducting software risk management activities in accordance with the
plan (e.g., identifying risks, taking mitigation actions, and
tracking risk mitigation actions to completion), and (4) measuring
and reporting on the status of acquisition risk management activities
to management. 


      FMS RISK MANAGEMENT
      PROCESSES ARE INEFFECTIVE
--------------------------------------------------------- Letter :10.1

The FMS project had one strength for this KPA.  The project has
designated responsibility for risk management to the project
management team.  However, the District is not performing any of the
other practices to satisfy this KPA.  For example, there is no
written policy or plan for acquisition risk management, resource
requirements for risk management have not been identified, and at the
time of this audit, neither the Authority nor the project manager
were reviewing the activities for risk management.  Table 8 shows the
strengths, weaknesses, and observations for the acquisition risk
management KPA and the specific findings supporting these ratings. 



                                     Table 8
                     
                       Acquisition Risk Management Findings

Feature       Key practice                 Finding                      Rating
------------  ---------------------------  ---------------------------  --------
Commitment 1  The acquisition              There is no written policy   Weakness
              organization has a written   for software acquisition
              policy for the management    risk management.
              of software acquisition
              risk.

Commitment 2  Responsibility for software  Responsibility for software  Strength
              acquisition risk management  acquisition risk management
              activities is designated.    was designated to the
                                           acquisition management
                                           contractor and the program
                                           office.

Ability 1     A group that is responsible  No group is responsible for  Weakness
              for coordinating software    coordinating software
              acquisition risk management  acquisition risk management
              activities exists.           activities.

Ability 2     Adequate resources are       Resource requirements for    Weakness
              provided for software        acquisition risk management
              acquisition risk management  have not yet been defined.
              activities.

Ability 3     Individuals performing       Individuals designated to    Weakness
              software acquisition risk    perform software
              management activities have   acquisition risk management
              experience or receive        do not have experience and
              required training.           have not yet received
                                           training.

Activity 1    Software acquisition risk    Software acquisition risk    Weakness
              management activities are    management activities were
              integrated into software     not integrated into
              acquisition planning.        software acquisition
                                           planning.

Activity 2    The software acquisition     A software acquisition risk  Weakness
              risk management plan is      management plan has not
              developed in accordance      been developed in
              with the project's defined   accordance with a defined
              software acquisition         software acquisition
              process.                     process.

Activity 3    The project team performs    There is no documented       Weakness
              its software acquisition     acquisition risk management
              risk management activities   plan.
              in accordance with its
              documented plans.

Activity 4    Risk management is           Risk management is not       Weakness
              conducted as an integral     conducted as an integral
              part of the solicitation,    part of the solicitation,
              project performance          project performance
              management, and contract     management, and contract
              performance management       performance management
              processes.                   processes.

Activity 5    Software acquisition risk    Software acquisition risk    Weakness
              handling actions are         handling actions are not
              tracked and controlled       tracked and controlled
              until the risks are          until the risks are
              mitigated.                   mitigated.

Measurement   Measurements are made and    Measurements are not made    Weakness
1             used to determine the        for risk management.
              status of the acquisition
              risk management activities
              and resultant products.

Verification  Acquisition risk management  Risk management activities   Weakness
1             activities are reviewed by   are not reviewed by the
              acquisition organization     Authority.
              management on a periodic
              basis.

Verification  Acquisition risk management  Risk management activities   Weakness
2             activities are reviewed by   are not reviewed by the
              the project manager on both  project manager.
              a periodic and event-
              driven basis.
--------------------------------------------------------------------------------

   CONCLUSIONS
----------------------------------------------------------- Letter :11

Leading software acquisition organizations rely on defined and
disciplined software acquisition processes to deliver promised
software capabilities on time and within budget, first on a
project-by-project basis, and later, as the organization's processes
become more mature, consistently across the institution.  While the
District has many strengths in its acquisition processes for FMS, it
also has many weaknesses that, overall, make its processes
undisciplined and immature.  As a result, the District's success or
failure in acquiring FMS depends largely on specific individuals
rather than on well-defined software acquisition management
practices.  This greatly reduces the probability that the system will
consistently perform as intended and be delivered on schedule and
within budget. 

To satisfy the intent of all the software acquisition key process
areas and thereby have a reasonable assurance that acquisition
efforts are effectively planned, managed, evaluated, and tracked, the
District must address the many weaknesses identified in this report. 
This would entail the District formulating and implementing a written
policy for software acquisition planning, requirements development
and management, project management, contract tracking and oversight,
evaluation, and acquisition risk management.  In addition, it is
important for the District to track the various activities for each
KPA to ensure that they are being performed and that evaluation and
risk management activities are being planned and effectively
conducted. 


   RECOMMENDATIONS
----------------------------------------------------------- Letter :12

We recommend that the Chairman of the District of Columbia Financial
Responsibility and Management Assistance Authority direct the
District's Chief Financial Officer to (1) take the following actions
for the six KPAs we reviewed to ensure that the current FMS
acquisition and implementation is satisfactorily completed and (2)
apply these actions to any future software acquisitions. 

Software Acquisition Planning: 

  -- Document decisions and update the planning documents to ensure
     that large acquisitions such as FMS can be effectively managed. 

  -- Designate responsibility for software acquisition planning
     activities. 

  -- Determine required resources for acquisition planning. 

  -- Ensure that measurements of software acquisition activities are
     taken. 

  -- Ensure that the software acquisition planning documentation is
     updated as well as make program changes regarding outsourcing of
     the data center and upgrading the current system versus buying
     off-the-shelf. 

  -- Ensure that the software acquisition planning documentation
     addresses life-cycle support of the software. 

  -- Develop a written policy for software acquisition planning. 

Requirements Development and Management: 

  -- Develop an organizational policy for establishing and managing
     software-related requirements. 

  -- Clearly assign responsibility for requirements development and
     management. 

  -- Document either resource requirements or resources expended for
     requirements development activities. 

  -- Develop the capability to trace between contractual requirements
     and the contractor's work products. 

  -- Develop measurements to determine the status of the requirements
     development and management activities. 

Project Management: 

  -- Develop a written policy for the execution of the software
     project. 

  -- Authorize the project manager to independently alter either the
     performance, cost, or schedule. 

  -- Require that measurements be taken to determine the status of
     project management activities. 

Contract Tracking and Oversight: 

  -- Develop written policy for contract tracking and oversight
     activities for the financial management system project. 

  -- Support the project team with contracting specialists. 

  -- Require that the project team review the contractor's planning
     documents (for example, the project management plan, software
     risk management plan, software engineering plan, configuration
     management plan). 

  -- Assign someone responsibility for maintaining the integrity of
     the contract. 

  -- Take measurements to determine the status of contract tracking
     and oversight activities. 

Evaluation: 

  -- Develop written policy for managing the evaluation of acquired
     software products and services. 

  -- Develop a documented evaluation plan. 

  -- Develop evaluation requirements in conjunction with system
     requirements. 

  -- Assess the contractor's performance for compliance with
     evaluation requirements. 

  -- Develop measurements to determine the status of evaluation
     activities. 

  -- Ensure that the Authority and the project manager review the
     status of evaluation activities. 

Acquisition Risk Management: 

  -- Develop written policy for software acquisition risk management. 

  -- Designate a group to be responsible for coordinating software
     acquisition risk management activities. 

  -- Define resource requirements for acquisition risk management. 

  -- Ensure that individuals designated to perform software
     acquisition risk management have adequate experience and
     training. 

  -- Integrate software acquisition risk management activities into
     software acquisition planning. 

  -- Develop a software acquisition risk management plan in
     accordance with a defined software acquisition process. 

  -- Develop a documented acquisition risk management plan and
     conduct risk management as an integral part of the solicitation,
     project performance management, and contract performance
     management processes. 

  -- Track and control software acquisition risk handling actions
     until the risks are mitigated. 

  -- Ensure that risk management activities are reviewed by the
     Authority and the project manager. 


   AGENCY COMMENTS AND OUR
   EVALUATION
----------------------------------------------------------- Letter :13

GAO requested comments on a draft of this report from the Chairman,
District of Columbia Financial Responsibility and Management
Assistance Authority, and the District's Chief Financial Officer. 
They provided us with written comments that are reprinted in
appendixes I and II. 

In their comments, the District of Columbia Financial Responsibility
and Management Assistance Authority's Executive Director and the
District of Columbia's Chief Financial Officer acknowledged that the
software acquisition project for the new financial management system
was a high risk initiative and that the District's processes were not
sufficiently mature.  The District Chief Financial Officer identified
initiatives in each of the key process areas.  Both cited ongoing
corrective actions, which, if properly implemented, will address
several of our recommendations.  For example, the Chief Financial
Officer stated that the District is developing a Risk Management Plan
and is evaluating various strategies to identify and manage risks,
and that the Chief Technology Officer for the District of Columbia is
developing policies and procedures for information resource
management which will include software acquisition. 

However, the District Chief Financial Officer also added that their
efforts to date have achieved a sound acquisition state consistent
with the intent of the SA-CMM.  As discussed in the report,
significant improvements would be necessary to achieve the minimally
acceptable level of maturity as defined by the Software Engineering
Institute's Software Acquisition Maturity Model to satisfy the intent
of all the software acquisition key process areas.  Accordingly, the
District has not yet achieved a sound acquisition state consistent
with the intent of the SA-CMM.  If the District is to instill the
needed discipline into its systems acquisition processes consistent
with the intent of SA-CMM, it will need to effectively implement all
of our recommendations. 

We are sending copies of this report to the Ranking Minority Member
of your Subcommittee and to the Chairmen and Ranking Minority Members
of the Subcommittee on Oversight of Government Management,
Restructuring, and the District of Columbia, Senate Committee on
Governmental Affairs, the Subcommittee on the District of Columbia,
Senate Committee on Appropriations, and the Subcommittee on the
District of Columbia, House Committee on Government Reform and
Oversight.  We are also sending copies to the Director of the Office
of Management and Budget, the Chairman of the District of Columbia
Financial Responsibility and Management Assistance Authority, and the
Chief Financial Officer of the District of Columbia.  Copies will be
made available to others upon request.  If you have questions or wish
to discuss the issues in this report, please contact me at (202)
512-6412.  Major contributors to this report are listed in appendix
III. 

Sincerely yours,

Dr.  Rona B.  Stillman
Chief Scientist for
Computers and Telecommunications




(See figure in printed edition.)Appendix I
COMMENTS FROM THE CHAIRMAN,
DISTRICT OF COLUMBIA FINANCIAL
RESPONSIBILITY AND MANAGEMENT
ASSISTANCE AUTHORITY
============================================================== Letter 



(See figure in printed edition.)




(See figure in printed edition.)Appendix II
COMMENTS FROM THE DISTRICT OF
COLUMBIA CHIEF FINANCIAL OFFICER
============================================================== Letter 



(See figure in printed edition.)



(See figure in printed edition.)



(See figure in printed edition.)



(See figure in printed edition.)


MAJOR CONTRIBUTORS TO THIS REPORT
========================================================= Appendix III

ACCOUNTING AND INFORMATION
MANAGEMENT DIVISION, WASHINGTON,
D.C. 

Keith Rhodes, Technical Director
Nabajyoti Barkakati, Assistant Director
Hodge Herry, Assistant Director
John C.  Martin, Assistant Director
Prithviraj Mukherji, Assistant Director
Madhav Panwar, Assistant Director
Norma Samuel, Project Manager

OFFICE OF GENERAL COUNSEL

Richard Cambosos, Senior Attorney


*** End of document. ***