2000 Census: New Data Capture System Progress and Risks (Letter Report,
02/04/2000, GAO/AIMD-00-61).
Pursuant to a congressional request, GAO provided information on the
Bureau of the Census' Data Capture System (DCS) 2000 for the tabulation
of 2000 census data, focusing on the status and quality of DCS 2000, as
well as the risks Census faces in successfully completing the system.
GAO noted that: (1) Census has made considerable progress on DCS 2000;
(2) however, Census faces a formidable challenge in delivering promised
DCS 2000 capabilities on time because much remains to be accomplished
and little time remains to accomplish it; (3) Census has extended the
system's schedule by 4 months due largely to requirement changes; (4)
and, before its revised completion date of February 25, 2000, Census
must complete many important system development and testing activities,
including the completion of the final two software releases as well as
system acceptance, site acceptance, and operational tests; (5) the
numbers of yet-to-be-resolved defects for DCS 2000 have yet to show a
clear and sustained downward trend that is expected as a system begins
to mature; (6) to expedite the completion of DCS 2000, Census and its
development contractor are following an incremental development and
deployment strategy; (7) such a strategy can save time because it can
get a system into the hands of users faster so that problems can be
identified sooner rather than later; (8) while Census' implementation of
this strategy has introduced considerable development and test
concurrency, which increases the risk of defects being found and
corrected independently and thus inconsistent system baselines being
produced, this risk is being mitigated by the development contractor
through effective management controls, such as project risk management
and configuration management; (9) Census and its development contractor
have taken other steps to ensure the successful delivery of DCS 2000;
(10) another factor strongly in Census' favor is that its development
contractor has been independently assessed as having highly effective
software development capabilities in such important areas as software
project planning, tracking and oversight, configuration management,
software quality management, and defect prevention; (11) nevertheless,
delivering promised DCS 2000 capabilities remains at risk because less
than 2 months remain before data capture operations are to begin,
leaving very little room for error; (12) many important development and
test activities remain that will likely reveal more system defects and
thus compound an already uncertain system maturation picture; and (13)
in discussing this risk with DCS 2000 program officials, they agreed
that delivering promised system capabilities on time is a risk, and they
subsequently provided evidence that they have: (a) designated this as a
high risk under the DCS 2000 risk management program; and (b) defined
and initiated proactive steps to mitigate the risk and its potential
impact on the program.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: AIMD-00-61
TITLE: 2000 Census: New Data Capture System Progress and Risks
DATE: 02/04/2000
SUBJECT: Census
Systems design
Risk management
Population statistics
Data collection
Management information systems
Information resources management
Computer software verification and validation
IDENTIFIER: Census Bureau Data Capture System 2000
Internet
2000 Decennial Census
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO report. Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved. Major **
** divisions and subdivisions of the text, such as Chapters, **
** Sections, and Appendixes, are identified by double and **
** single lines. The numbers on the right end of these lines **
** indicate the position of each of the subsections in the **
** document outline. These numbers do NOT correspond with the **
** page numbers of the printed product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
** A printed copy of this report may be obtained from the GAO **
** Document Distribution Center. For further details, please **
** send an e-mail message to: **
** **
** **
** **
** with the message 'info' in the body. **
******************************************************************
Report to the Subcommittee on the Census, Committee on Government Reform,
House of Representatives
February 2000
2000 CENSUS
New Data Capture System Progress and Risks
*****************
*****************
GAO/AIMD-00-61
Letter 3
Appendixes
Appendix I: Briefing on the Decennial Census
18
Appendix II: Comments From the Department of Commerce
99
Figure 1: Remaining DCS 2000 Activities 9
Figure 2: Results of Cluster Accuracy Tests for Optical Mark
Recognition 10
Figure 3: Results of Cluster Accuracy Tests for Optical Character
Recognition10
Figure 4: Trends in Defects--All Unresolved PTRs by Week12
Figure 5: Trends in Defects--Unresolved Severe PTRs by Week13
DCS Data Capture System
PTR program trouble report
SEI Software Engineering Institute
Accounting and Information
Management Division
B-284367
February 4, 2000
The Honorable Dan Miller
Chairman
The Honorable Carolyn B. Maloney
Ranking Minority Member
Subcommittee on the Census
Committee on Government Reform
House of Representatives
The U.S. Census Bureau is preparing to conduct the 2000 decennial census-
an event that will significantly influence the lives of every U.S.
resident and constitutes the largest peacetime mobilization event in our
nation's history. In particular, Census will be staffing 1.35 million
temporary field positions to capture 1.5 billion pages of data from about
119 million households. To meet this important and massive challenge, the
bureau plans to rely extensively on information technology, including its
new Data Capture System (DCS) 2000. DCS 2000 is to be operated at four
bureau data capture centers/Footnote1/ to check in, digitally image, and
optically read the data handwritten onto census forms and convert these
data into files that will be transmitted to bureau headquarters for
tabulation and analysis.
At your request, we determined the status and quality of DCS 2000 as well
as the risks the bureau faces in successfully completing the system. This
report summarizes the information presented at our December 14, 1999,
briefing to your office. A copy of our briefing is provided in appendix I.
Results in Brief
The bureau has made considerable progress on DCS 2000. For example, as of
January 7, 2000, DCS 2000 hardware has been installed at all four data
capture centers, and 21 of the system's 23 planned application software
releases have been completed. Despite this progress, however, the bureau
faces a formidable challenge in delivering promised DCS 2000 capabilities
on time because much remains to be accomplished and little time remains to
accomplish it. In particular, the bureau has extended the system's
schedule by 4 months due largely to requirements changes. And, before its
revised completion date of February 25, 2000, the bureau must complete
many important system development and testing activities, including the
completion of the final two software releases as well as system
acceptance, site acceptance, and operational tests. Furthermore, the
numbers of yet-to-be-resolved defects for DCS 2000 have yet to show a
clear and sustained downward trend that is expected as a system begins to
mature.
To expedite the completion of DCS 2000, the bureau and its development
contractor are following an incremental development and deployment
strategy. Such a strategy can save time because it can get a system into
the hands of users faster so that problems can be identified sooner rather
than later. While the bureau's implementation of this strategy has
introduced considerable development and test concurrency, which increases
the risk of defects being found and corrected independently and thus
inconsistent system baselines being produced, this risk is being mitigated
by the development contractor through effective management controls, such
as project risk management and configuration management.
The bureau and its development contractor have taken other steps to ensure
the successful delivery of DCS 2000. For example, they have implemented a
formal risk management program to address risks proactively, and they have
defined and are following requirements management processes to ensure that
only those changes that are justified on the basis of costs, benefits, and
risks are approved and made. Also, they have added a test event just prior
to the system's operational date to evaluate the system's capabilities in
a true operational setting. Another factor strongly in Census' favor is
that its development contractor has been independently assessed as having
highly effective software development capabilities in such important areas
as software project planning, tracking and oversight, configuration
management, software quality management, and defect prevention.
Nevertheless, delivering promised DCS 2000 capabilities remains at risk
because less than 2 months remain before data capture operations are to
begin, leaving very little room for error. Moreover, many important
development and test activities remain that will likely reveal more system
defects and thus compound an already uncertain system maturation picture.
In discussing this risk with DCS 2000 program officials, they agreed that
delivering promised system capabilities on time is a risk, and they
subsequently provided evidence that they have (1) designated this as a
high risk under the DCS 2000 risk management program and (2) defined and
initiated proactive steps to mitigate the risk and its potential impact on
the program.
In commenting on a draft of this report the bureau generally agreed with
our observations and conclusions and stated that it appreciated our
insights and contributions.
Background
The Constitution requires a decennial census of the population in order to
reapportion seats in the House of Representatives. Public and private
decisionmakers also use census data on population counts and social and
economic characteristics for a variety of purposes. For example, state and
local redistricting; allocations of government funding; and many planning
and evaluation activities, such as site selection for new schools, market
research, and evaluations of local labor markets, rely on decennial census
data. In addition, the census is the only national source of detailed
population statistics for small geographic areas, such as towns and school
districts, and for population groups, such as Native Americans.
Since 1970, the bureau has used essentially the same methodology to
conduct the census. It develops an address list of the nation's housing
units and delivers census forms to those housing units, requesting that
occupants mail back the completed forms. Most households are sent a short
form to complete; however, some are asked to complete a long
form./Footnote2/ The bureau then hires temporary census-takers, known as
enumerators, by the hundreds of thousands to gather the requested
information for each nonresponding housing unit. For the 2000 census, some
households will have the option of completing questionnaires via a bureau-
operated toll-free telephone number or a bureau-operated Internet site.
To conduct the 2000 census, the bureau will rely on 10 key systems. These
systems will enable the bureau to develop and maintain address lists,
maps, and geographic reference files; collect census data via the
Internet; scan and process household-completed paper forms; analyze census
data; recruit and support temporary workers; facilitate follow-up surveys;
and track costs and performance related to taking the census. A simplified
diagram of the census 2000 systems architecture as well as descriptions of
each system are provided in appendix I.
An important system for this census will be DCS 2000. The bureau will use
this system to check in forms and digitally image and optically read the
data handwritten onto census forms and convert these data into files that
will be transmitted to bureau headquarters for tabulation and analysis.
Specific DCS 2000 subsystems include:
o The data verification and receipt subsystem, which, among other
things, (1) receives the paper census forms and prepares them for
imaging and
(2) identifies respondents so the bureau can identify and follow up
with nonrespondents.
o The scanning and imaging subsystem, which creates an electronic image
of the paper form.
o The optical recognition subsystem, which captures census data from
the electronic form images.
o The keying subsystem, which is used to manually input data that
cannot be satisfactorily read from the paper form.
o The data preparation function, which formats data from the optical
recognition and keying subsystems and then sends the data to Census
Bureau headquarters.
The bureau has contracted with Lockheed Martin-Mission Systems to develop,
deploy, and maintain DCS 2000. As of November 5, 1999, the bureau had
obligated $127 million for this contract. The total contract value as of
September 30, 1999, was $153 million. The bureau has also contracted with
TRW for space and facility management and for DCS 2000 operation at three
of the four data capture centers./Footnote3/
Objectives, Scope, and Methodology
Our first objective was to determine DCS 2000's status and quality, i.e.,
whether the system performs in a way that meets specified requirements and
satisfies user needs. Our second objective was to assess the risks the
bureau faces in successfully completing the system.
To determine the status and quality of DCS 2000, we first reviewed system
development plans, project management plans, activity plans (test,
deployment, etc.), project status reports, program management briefings,
and system functional requirements and specifications. We then compared
original to revised plans and current plans to status and test reports. We
also identified how the bureau measured the maturity and quality of DCS
2000 throughout its development life cycle and analyzed these and other
measures for indicators of progress and system health, including trends in
defects and the results of testing activities. We observed the DCS 2000
operational test at the Baltimore data capture center and attended
periodic DCS 2000 project review meetings between the bureau and the
development contractor.
To determine DCS 2000 risks and risk mitigators, we determined bureau and
development contractor key management processes (e.g., requirements
management, configuration management, project planning, and project
tracking and oversight). We analyzed the results of status and quality
determinations vis-****ITCCentury Book:x88****-vis DCS 2000 expectations
and remaining steps to complete the system. We also analyzed plans for
completing DCS 2000 and compared these plans to generally accepted system
engineering principles.
We interviewed bureau and contractor officials throughout our review,
briefed them on the results of our work, and incorporated updated
information they provided into our briefing. We performed our work at the
Census Bureau's headquarters in Suitland, Maryland; the DCS 2000 program
office in Lanham, Maryland; bureau and development contractor facilities
in Bowie, Maryland; and the Census Bureau's data capture center in
Baltimore, Maryland, from June 1999 through January 2000, in accordance
with generally accepted government auditing standards. We requested
comments on a draft of this report from the Department of Commerce. The
Secretary of Commerce provided us with written comments, which are
presented in appendix II, and are discussed in the "Agency Comments and
Our Evaluation" section of this letter.
Considerable Progress Made on DCS 2000 But Much Remains to Be Done in
Little Time
The Census Bureau has made considerable progress on DCS 2000. For example,
as of January 7, 2000, 21 of 23 planned software releases had been
completed and 6 of 10 major test events had been completed. Further,
according to the bureau, all DCS 2000 hardware has been installed at all
sites. As of January 7, 2000, the bureau was reporting that remaining DCS
2000 tasks were on target with revised expectations.
Nevertheless, DCS 2000's expected completion is much later than earlier
planned and, in fact, perilously close to the date when census 2000 is to
begin. In February 1999, the bureau expected to complete DCS 2000 by
October 15, 1999. However, the scheduled completion date has since changed
to February 25, 2000, which is an extension of over 4 months and is less
than 2 weeks before data capture operations are to begin on
March 6, 2000. Under normal circumstances, completion deadlines can be
changed and some schedule slippage can be tolerated. However, the demands
of the decennial census necessitate that remaining development, testing,
and deployment activities be completed in less than 2 months.
DCS 2000 program officials told us that this extension is due largely to
system requirement changes. Specifically, from May 15, 1999, through
October 30, 1999, 63 requirements changes were initiated for DCS 2000.
And, in October and November 1999, two new software releases were added to
DCS 2000-one slated for January 19, 2000, and one for
February 11, 2000. The bureau has no plans for additional requirements
changes. However, if the results of ongoing and planned operational tests
show that DCS 2000 does not adequately support production environment data
capture operations, then more changes are possible.
In addition to the development and deployment of final software releases,
significant testing activities--including system, site, and operational
tests/Footnote4/-remain to be completed before February 25, 2000. These
activities are illustrated in figure 1.
Figure****Helvetica:x11****1: Remaining DCS 2000 Activities
*****************
*****************
Results from early site acceptance tests showed that not all performance
targets were achieved; however, the tests were reported as being
successful either because the problems were corrected and retested or
because they were scheduled for correcting and retesting as part of a
future test event. More recent results show that key performance measures
are being met. For example, as shown in figures 2 and 3, results from
"cluster" accuracy tests--which were performed to demonstrate the ability
of each individual configuration (known as "clusters") of scanners,
workstations, and servers to scan census forms and recognize data on those
forms--were within expected tolerances for errors.
Figure****Helvetica:x11****2: Results of Cluster Accuracy Tests for
Optical Mark Recognition
*****************
*****************
Figure****Helvetica:x11****3: Results of Cluster Accuracy Tests for
Optical Character Recognition
*****************
*****************
Similarly, results from the operational test conducted at the Baltimore,
Maryland, data capture center identified problems that have since been
reported as fixed and scheduled to be retested as part of a future
operational test. An additional operational test was conducted at the
Baltimore data capture center by an independent research organization to
assess DCS 2000's accuracy. Results from this test showed that DCS 2000
exceeded accuracy goals for both short and long Census forms.
Despite these reported successes, DCS 2000's readiness is still uncertain
due to the fact that (1) the numbers of yet-to-be-resolved system defects
have yet to show a clear and sustained downward trend expected as a system
begins to mature and (2) the time left between the conclusion of test
events and the day that DCS 2000 must be operational is less than the
average time it has been taking to resolve defects. Defects are system
problems (both hardware and software) that require a resolution (referred
to by Lockheed Martin as Program Trouble Reports, or PTRs). They can be
attributed to a failure to meet system specifications or to a requirement
that was not previously anticipated. As figures 4 and 5 further
illustrate, we found that the number of unresolved PTRs each week during
the last 12 months does not show a maturing trend, including those defects
categorized as severe./Footnote5/ In addition, as mentioned earlier, many
key test events, which are major producers of PTRs, have yet to be
conducted for DCS 2000. Lastly, the time left between conclusion of the
last test event, scheduled for February 25, 2000, and the day that DCS
2000 must be operational is 9 days, whereas it has been taking an average
of 16 days to resolve all PTRs over the last 6 months and 10 days for
severe PTRs.
Figure****Helvetica:x11****4: Trends in Defects--All Unresolved PTRs by
Week
*****************
*****************
Figure****Helvetica:x11****5: Trends in Defects--Unresolved Severe PTRs
by Week
*****************
*****************
Risk Mitigation Steps Are Being Taken to Address DCS 2000 Risks
The bureau and its contractors are employing measures to expedite the
completion of DCS 2000 and to minimize risks related to its tight schedule
and other aspects of development and deployment. The bureau and Lockheed
Martin are following an incremental development and deployment strategy.
Such a strategy is recognized as a best practice for large, complex
systems by leading public and private organizations because it can get a
system into the hands of users faster so that problems can be identified
sooner rather than later.
This approach needs to be carefully managed because it has introduced
considerable development and test concurrency in the DCS 2000 schedule.
Such concurrency can increase demands for test resources as well as the
possibility of defects being found and corrected independently, resulting
in more than one baseline test configuration. Moreover, if defects surface
during concurrent testing that are severe enough to require all affected
test activities to be stopped until the baseline is fixed, the deployment
schedule could be delayed. We found that these risks are being managed by
Lockheed Martin through the use of multiple test labs and effective
configuration management processes.
The bureau and Lockheed Martin are also jointly managing DCS 2000 risks
through a formal risk management program that includes proactive
identification of risks and analyses of their impacts and probabilities,
definition of risk mitigation plans, and tracking and reporting of the
plans' implementation and effectiveness. Such programs are also recognized
as a key practice in government and the private sector. In following the
risk mitigation program, the bureau and Lockheed Martin have already acted
to resolve several risks related to DCS 2000 site staffing, daily system
maintenance, and technical support. For example, they have strengthened
training programs for on-site DCS 2000 personnel, defined technical
support roles and responsibilities, and increased central technical
support.
More important, the bureau and Lockheed Martin have also identified the
tight development schedule as a risk and taken measures to reduce the
risk. For example, they have
o added staff to manage PTR progress;
o begun holding regular PTR status meetings with sites and other
support areas to review resolution progress;
o developed a rapid response process that requires a fix or workaround
within 24 hours;
o established an Engineering Review Board to review all change requests
to ensure that only changes that are justified on the basis of costs,
benefits, and risks are approved and made; and
o implemented processes to ensure that all releases with new
functionality are tested during an operational evaluation.
Furthermore, Census and Lockheed Martin have added another test to the DCS
2000 schedule to demonstrate simultaneous operation of all four data
capture centers. The test, planned for February 22 through February 25,
will include assessments of DCS 2000's ability to process workload
equivalent to that expected during actual data capture operations, and it
will include operational testing of the final DCS 2000 software releases.
Another important risk mitigating factor is that Lockheed Martin-Mission
Systems has been independently assessed as having highly effective
software development capabilities in important areas such as software
project planning, tracking and oversight, and configuration management.
This assessment was performed using models and methods developed by
Carnegie Mellon University's Software Engineering Institute (SEI), which
is recognized for its expertise in software processes. With DCS 2000 as
one of three systems included in the evaluation, Lockheed Martin-Mission
Systems was rated as a "Level 5" (on a scale of 1 to 5), meaning that all
projects use standard processes for developing and maintaining software
and that such processes are continually improved through quantitative
feedback and from piloting innovative ideas and technologies. Having such
capability should help to reduce risks related not only to developmental
activities such as project planning and oversight, but all remaining DCS
2000 development, testing, and deployment activities.
Conclusions
The Census Bureau has made considerable progress in acquiring and
deploying DCS 2000; however, its prospects for completing this system on
time are still mixed. In particular, significant system functionality
remains to be developed, acceptance tested, deployed, and site tested.
Further, a key measure of system quality-trends in unfixed severe defects-
is not yet moving in a direction consistent with that of a maturing
system. Given that a number of significant test events have yet to occur
and these are likely to uncover more defects, the chances of the defect
trend moving in a favorable direction soon are uncertain. Moreover, given
that the average time to fix severe defects (about 10 days) over the last
6 months would consume the time between the conclusion of later test
events and the day that DCS 2000 must be operational, the chances of all
specified capabilities being ready on time are reduced.
At the same time, results from system-level tests performed so far are
showing that key DCS 2000 performance targets are being met. Additionally,
to meet the challenge of a very demanding and fault-intolerant schedule,
the bureau has been following a plan that provides for development and
testing of DCS 2000 in a series of increments. This strategy could save
time by allowing early system capability to get into the hands of end
users faster. Lastly, the development contractor has been following
effective processes for risk management, project planning, tracking and
oversight, configuration management, software quality management, and
defect prevention-all critical to the successful completion of DCS 2000.
Nevertheless, the schedule challenge facing the bureau on DCS 2000,
particularly in light of where the system currently stands and what
remains to be accomplished, is formidable. Both the bureau and development
contractor agree and, accordingly, have initiated proactive and
appropriate risk management steps, such as strengthening oversight over
PTR progress and requirements changes. While these steps do not guarantee
success, they should mitigate the risk and its potential impact on the
program.
Agency Comments and Our Evaluation
In commenting on a draft of this report, the bureau stated that it
generally agreed with our observations and conclusions. Additionally, the
bureau stated that it shared our awareness of the challenges the bureau
faces in delivering promised DCS 2000 capabilities on time, and that it
appreciated our insights and contributions.
The bureau also offered two additional comments. First, it stated that one
sentence in the draft report implies that all of the DCS 2000 system must
be developed, tested, and deployed in 2 months while, in fact, the system
is in the final stages of testing, and only a small portion of development
remains to be completed before becoming fully operational. It was not our
intent to imply this and we have accordingly modified the sentence.
Second, the bureau stated that the 16 days that the bureau has averaged
over the last 6 months to resolve all PTRs that we cite in the draft
report is not a true representation of the time it has been recently
taking to resolve emergency PTRs, which according to the bureau was 2 days
for 7 recent emergency PTRs. We did not modify our report in response to
this comment. The report does not state that the 16-day average time
needed over the last 6 months (which covered June through November 1999)
to resolve all PTRs is the operative indicator of the time to resolve
emergency PTRs, which is the most severe category of PTRs. Rather, it
cites the time necessary to resolve severe PTRs--which was 10 days--as the
operative indicator. This category represents the average time that the
bureau's own data show it has taken over the last 6 months to resolve both
emergency and critical PTRs. We focused on both emergency and critical
because both types of PTRs, according to the Bureau's own definition of
them, will have to be resolved before March 6, 2000, if DCS 2000 is to
operate as intended.
We are sending copies of this report to the Honorable William M. Daley,
Secretary of Commerce; the Honorable Kenneth Prewitt, Director of the U.S.
Census Bureau; the Honorable Jacob J. Lew, Director of the Office of
Management and Budget; and other interested parties. Copies will be made
available to others upon request.
If you have any questions on matters discussed in this letter, please
contact me at (202) 512-6240. Other key contributors to this report
include Mark Bird, Cristina Chaplain, Garry Durfey, Richard Hung, and
Aaron Thorne.
*****************
*****************
Randolph C. Hite
Associate Director
Governmentwide and Defense Information Systems
--------------------------------------
/Footnote1/-^The data capture centers are located in Baltimore, Maryland;
Jeffersonville, Indiana; Pomona, California; and Phoenix, Arizona.
/Footnote2/-^The Census short form-three pages with eight questions-will
be delivered to approximately 83 percent of all housing units. It will
ask the respondent to provide information for up to six household
members including information regarding name, age, sex, relationship,
and race. The long form-40 pages with 53 questions-will be delivered to
approximately 17 percent of all housing units. For up to six household
members, it will ask the same questions as the short form as well as
questions on social, economic, financial, and physical characteristics.
/Footnote3/-^The fourth data capture center, located at Census' National
Processing Center in Jeffersonville, Indiana, will be operated by the
bureau.
/Footnote4/-^DCS 2000 is currently undergoing three types of major test
activities: (1) system acceptance testing, which assesses whether the
system performs according to specifications, (2) site acceptance
testing, which assesses whether the system performs correctly at its
deployment site, and (3) operational testing, which assesses whether the
system performs as intended when operated on-site by those expected to
use it.
/Footnote5/-^Two categories of severe defects include emergency-a problem
with no deliverable/procedural workaround available, which blocks
operational use of the product or completion of all or some test cases
or test phases-and critical-a problem with no deliverable/procedural
workaround available, which significantly degrades operational use or
which blocks a top-level test case or test phase from completion.
BRIEFING ON THE DECENNIAL CENSUS
================================
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
*****************
COMMENTS FROM THE DEPARTMENT OF COMMERCE
========================================
*****************
*****************
*****************
*****************
The following are GAO's comments on the Department of Commerce's January
27, 2000, letter responding to a draft of this report.
GAO Comments
1.It was not our intent to imply this. As we discuss throughout the
report, DCS 2000 development, testing, and deployment has been
ongoing on an incremental basis and we clearly identify what
activities have been accomplished to date. Further, we identify
what development, test, and deployment activities remain to be
accomplished, and we characterize these remaining activities as
significant and formidable in light of the short time remaining
before DCS 2000 must be operational. Nevertheless, we have modified
the sentence that the bureau cited to eliminate the possibility of
misinterpretation.
2.We agree that the 16-day average time for resolving all PTRs is not a
good indicator of the time necessary to resolve emergency PTRs
because all PTRs include five different categories of PTRs and each
category receives a different priority, emergency being the most
severe category and the highest priority for resolution. For this
reason, we do not state anywhere in our report that the 16-day
average time needed over the last 6 months (which covered June
through November 1999) to resolve all PTRs is the operative
indicator of the time to resolve the most severe category of PTRs
(emergency). Rather, the operative indicator that we cite in the
report for the time necessary to resolve severe PTRs is 10 days,
which is the average time that the bureau's own data show it has
taken over the last 6 months to resolve both emergency and critical
PTRs. We focused on these two most severe categories of PTRs in our
analysis because the bureau's own definition of emergency and
critical PTRs (emergency PTRs are problems that prevent system
operation and critical PTRs are problems that significantly degrade
system operation) mean that both of these categories of PTRs will
have to be resolved before March 6, 2000, if DCS 2000 is to operate
as intended. Accordingly, we state in the report that "it has been
taking an average of 16 days to resolve all PTRs over the last 6
months and 10 days for severe PTRs."
(511178)
*** End of document. ***