2000 Census: Headquarters Processing System Status and Risks (Letter
Report, 10/17/2000, GAO/GAO-01-1).
The Bureau of the Census is conducting the 2000 decennial census. The
accuracy of this census depends in part on the proper functioning of 10
interrelated information systems, one of which is the Bureau's
headquarters (HQ) processing system. The HQ processing system consists
of 48 applications, all developed internally by the Bureau, that support
a variety of census operations such as updating address files, creating
a file of census responses, and preparing data for tabulation and
dissemination. GAO found that the Bureau lacks effective, mature
software and system development processes to control development of its
HQ processing system applications.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: GAO-01-1
ACCNO: 164196
TITLE: 2000 Census: Headquarters Processing System Status and
Risks
DATE: 10/17/2000
SUBJECT: Census
Information resources management
Computer software verification and validation
Strategic information systems planning
Computer software
Information systems
IDENTIFIER: 2000 Decennial Census
Software Capability Maturity Model
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Testimony. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-01-1
A
Report to the Subcommittee on the Census, Committee on Government Reform,
House of Representatives
October 2000 2000 CENSUS Headquarters Processing System Status and Risks
GAO- 01- 1
Lett er
October 17, 2000 The Honorable Dan Miller Chairman The Honorable Carolyn B.
Maloney Ranking Minority Member Subcommittee on the Census Committee on
Government Reform House of Representatives
As you know, the U. S. Census Bureau is conducting the 2000 decennial
census. The accuracy of this census depends in part on the proper
functioning of 10 interrelated information systems, one of which is the
Bureau's headquarters (HQ) processing system. Given the criticality of this
system, you asked us to (1) identify the nature and status of the HQ
processing system and (2) assess the quality of the system and the risks
facing the Bureau if effective quality controls are not in place.
This report summarizes the information presented at our September 14, 2000,
briefing to your staff. A copy of our briefing is included in appendix I. We
performed our work from July through September 2000 in accordance with
generally accepted government auditing standards.
Results in Brief The HQ processing system consists of 48 applications, all
developed internally by the Bureau, that support a variety of census
operations such
as updating address files, creating a file of census responses, and
preparing data for tabulation and dissemination. The Bureau reported that,
as of July 20, 2000, 31 of the 48 applications were complete and supported
or are supporting census 2000 operations. The Bureau lacks effective, mature
software and system development processes to control development of its HQ
processing system applications. It relies instead on the efforts of
individuals to deliver applications on time and within budget- an approach
that increases the
risk that the applications will not be available when needed and/ or perform
as intended. As a result, the Bureau lacks adequate assurance that the
functions performed by the HQ processing system applications- such as
ensuring accurate and complete address files and identifying the correct
households for enumerators to contact- are properly executed.
Given the short amount of time remaining before the results of the decennial
census will be used, the Bureau will need to take immediate steps to
mitigate the near- term risks it faces with the quality of the applications
that these process weaknesses may have caused. Longer- term improvements
will also be required.
HQ Processing System The HQ processing system comprises 48 applications that
support census Nature and Status 2000, grouped into the following three
functional areas. Address list capture operations consists of 10
applications that refine, update, and edit the Bureau's address files and
database prior to conducting census 2000.
Decennial management controls includes 22 applications that create the
Decennial Response File and control census 2000 operations, such as
identifying households for follow- up. Post- response processing systems
comprises 16 applications that resolve inconsistencies, code handwritten
responses, perform edits, add missing data, and prepare data for tabulation
and dissemination.
According to the Bureau, as of July 20, 2000, 17 applications were complete
and supported census 2000 operations that are complete; 14 applications are
complete and support ongoing census 2000 operations; and 17 applications are
under development to support future census 2000 operations. All of the
applications are developed internally, and are
mainframe- based at the Bureau's computer center in Bowie, Maryland, and its
National Processing Center in Jeffersonville, Indiana. HQ Processing System
The Software Engineering Institute's (SEI) Capability Maturity Model 1 and
our test management guide 2 define effective software/ system development
Quality and Risks
processes. The risk of poor software/ system performance, late delivery, and
cost overruns rises when effective processes are not in place. The Bureau
lacks effective processes to control development of its HQ processing system
applications in three particularly important process areas- requirements
management, risk management, and test management. 1 Capability Maturity
Model SM is a service mark of Carnegie Mellon University, and CMM � is
registered in the U. S. Patent and Trademark Office.
2 Year 2000 Computing Crisis: A Testing Guide (GAO/ AIMD- 10. 1. 21,
November 1998).
Requirements management involves establishing and maintaining an agreement
between the system/ software development organization and the customer (i.
e., users) on the requirements for the software. The practices the Bureau
uses to develop the HQ processing system partially satisfied each of the
four specific criteria for requirements management, including the need for
documented, reviewed, validated, baselined, and
controlled requirements (i. e., the Bureau sometimes- but not always-
implements the requisite management control). Risk management involves
proactively identifying potential problems as opposed to reacting to
problems only after they materialize. However, the practices the Bureau uses
to develop the HQ processing system software did not satisfy each of the
five specific criteria for risk
management, including the need for risk management planning; risk
identification, documentation, and analysis; and risk mitigation strategy
development, tracking, and reporting. Test management entails the planning
and conduct of structured and
disciplined system testing to provide reasonable assurance that systems
perform as intended. The Bureau's test management practices for the HQ
processing system did not satisfy the criteria for documenting testing
policies and procedures; did satisfy the criteria for documenting and
correcting problems resulting from testing; and partially satisfied the
remaining three criteria for planning test activities, documenting test
cases, and reporting test results.
Conclusions The Bureau does not have adequate assurance that the functions
performed by the HQ processing applications, such as having accurate and
complete address files and identifying the correct households for
enumerators to contact, are properly executed. While Bureau management has
implemented some practices to promote HQ processing applications' quality,
the Bureau does not have effective and mature software and system
development processes, such as those specified in SEI's CMM and our test
management guide. Instead, Bureau management is counting on the efforts of
individuals to deliver quality applications on time and within budget. This
approach unnecessarily increases the risk that these applications will not
be available when needed and will not perform as intended. In particular,
the Bureau's approach to testing does not provide the necessary analytical
and documented basis for knowing whether applications will function
correctly.
Recommendations for Correcting its many HQ processing system development
process Executive Action
weaknesses should not be the Bureau's near- term focus. Rather, given the
short amount of time remaining before the decennial census results will be
used, the Bureau needs to take immediate steps to understand the nearterm
risks it faces with the quality (i. e., availability and performance) of HQ
processing system applications that these process weaknesses may have
caused. Complicating this situation is the fact that at this late juncture,
the staff who can quickly and effectively assess and address risks are
essentially the same individuals who are fully engaged in developing the
applications. Thus, we are making both near- term and longer- term
recommendations to the Director of the Census Bureau. We recommend that the
Director of the Census Bureau require the Associate Director for Decennial
Census to
collaborate with the Bureau's chief information officer (CIO) to identify
Decennial Systems and Contracts Management Office (DSCMO) staff who, in
light of workloads and competing priorities, can be diverted on a short-
term basis to work with CIO staff to (1) assess the actual scope and
coverage of testing performed to date and planned on each HQ processing
system application (vis- �- vis application functional, performance, and
interface requirements), (2) assess each application's potential impact on
the quality of the decennial census if it does not perform as intended
(either because it is unavailable or does not function correctly), and (3)
use these assessments of the probability and
impact of HQ processing system application problems to prepare a risk
profile for each application; use the risk profile to establish a plan( s)
for thoroughly testing the
applications on a priority basis, including the use of documented test
conditions and cases that are traceable to requirements, documented test
results, and documented resolution of defects; and execute the plan( s) and
report to the Director on the results.
We further recommend that following the completion of the 2000 decennial
census, that the Director require the Associate Director for Decennial
Census to correct each of the HQ processing system applications development
weaknesses that we identified and require that future development efforts be
conducted in accordance with mature and effective management processes.
Agency Comments and The Bureau's Director provided written comments on a
draft of this report. Our Evaluation
In the comments, the Bureau agreed that its software and system development
procedures do not provide the kind of rigor and discipline advocated in SEI
and GAO guidance and cited in this report. The Bureau also agreed that
decennial census operations could have benefited from earlier implementation
of our recommendations, and it stated that it welcomes the opportunity to
work with GAO in enhancing the Bureau's procedures prior to commencement of
decennial census 2010 operations.
However, the Bureau disagreed with our recommendations that it needs to take
immediate steps to assess and understand the near- term risks it faces with
HQ processing system applications supporting decennial census 2000, and to
thoroughly test these applications on the basis of the priorities
established by this risk assessment. The Bureau's key points supporting its
position are discussed below along with our response to each.
First, the Bureau stated that it believes that procedures are in place to
meet all risk management, system testing, and quality- assurance
requirements. We do not agree. Our report cites the results of our analysis
of these procedures against SEI and GAO guidance, which describe the kind of
rigorous and disciplined processes that mature development organizations
employ. These results show that the Bureau lacks effective processes.
Second, the Bureau stated that since it is impossible to control the nature
of incoming census data, effective and essential testing of HQ
processing system applications occurs once actual processing of census data
is underway. It also stated that to permit assessment and correction of
system problems discovered during actual operations, it builds sufficient
time into its census operations schedule to accomplish this. In
our view, this is not a cost- effective and efficient way to test systems
because it postpones efforts to identify the presence of system problems or
defects to later in the system development cycle when correcting defects is
more difficult and expensive. In contrast, effective test management
processes provide for structuring testing activities in an
incremental fashion, so that problems and defects with system increments can
be discovered earlier, and correcting the problems demands less time and
resources. Moreover, given the Bureau's corporate knowledge and experience
with prior decennial census
operations and the associated incoming data, constructing a meaningful set
of test cases that are representative of incoming data, while challenging,
is not impossible.
Third, the Bureau stated that the 31 HQ processing system applications that
were completed as of July 30, 2000, have successfully performed the
operations for which they were designed. The Bureau further states that
these applications' operational performance is the best indication
that the Bureau's software and system development procedures are in place
and functioning successfully. We do not agree. Without having thoroughly
tested the 31 applications' functionality, we do not understand what basis
the Bureau is using to make this claim of successful operational
performance. The fact that the 31 applications
have been operationally executed and have produced an output is not evidence
of success (i. e., does not provide adequate assurance that the output is
complete and correct). To obtain reasonable assurance, application functions
need to be tested using defined test cases with expected outcomes that are
designed to provide a predetermined level of function coverage. As we state
in the report, the Bureau has not consistently performed such testing.
Fourth, it stated its commitment to meeting the December 31, 2000, deadline
for producing apportionment counts and the April 1, 2001, deadline for
producing other census data, and it stated that diverting staff to
implementing our near- term recommendation at this late stage would
jeopardize its ability to produce census data on time, and would
require changes in the existing schedule for dissemination of the census
2000 counts. It also stated that staff and resources have been appropriately
committed to control the development, and ensure the accuracy, of HQ
processing system applications, and that insufficient funding was provided
for early requirements definition and testing against those requirements. We
do not question the Bureau's commitment. However, the Bureau's chosen course
of action provides for meeting the deadlines using applications that are at
risk of not performing as intended. Therefore, we question the Bureau's
decision to not apply its staff and resources in a way that mitigates the
risk of this
occurring. In light of the risk and uncertainty associated with these
applications due to the Bureau's lack of effective and mature development
processes, we continue to believe, as we recommended, that the prudent step
for the Bureau at this juncture is to quickly, and with minimal investment
of time and resources, understand which applications are at greatest risk
and thus deserving of priority attention, and to take appropriate actions in
light of these risk- based priorities to ensure that applications are
adequately tested.
The full text of the Bureau's comments is reprinted in appendix II.
We are sending copies of this report to the Honorable Norman Y. Mineta,
Secretary of Commerce; the Honorable Kenneth Prewitt, Director of the U. S.
Census Bureau; the Honorable Jacob J. Lew, Director, Office of Management
and Budget; and other interested parties. Copies will also be made available
to others upon request.
Should you have any questions on matters discussed in this report, please
contact me at (202) 512- 6240 or by e- mail at hiter. aimd@ gao. gov. Other
key contributors to this report were Mark Bird, Garry Durfey, Michael
Fruitman, and Richard Hung.
Randolph C. Hite Director, Information Technology
Appendi xes Briefing on Census' Headquarters Processing
Appendi x I
System Bureau of Census Headquarters Processing System
Briefing to the Subcommittee on the Census Staff, Committee on Government
Reform,
House of Representatives September 14, 2000
Overview
Objectives, Scope, and Methodology Background Nature and Status of
Headquarters (HQ) Processing
System Applications Quality of Development Processes for HQ Processing
System Applications and Associated Risks Conclusions and Recommendations
2
Objectives, Scope, and Methodology
Objectives: What is the nature and status of Census' HQ processing
systems? What is the quality of HQ processing systems and what
risks does Census face if effective quality controls are not in place for
developing these systems?
Scope and Methodology: To determine nature and status of systems and
effectiveness of controls, we Identified HQ processing system applications
(purpose,
relationships, development and execution status, environments) developed by
Decennial Systems and Contracts Management Office (DSCMO);
3
Objectives, Scope, and Methodology
Analyzed Census documentation/ oral descriptions of key management controls
used by DSCMO (requirements management, risk management, test management) to
develop these systems as well as test management controls used by user
organizations (Decennial Statistical Studies Division and Decennial
Management Division) and an independent test organization (DSCMO's Beta
Site);
Compared these management controls to recognized best practices, such as
those specified in the Software Engineering Institute's (SEI) Capability
Maturity Model (CMM) 1 and GAO's test management guide; 2
1 Capability Maturity Model SM is a service mark of Carnegie Mellon
University, and CMM � is registered in the U. S. Patent and Trademark
Office. 2 Year 2000 Computing Crisis: A Testing Guide (GAO/ AIMD- 10.1.21,
November 1998).
4
Objectives, Scope, and Methodology
Examined documentation to see how these controls were implemented on one
system application that the Bureau identified as illustrative of current
development practices; and
Assessed risks, if any, in light of management controls in place.
Performed work between July and September 2000 in accordance with generally
accepted government auditing standards.
As agreed with Subcommittee staff, deferred reviewing specific system
applications to determine their quality until after meeting with the
Subcommittee staff to discuss our work to date.
5
Objectives, Scope, and Methodology
Requested comments on a draft of this briefing from the Commerce Department
and the Bureau and incorporated the comments they provided where
appropriate.
6
Background
An accurate census depends in part on the proper functioning of ten
interrelated systems, one of which is HQ Processing. The other nine systems
are listed below and described in appendix I.
Data Capture System (DCS) 2000 Geographic Support System (GSS) Operations
Control System (OCS) 2000 Pre- Appointment Management System/ Automated
Decennial Administrative Management System (PAMS/ ADAMS)
Telephone Questionnaire Assistance and Coverage Edit Follow- Up (TQA/ CEFU)
7
Background
Internet Data Collection/ Internet Questionnaire Assistance (IDC/ IQA)
Accuracy and Coverage Evaluation System (ACE) Management Information System
(MIS) 2000 Data Access and Dissemination System (DADS)
8
Background Simplified Diagram of Decennial Census Systems
Internet IDC Responses
MIS 2000
Enumerator- completed Paper Forms
Census
Headquarters
Census DADS Census
DCS 2000 Data
Processing
Data Data
Users Census
Household- completed
Data
Paper Forms
Enumerator Assignment
Address Telephone
TQA Data
Files A. C. E. Responses
OCS 2000 Enumerator Assignments
Enumerator Personnel Data
PAMS/ GSS ADAMS
9
Nature and Status of HQ Processing High- Level Application Functions
The HQ processing system consists of 48 applications that are grouped into 3
functional areas:
Address List Capture Operations (ALCO) -- 10 applications that include a
series of operations to refine, update, and edit the bureau's Master Address
File (MAF), Decennial Master Address File (DMAF), and Topographically
Integrated Geographic Encoding and Referencing (TIGER) database prior to
conducting the Census.
10
Nature and Status of HQ Processing High- Level Application Functions (cont.)
Decennial Management Controls (DMC) -- 22 applications that collect and
process census data from a variety of sources (e. g. DCS 2000, TQA) to
create the Decennial Response File (DRF) and control subsequent census 2000
operations, such as Non- Response FollowUp (NRFU).
Post Response Processing Systems (PRPS) -- 16 applications that resolve
inconsistencies, code handwritten responses, perform edits, add missing
data, and prepare for tabulation and dissemination.
11
Nature and Status of HQ Processing Simplified Diagram of HQ Processing
Architecture
Geographic Support System MAF TIGER OCS 2000
Internet Responses
IDC Telephone
DMC ALCO Responses
TQA
Enumerator- completed Paper Forms
DMAF DCS 2000
Census Data
DRF Users
Household- completed Paper Forms
PRPS DADS
A. C. E.
HQ Processing
12
Nature and Status of HQ Processing Application Status
All HQ processing system applications are developed internally by the
Bureau.
According to the Bureau's 7/ 20/ 00 status report: Status ALCO DMC PRPS
Applications are complete and supported Census 2000 operations that are
7 10 0 complete. Applications are complete and support
Census 2000 operations that are ongoing. 2 10 2 Applications are under
development to support future Census 2000 operations. 1 2 14
13
Nature and Status of HQ Processing Application Characteristics
Applications are mainframe- based at the Computer Center in Bowie, Maryland
and the National Processing Center in Jeffersonville, Indiana.
Applications are written in the FORTRAN, Pascal, C, and C++ programming
languages.
The Bureau estimates the size of the HQ processing system applications at
about 1,250,000 lines of code.
The HQ processing system applications' status, functions, and relationships
are described and depicted in greater detail in appendix II.
14
Aug- 96 Nov- 96
Feb- 97 May- 97
Aug- 97 Nov- 97
Feb- 98 May- 98
Aug- 98 Status of HQ Processing
Application Use Timeline
Apr 02 Nov 00 Sep 00
15
Nov- 98 Feb- 99
May- 99 Aug- 99
99 Nov- Feb- 00
May- 00 Aug- 00 Nature and PRPS
May 99
DMC
Oct 98
ALCO
Mar 97
Nov- 00 Feb- 01
May- 01 Aug- 01
Nov- 01 Feb- 02
May- 02
Quality of Development Processes and Risks Reported Application Problems
Earlier this year, the Bureau reported two problems associated with the NRFU
HQ processing system application within the DMC functional area. The bureau
devised workarounds to minimize the impact of these problems. The two
reported problems were:
Omission of surnames from address lists used by census workers during the
NRFU operation. The Bureau created supplemental surname lists for census
workers and identified the cost of this workaround as nominal.
Miscalculation of bar code digits for address labels used on census forms as
part of the NRFU operation. Because of the miscalculation, DCS 2000 required
modification before enumerator forms could be processed.
16
Quality of Development Processes and Risks Actions to Address reported
Application Problems
As a result of these problems, the Bureau reports that it has strengthened
its HQ processing application development practices, beginning with the
Coverage Improvement Follow- Up (CIFU) application as identified in the
following table.
Software/ System Development Practice Used for NRFU Used for CIFU
Requirements walkthrough No Yes External review of output against input
files No Yes End user review No Yes Independent external code validation No
Yes Internal software quality review Yes Yes Peer code review Yes Yes
External review of output Yes Yes Post production evaluation Yes Yes Test
deck evaluation No No 1
1 According to Bureau documentation, time does not permit the use of a
formal test deck evaluation for CIFU. However, future software development
efforts will include test deck evaluation.
17
Quality of Development Processes and Risks Effective System Development
Processes
The Clinger- Cohen Act requires establishment of effective IT management
processes. Such processes for managing software/ system development are
defined in various published models and guides, such as SEI's CMM and GAO's
testing guide.
Risk of system/ software development product problems (i. e., poor
performance, late delivery, cost overruns) is higher when system/ software
development processes (i. e., management controls) are ad hoc and chaotic.
Requirements management, risk management, and system/ software testing are
three process areas important to ensuring that systems/ software are
delivered on time, within budget, and perform as intended.
18
Quality of Development Processes and Risks (cont.)
Requirements Management Requirements management involves establishing and
maintaining an agreement between the system/ software development
organization and the customer (i. e., users) on the requirements for the
software project.
Requirements management includes the analysis of business needs and their
translation into documented, validated, and baselined functional,
performance, and interface requirements, as well as formally controlling
changes to the requirements baselines.
Effectively managed system/ software requirements provide the essential
specifications against which systems/ software are built and tested. Without
them, the chances of development efforts not performing as intended and
costing more and taking longer than planned is increased.
19
Quality of Development Processes and Risks (cont.)
Criteria Finding Policies and procedures for requirements
Partially The Bureau has a documented policy and procedures for
management should be documented. satisfied
managing census operational requirements. According to Bureau officials,
they have an undocumented practice to derive detailed requirements for use
by developers. Requirements should be documented. Partially
According to Bureau officials, their undocumented practice satisfied
is to document functional, performance, and interface requirements based on
user- defined business needs, known system interface definitions, and
expected workload estimates. However, they acknowledged that requirements
have not been documented for all applications, and that they are sometimes
documented after the application has been operationally used. For the
application we reviewed, requirements were documented in a memo with e-
mail, other memo attachments, and hand written notes. Requirements should be
reviewed and
Partially According to Bureau officials, their undocumented practice
validated. satisfied
is for a designated official in the developing organization to review and
approve documented requirements. However, they stated that this is not
always done. For the application we reviewed, requirements were approved by
this official. Requirements should be baselined and
Partially The Bureau has a documented procedure for baselining and
changes to requirements should be controlled. satisfied controlling census
operational requirements changes. According to Bureau officials, they have
an undocumented practice to baseline and control detailed requirements for
use by developers. For the application we reviewed, the requirements memo
had a numbering scheme that could be used to identify the baselined version,
however, we found that changes to the detailed requirements baseline were
not controlled.
20
Quality of Development Processes and Risks (cont.)
Risk Management Risk is the possibility of a problem or loss. Risk
management is a proactive approach to avoiding loss or problems before they
occur.
Risk management includes formal identification of potential problems,
analysis of their impacts and probability of occurrence, definition of risk
mitigation plans, accountability for implementing the plans, and tracking
and reporting of progress against the plans.
Without effective risk management, an organization is forced to react to
problems only after they materialize. Such a reactive approach provides
little assurance that projects will meet cost, schedule, and performance
expectations.
21
Quality of Development Processes and Risks (cont.)
Criteria Finding Policies and procedures for risk management
Not The Bureau does not have documented policies and
should be documented. satisfied
procedures for risk management. Instead, the Bureau relies on informal
practices that developers have learned over time and understand. Risk
management activities should be
Not According to Bureau officials, risk management plans are
planned. satisfied
not prepared. For the application we reviewed, we found no evidence of a
risk management plan. Risks should be identified and documented. Not
According to Bureau officials, risks are not systematically satisfied
and proactively identified and they are not documented. For the application
we reviewed, we found no evidence of identified risks. Risks should be
analyzed and appropriate risk
Not According to Bureau officials, their undocumented practice
mitigation strategies developed. satisfied is to take action to address a
risk if one is informally surfaced. However, these actions (the risk
mitigation strategy) are not documented. For the application we reviewed, we
found no evidence of risk action planning/ mitigation strategies.
Implementation of mitigation strategies should
Not According to Bureau officials, progress in addressing
be tracked and reported. satisfied
identified risks may be documented as part of regular project reporting and
tracking reports and meetings. For the application we reviewed, we found no
evidence that implementation of plans for addressing risks were tracked and
reported.
22
Quality of Development Processes and Risks (cont.)
Test Management Complete and thorough system testing is essential to
provide reasonable assurance that systems perform as intended.
To be effective, testing should be planned and conducted in a structured and
disciplined fashion that includes processes to control and direct each
incremental level of testing, including testing of individual software units
or modules, the integration of these units or modules in creating an
application, and the integration of related applications in creating a
system.
Testing processes should include assignment of authority and responsibility
for testing, development of test plans and procedures, execution of plans
and procedures, documentation of results, and correction of problems.
23
Quality of Development Processes and Risks (cont.)
Criteria Finding Policies and procedures for unit, integration,
Not The Bureau does not have documented policies and
and system testing should be documented. satisfied
procedures for unit, integration, and system testing. Instead, the Bureau
relies on practices that the development organization has used over time and
understands. Testing activities should be planned. Partially
According to Bureau officials, various levels of satisfied testing/ checks
occur, but not all applications are subjected
to all levels of testing. The levels are: (1) programmers perform checks/
reviews on their own units or modules of code and the integration of these
units/ modules; (2) an independent test group tests the entire application;
(3) and two user organizations (either the Decennial Management Division or
the Decennial Statistical Studies Division) test the entire application.
Neither the programmers test activities nor the Decennial Statistical
Studies Division tests follow written plans. In contrast, the independent
test group and the Decennial Management Division have written test plans.
For the application we reviewed, we found no test plans for either the
programmer checks/ reviews or the Decennial Statistical Studies Division
tests. The Decennial Management Division had a test plan, and the
independent test group did not perform testing.
24
Quality of Development Processes and Risks (cont.)
Criteria Finding Test cases should be traceable to
Partially According to Bureau officials, test cases are documented for
requirements, documented, and executed. satisfied
the independent test group tests, but since requirements are not always
documented, traceability is not always possible. However, test cases are not
documented for the checks/ reviews performed by each programmer and for the
testing performed by one of the user groups. Instead, the Bureau will either
leave the scope and nature of programmer checking and review to the
discretion of the development organization, or in some cases where time and
resources permit, it will employ a technique referred to as “double
programming” whereby two programmers write code for the same
requirements, compare their results, and address discrepancies. For
applications where user testing is performed, this technique is sometimes
employed, meaning that the user organization writes its own code (i. e.,
“triple programming”) for the same requirements, compares its
results with the results of the developer's executed code, and addresses
discrepancies. User testing can, but does not always, include development of
test cases. For applications where independent testing is performed, test
cases are documented prior to test execution. For the application we
reviewed, we found there were no documented programmer or Decennial
Statistical Studies Division test cases. The Decennial Management Division
had test cases.
25
Quality of Development Processes and Risks (cont.)
Criteria Finding Problems resulting from testing should be
Satisfied According to Bureau officials, problems resulting from testing
documented and corrected.
are documented in e- mail messages and they are corrected. For the
application we reviewed, problems resulting from programmer and Decennial
Management Division testing were documented and the Decennial Statistical
Studies Division reported that there were no problems. Test results should
be reported. Partially
According to Bureau officials, programmer and Decennial satisfied
Statistical Studies Division test results are not documented and reported.
Independent test organization and Decennial Management Division test results
are documented and reported. For the application we reviewed, a programmer
test report was not prepared. A Decennial Management Division test report
was prepared and incorporated a Decennial Statistical Studies Division test
report
26
Quality of Development Processes and Risks (cont.)
In addition to the previously described software/ system practices, after
development is completed the Bureau performs reviews of production data
resulting from executing some of the PRPS applications. These reviews are
conducted by two user organizations (i. e., the Population Division and the
Housing and Household Economic Statistics Division) to check the
reasonableness and consistency with other data sources before approving data
for subsequent processing for creation of census data products.
27
Conclusions and Recommendations
Conclusions The quality (availability and performance) of the
Bureau's HQ processing applications is a function of the effectiveness of
the processes used to build and test the applications. While process
weaknesses do not mean that product problems in fact exist, they are a
predictor. Leading software and system authorities have shown a correlation
between immature processes and late product delivery and poor product
performance. In the Bureau's case, this means that the Bureau does not have
adequate assurance that the functions performed by the HQ processing
applications, such as having accurate and complete address files and
identifying the correct households for enumerators to contact, are properly
executed.
28
Conclusions and Recommendations (cont.)
While Bureau management has implemented some practices to promote HQ
processing applications' quality, the Bureau does not have effective and
mature software and system development processes, such as those specified in
SEI's CMM and GAO's test management guide. Instead, Bureau management is
counting on the heroic efforts of individuals to deliver quality
applications on time and within budget. This approach unnecessarily
increases the risk that these applications will not be available when needed
and will not perform as intended. In particular, the Bureau's approach to
testing does not provide the necessary analytical and documented basis for
knowing whether applications will function correctly.
29
Conclusions and Recommendations (cont.)
Correcting its many HQ processing system development process weaknesses
should not be the Bureau's nearterm focus. Rather, given the short amount of
time remaining before the decennial census results will be used, the Bureau
needs to take immediate steps to understand the near- term risks it faces
with the quality (availability and performance) of HQ processing system
applications that these process weaknesses may have caused. Complicating
this situation is the fact that at this late juncture, the persons that can
quickly and effectively assess and address risks are essentially the same
individuals that are fully engaged in developing the applications.
Accordingly, we are making both near- term and longer- term recommendations
to the Director of the Census Bureau.
30
Conclusions and Recommendations (cont.)
Recommendations We recommend that the Director of the Census Bureau
require the Associate Director for Decennial Census to: Collaborate with the
Bureau's Chief Information
Officer (CIO), to identify DSCMO staff who, in light of workloads and
competing priorities, can be diverted on a short- term basis to work with
CIO staff to:
Assess the actual scope and coverage of testing performed to date and
planned on each HQ processing system application (vis- �- vis application
functional, performance, and interface requirements);
31
Conclusions and Recommendations (cont.)
Assess each application's potential impact on the quality of the decennial
census if it does not perform as intended (either because it is unavailable
or does not function correctly); and
Use these assessments of the probability and impact of HQ processing system
application problems to prepare a risk profile for each application;
Use the risk profile to establish a plan( s) for thoroughly testing the
applications on a priority basis, including the use of documented test
conditions and cases that are traceable to requirements, documented test
results, and documented resolution of defects; and
Execute the plan( s) and report to the Director on the results.
32
Conclusions and Recommendations (cont.)
We further recommend that following the completion of the 2000 decennial
census that the Director require the Associate Director for Decennial Census
to correct each of the HQ processing system applications development
weaknesses that we identified, and require that future development efforts
are conducted in accordance with mature and effective management processes.
33
Appendix I: Description of Decennial Census Systems
System Name Description Geographic Support System (GSS) Provides the basic
census address list, maps, and geographic reference files
for all census programs, including the 2000 decennial census. Pre-
Appointment Management
Supports temporary bureau employee applicant tracking and processing,
System/ Automated Decennial
temporary employee selection records, recruiting reports, personnel and
Administrative Management System
payroll processing, and archiving of historical employment data. (PAMS/
ADAMS) Operations Control System (OCS)
Supports, manages, and controls all field operations for Census 2000. 2000
Management Information System
Provides the official source of management information on Census 2000 and
(MIS) 2000
will provide information on scheduling, progress to date, cost to date
compared to budget, and performance anomalies. Telephone Questionnaire
Assistance
Uses both automated interactive voice response technology and live operator
(TQA) and Coverage Edit Follow- Up
responses to ensure that the public's needs are addressed (TQA). Flags edit
(CEFU)
failures on census forms and sends them to a telephone follow- up operation
where a generic follow- up interview will be conducted (CEFU). Internet Data
Collection/ Internet
Provides respondents, on a limited basis, the ability to complete their
short Questionnaire Assistance (IDC/ IQA) form questionnaire on- line over
the Internet and provides respondents with
answers to questions via the Internet. Accuracy and Coverage Evaluation
Supports a follow- up survey for a representative sample of 300,000 housing
(ACE)
units across the nation. Data Capture System (DCS) 2000 Checks in, digitally
images, and optically reads data from census forms and
converts these data into files that will be transmitted to headquarters for
tabulation and analysis. Data Access and Dissemination
Provides access to census results contained in two primary subsystems,
System (DADS)
American FactFinder and Data Products Production, through the Internet. 34
Appendix II: Description of HQ Processing System Applications
# Application Description Census 2000
Census 2000 Under
Name Development and
Development Development for
Operations Complete and
Future Census Complete
Ongoing 2000 Operations
Operations
1 Decennial Master Controls and tracks census operations. Initially
4
Address File created from the Master Address File (MAF).
(DMAF) Creation 2 Language Form
Responds to requests for foreign language
4
Determination forms.
3 Form Type Identifies the households to receive the long
4
Sampling forms. 4 Surname
Identifies the households for priority
4
Determination processing capability to capture surnames for follow- up
operations. 5 Accuracy and
Selects block clusters with housing unit counts
4
Coverage for A. C. E. sample determination.
Evaluation (A. C. E.) Sample Identification 6 Creation of
Extracts files from the DMAF for creation of
4
Address File addresses, bar codes, and other control
Tapes information for production of census forms.
7 DMAF Updates Updates the DMAF from MAF refresh files. 4
8 Decennial Stores every census response from continuous
4
Response File (daily) updates from census operations.
(DRF) Processing 9 Check- in of
Captures the census ID and reason code and
4
Undeliverable As loads information into the DMAF.
Addressed (UAA) Returns
Note: Development and operational status of applications is as of July 20,
2000. 35
Appendix II: Description of HQ Processing System Applications (cont.)
# Application Description Census 2000
Census 2000 Under
Name Development and
Development Development for
Operations Complete and
Future Census Complete
Ongoing 2000 Operations
Operations
10 Receive Check- in Captures the IDs of mail- back responses and
4
Files of Mailback enumerator response forms for housing units
and Enumerator (not including GQ) and updates the DMAF.
Returns. 11 Receive Data
Receives data capture files from mail responses
4
Capture Files of and enumerator response forms for housing
Mailback and units (not including GQ) and loads the response
Enumerator data into the DRF.
Forms 12 Receive Check- in
Captures the IDs of GQ/ SP forms and updates
4
of Group the DMAF.
Quarters/ Special Place (GQ/ SP) Responses 13 Receive Check- in
Receives daily transfers of TQA responses. 4
of TQA Responses 14 DMAF/ MAF Sends responses that do not have an address
4
Interface identifier to Geography for assignment and
updates the DMAF or the DRF accordingly. 15 Decennial Field
Controls field collection activities at Regional
4
Interface Census Centers (RCCs) and Local Census
(DFI)/ OCS2000 Offices (LCOs).
16 Computer Provides joint check- in/ data capture output for
4
Assisted census internet responses.
Telephone Interview (CATI)/ Internet Interface
36
Appendix II: Description of HQ Processing System Applications (cont.)
# Application Description Census 2000
Census 2000 Under
Name Development and
Development Development for
Operations Complete and
Future Census Complete
Ongoing 2000 Operations
Operations
17 DCS 2000/ DCSC Transfers the data capture files and control
4
Interface information from the data capture centers to
HQ. 18 Address
Identifies new addresses requiring field
4
Verification/ Fraud verification, such as BCF/ TQA responses.
Detection 19 Non- Response
Identifies NRFU universe based on DMAF data. 4
Follow- up (NRFU) Identification 20 Coverage Edit
Selects failed responses, based on coverage
4
Processing edit indicators in the DRF, for coverage edit
follow- up. 21 Large Household
Identifies housing units with more than 6
4
Processing persons from the DMAF for a continuation
follow- up package. 22 Coverage
Identifies, following NRFU, vacant and deleted
4
Improvement housing units, blank mail returns, MAF adds
Follow- up (CIFU) after earlier operations.
37
Appendix II: Description of HQ Processing System Applications (cont.)
# Application Description Census 2000
Census 2000 Under
Name Development and
Development Development for
Operations Complete and
Future Census Complete
Ongoing 2000 Operations
Operations
23 Multiple Response Identifies and flags for removal redundant
4
Processing person and housing unit records. The system uses the Primary
Selection Algorithm (PSA) to unduplicate the DRF. 24 Create Census
Merges the DMAF control file with PSA results
4
Unedited File on the DRF.
(CUF) 25 Edit and
Creates the Census Edited File (CEF) by
4
Imputation for the applying a series of line- item edits to the CUF
100% Data file and imputing for missing data.
26 Service Based Performs statistical estimations based on
4
Enumeration information collected during the SBE
(SBE) Estimation operations.
27 A. C. E. Estimation Not applicable for Census 2000. Preparation 28 A. C.
E. Estimations Automates statistical equations to incorporate
4
the results of the A. C. E. program into the census results. Application of
dual system estimation. 29 Create
Tabulates final unadjusted person counts by
4
Apportionment state for use in computing the assignment of
Counts congressional seats to the states.
30 Estimation Review Not applicable for Census 2000. System 38
Appendix II: Description of HQ Processing System Applications (cont.)
# Application Description Census 2000
Census 2000 Under
Name Development and
Development Development for
Operations Complete and
Future Census Complete
Ongoing 2000 Operations
Operations
31 Disclosure Applies necessary disclosure avoidance
4
Avoidance techniques to insure privacy of the
Processing respondents.
32 Sample Data Defines and produces the weighted counts from
4
Weight Processing the sample data.
33 Edit and Creates the Sample Census Edited File (SCEF)
4
Imputation for the by applying a series of line- item edits to the
Sample Data Sample Census Unedited File (SCUF) file and
imputing for missing data. 34 Compute
Calculates confidence intervals that represent
4
Variances the bounds of the census data/ counts.
35 Tabulation Applies tabulation recodes to convert the data
4
Recoding from raw values to publication values in
preparation for loading into DADS/ American Factfinder System. 36 Geographic
Codes the write- in responses pertaining to
4
Extract Place of Birth (POB), Place of Work (POW) and
Migration (MIG). 37 General Coding Automates process to code the write- in
4
responses (Race, Relationship, Spanish Origin, Ancestry and Language) to
selected questions on the census form and then allows clerical code response
entries. 38 Industry &
Automates process to code the write- in
4
Occupation responses to questions pertaining to industry
(I & O) Coding and occupation on the census form and then
allows clerical code response entries.
39
Appendix II: Description of HQ Processing System Applications (cont.)
# Application Description Census 2000
Census 2000 Under
Name Development and
Development Development for
Operations Complete and
Future Census Complete
Ongoing 2000 Operations
Operations
39 Special Place and Collects all SP and GQ information. All GQ
4
Group Quarters data must be represented on the MAF and
Data System DMAF.
(SPGQDS) 40 Decennial/ Beta
Supports clerical operations conducted in the
4
Site/ NPC Interface NPC from the Beta Site.
41 A. C. E. Housing Not part of HQ processing system. Unit Computer Matching
42 A. C. E. Matching
Not part of HQ processing system. Review & Coding System (MaRCS) Housing
Unit Clerical Matching 43 A. C. E. Post
Not part of HQ processing system. Housing Unit Match Processing 44 A. C. E.
Person/
Not part of HQ processing system. Dual System Estimation (DSE) Computer
Matching 45 A. C. E. MaRCS
Not part of HQ processing system. DSE/ Person Matching 46 A. C. E. Final
Not part of HQ processing system. Housing Unit Matching
40
Appendix II: Description of HQ Processing System Applications (cont.)
# Application Description Census 2000
Census 2000 Under
Name Development and
Development Development for
Operations Complete and
Future Census Complete
Ongoing 2000 Operations
Operations
47 Address Listing Captures over 23 million addresses in address
4
Data Capture listing books. Operation performed by NPC.
48 Address Listing Captures map spot indicator data from the
4
Map Spot address listing books.
Digitizing 49 Block Canvas
Captures city- style address changes from block
4
(BC) Data Capture canvas listings.
50 Postal Casing Not applicable for Census 2000. Check (P. C.) Data Capture
51 Address List
Captures data after local and tribal government
4
Review (ALR98 review and updates of the MAF.
and ALR99) and "New Construction" Data Capture 52 Independent
Captures over 1.9 million addresses from the
4
Listing Book A. C. E. address listing project at NPC.
(A. C. E.) Data Capture 53 Independent
Scans the A. C. E. Address Listing maps to
4
Listing Book provide electronic copies to the housing unit
(A. C. E.) Map matching clerks.
Scanning 54 Update/ Leave
Captures address changes from the U/ L address
4
(U/ L) Address books.
Book and Map Check- in
41
Appendix II: Description of HQ Processing System Applications (cont.)
# Application Description Census 2000
Census 2000 Under
Name Development and
Development Development for
Operations Complete and
Future Census Complete
Ongoing 2000 Operations
Operations
55 List/ Enumerate Captures data from the L/ E address books. 4
(L/ E) Address Listing Book Data Capture 56 NRFU U/ L
Not applicable for Census 2000. Updates Data Capture 57 Quality
Not applicable for Census 2000. Improvement Program (QIP) Address Listing
Data Capture 58 Island Areas
Captures data from the Island Areas address
4
Address Listing listing books. Captures data from
Book and questionnaires via key- from- paper.
Questionnaire Data Capture 59 Group Quarters
Captures GQ data at NPC. 4
Questionnaire Data Capture
Note: Application numbers 41, 42, 43, 44, 45, and 46 are not part of the HQ
processing system but are included to illustrate the relationships between
other HQ processing system applications. Application numbers 27, 30, 50, 56,
and 57 are part of the HQ processing system but are not used in the 2000
decennial system. We did not include these 11 applications in determining
that the HQ processing system consists of 48 applications.
42
Appendix II: Description of HQ Processing System Applications (cont.)
Address List Capture Operations Address Listing
Island Areas Address Data Capture
Listing Book & Island Area
Questionnaire Data Capture Address File
47 58
Block Canvass Island Area
Data Capture Data File
49 Address List Review (ALR98 & ALR99) &
“New Construction”
MAF
Data Capture 51
List/ Enumerate Address Listing Book
Data Capture 55
Update/ Leave
TIGER
Address Book & Map Checkin
54 Address Listing Map Spot Digitizing
48 43
Appendix II: Description of HQ Processing System Applications (cont.)
Decennial Management Controls Language
Form Form
Type Surname
A. C. E. Sample Determination
Sampling Determination Determination
DMAF Creation 2 3 4 5
1
MAF
DMAF Updates 7
Address (Field) DMAF / MAF
Verification Interface
DMAF
18 14
Special Place & Group NRFU Identification
DFI Interface / OCS 2000
Quarters Data System OCS 2000
19 15
39 Receive Check- in of
CIFU Identification Group Quarters/
UAA Check- in Special Place
22 Responses
12 9
Group Quarters Questionnaire
Receive Check- in Data Capture
Files of Mailback & 59 Enumerator Returns
Creation of Address Tapes 10
Address File (printing)
Tapes Receive Data
6 DCS 2000
DCS 2000 / DCSC Capture Files of Interface
Mailback & DRF Processing 17
Enumerator Returns 11 8
IDC Internet Interface
16 Coverage Edit
DRF
Processing Receive Check- in
20 of TQA Responses
TQA 13
44
Appendix II: Description of HQ Processing System Applications (cont.)
Post Response Processing System Multiple
Geographic Response
Extract 36
DRF
Processing 23
General
SCUF
Coding General
(Sample)
SCIF
Coding 37
(100%) 37
Industry & Occupation Create CUF
HCUF
Coding 38
DMAF
24 Sample Data
Edit & Imputation Weight
for the Sample Processing
Data 32 33
A. C. E. Independent A. C. E. Housing
A. C. E MaRCS A. C. E. Post
A. C. E. Person Edit & Imputation
Listing Book Unit Computer
Housing Unit Housing Unit
Computer for the 100%
Data Capture Matching
Clerical Matching Match
Matching 52
41 42 Processing 43 44
Data 25
SCEF
A. C. E. Independent Listing Book
A. C. E. MaRCS Map Scanning
53 Person Clerical
Matching 45
HCEF
A. C. E. Estimations
(Sample) 28
A. C. E. Final Housing Unit
Matching 46
Compute Disclosure
Tabulation Variances
Avoidance Recoding
(Sample) Processing
34 (100%)
31 (100%) 35
A. C. E. Estimations
Disclosure Tabulation
Compute (100%)
28 Avoidance
Recoding Variances
Processing (Sample) HEDF
HDF
(Sample) 31 35
(100%) 34
SBE Estimation
26 Population Division
Create Apportionment
Apportionment
SEDF
Counts 29
45
Comments From the Department of
Appendi x II Commerce
(512029) Lett er
GAO United States General Accounting Office
Page 1 GAO- 01- 1 Census Headquarters Processing System
Contents Letter 3 Appendixes Appendix I: Briefing on Census' Headquarters
Processing
System 10 Appendix II: Comments From the Department of
Commerce 55
Page 2 GAO- 01- 1 Census Headquarters Processing System
Page 3 GAO- 01- 1 Census Headquarters Processing System United States
General Accounting Office
Washington, D. C. 20548 Page 3 GAO- 01- 1 Census Headquarters Processing
System
Page 4 GAO- 01- 1 Census Headquarters Processing System
Page 5 GAO- 01- 1 Census Headquarters Processing System
Page 6 GAO- 01- 1 Census Headquarters Processing System
Page 7 GAO- 01- 1 Census Headquarters Processing System
Page 8 GAO- 01- 1 Census Headquarters Processing System
Page 9 GAO- 01- 1 Census Headquarters Processing System
Page 10 GAO- 01- 1 Census Headquarters Processing System
Appendix I
Appendix I Briefing on Census' Headquarters Processing System
Page 11 GAO- 01- 1 Census Headquarters Processing System
Appendix I Briefing on Census' Headquarters Processing System
Page 12 GAO- 01- 1 Census Headquarters Processing System
Appendix I Briefing on Census' Headquarters Processing System
Page 13 GAO- 01- 1 Census Headquarters Processing System
Appendix I Briefing on Census' Headquarters Processing System
Page 14 GAO- 01- 1 Census Headquarters Processing System
Appendix I Briefing on Census' Headquarters Processing System
Page 15 GAO- 01- 1 Census Headquarters Processing System
Appendix I Briefing on Census' Headquarters Processing System
Page 16 GAO- 01- 1 Census Headquarters Processing System
Appendix I Briefing on Census' Headquarters Processing System
Page 17 GAO- 01- 1 Census Headquarters Processing System
Appendix I Briefing on Census' Headquarters Processing System
Page 18 GAO- 01- 1 Census Headquarters Processing System
Appendix I Briefing on Census' Headquarters Processing System
Page 19 GAO- 01- 1 Census Headquarters Processing System
Appendix I Briefing on Census' Headquarters Processing System
Page 20 GAO- 01- 1 Census Headquarters Processing System
Appendix I Briefing on Census' Headquarters Processing System
Page 21 GAO- 01- 1 Census Headquarters Processing System
Appendix I Briefing on Census' Headquarters Processing System
Page 22 GAO- 01- 1 Census Headquarters Processing System
Appendix I Briefing on Census' Headquarters Processing System
Page 23 GAO- 01- 1 Census Headquarters Processing System
Appendix I Briefing on Census' Headquarters Processing System
Page 24 GAO- 01- 1 Census Headquarters Processing System
Appendix I Briefing on Census' Headquarters Processing System
Page 25 GAO- 01- 1 Census Headquarters Processing System
Appendix I Briefing on Census' Headquarters Processing System
Page 26 GAO- 01- 1 Census Headquarters Processing System
Appendix I Briefing on Census' Headquarters Processing System
Page 27 GAO- 01- 1 Census Headquarters Processing System
Appendix I Briefing on Census' Headquarters Processing System
Page 28 GAO- 01- 1 Census Headquarters Processing System
Appendix I Briefing on Census' Headquarters Processing System
Page 29 GAO- 01- 1 Census Headquarters Processing System
Appendix I Briefing on Census' Headquarters Processing System
Page 30 GAO- 01- 1 Census Headquarters Processing System
Appendix I Briefing on Census' Headquarters Processing System
Page 31 GAO- 01- 1 Census Headquarters Processing System
Appendix I Briefing on Census' Headquarters Processing System
Page 32 GAO- 01- 1 Census Headquarters Processing System
Appendix I Briefing on Census' Headquarters Processing System
Page 33 GAO- 01- 1 Census Headquarters Processing System
Appendix I Briefing on Census' Headquarters Processing System
Page 34 GAO- 01- 1 Census Headquarters Processing System
Appendix I Briefing on Census' Headquarters Processing System
Page 35 GAO- 01- 1 Census Headquarters Processing System
Appendix I Briefing on Census' Headquarters Processing System
Page 36 GAO- 01- 1 Census Headquarters Processing System
Appendix I Briefing on Census' Headquarters Processing System
Page 37 GAO- 01- 1 Census Headquarters Processing System
Appendix I Briefing on Census' Headquarters Processing System
Page 38 GAO- 01- 1 Census Headquarters Processing System
Appendix I Briefing on Census' Headquarters Processing System
Page 39 GAO- 01- 1 Census Headquarters Processing System
Appendix I Briefing on Census' Headquarters Processing System
Page 40 GAO- 01- 1 Census Headquarters Processing System
Appendix I Briefing on Census' Headquarters Processing System
Page 41 GAO- 01- 1 Census Headquarters Processing System
Appendix I Briefing on Census' Headquarters Processing System
Page 42 GAO- 01- 1 Census Headquarters Processing System
Appendix I Briefing on Census' Headquarters Processing System
Page 43 GAO- 01- 1 Census Headquarters Processing System
Appendix I Briefing on Census' Headquarters Processing System
Page 44 GAO- 01- 1 Census Headquarters Processing System
Appendix I Briefing on Census' Headquarters Processing System
Page 45 GAO- 01- 1 Census Headquarters Processing System
Appendix I Briefing on Census' Headquarters Processing System
Page 46 GAO- 01- 1 Census Headquarters Processing System
Appendix I Briefing on Census' Headquarters Processing System
Page 47 GAO- 01- 1 Census Headquarters Processing System
Appendix I Briefing on Census' Headquarters Processing System
Page 48 GAO- 01- 1 Census Headquarters Processing System
Appendix I Briefing on Census' Headquarters Processing System
Page 49 GAO- 01- 1 Census Headquarters Processing System
Appendix I Briefing on Census' Headquarters Processing System
Page 50 GAO- 01- 1 Census Headquarters Processing System
Appendix I Briefing on Census' Headquarters Processing System
Page 51 GAO- 01- 1 Census Headquarters Processing System
Appendix I Briefing on Census' Headquarters Processing System
Page 52 GAO- 01- 1 Census Headquarters Processing System
Appendix I Briefing on Census' Headquarters Processing System
Page 53 GAO- 01- 1 Census Headquarters Processing System
Appendix I Briefing on Census' Headquarters Processing System
Page 54 GAO- 01- 1 Census Headquarters Processing System
Page 55 GAO- 01- 1 Census Headquarters Processing System
Appendix II
Appendix II Comments From the Department of Commerce
Page 56 GAO- 01- 1 Census Headquarters Processing System
Ordering Information The first copy of each GAO report is free. Additional
copies of reports are $2 each. A check or money order should be made out to
the Superintendent of Documents. VISA and MasterCard credit cards are
accepted, also. Orders for 100 or more copies to be mailed to a single
address are discounted 25 percent.
Orders by mail:
U. S. General Accounting Office P. O. Box 37050 Washington, DC 20013
Orders by visiting:
Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U. S. General
Accounting Office Washington, DC
Orders by phone:
(202) 512- 6000 fax: (202) 512- 6061 TDD (202) 512- 2537
Each day, GAO issues a list of newly available reports and testimony. To
receive facsimile copies of the daily list or any list from the past 30
days, please call (202) 512- 6000 using a touchtone phone. A recorded menu
will provide information on how to obtain these lists.
Orders by Internet:
For information on how to access GAO reports on the Internet, send an e-
mail message with “info” in the body to: info@ www. gao. gov or
visit GAO's World Wide Web home page at: http:// www. gao. gov
To Report Fraud, Waste, or Abuse in Federal Programs
Contact one:
Web site: http:// www. gao. gov/ fraudnet/ fraudnet. htm e- mail: fraudnet@
gao. gov 1- 800- 424- 5454 (automated answering system)
United States General Accounting Office Washington, D. C. 20548- 0001
Official Business Penalty for Private Use $300
Address Correction Requested Bulk Rate
Postage & Fees Paid GAO Permit No. GI00
*** End of document. ***