2010 Census: Cost and Design Issues Need to Be Addressed Soon
(15-JAN-04, GAO-04-37).
The key to a successful census is meticulous planning as it helps
ensure greater cost-effectiveness. However, the 2000 and previous
censuses have been marked by poor planning, which unnecessarily
raised the costs and risks of those efforts. GAO was asked to (1)
review the U.S. Census Bureau's (Bureau) current plans for 2010
and whether they might address shortcomings of the 2000 Census,
(2) analyze the Bureau's cost estimates, and (3) review the rigor
of the Bureau's 2010 planning process.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-04-37
ACCNO: A09094
TITLE: 2010 Census: Cost and Design Issues Need to Be Addressed
Soon
DATE: 01/15/2004
SUBJECT: Budget obligations
Census
Cost analysis
Cost control
Data collection
Data integrity
Future budget projections
Planning
Statistical data
Statistical methods
2000 Decennial Census
2010 Decennial Census
Census Bureau Accuracy and Coverage
Evaluation Program
Census Bureau American Community Survey
Census Bureau Master Address File
Census Bureau Topologically Integrated
Geographic Encoding and Referencing
System
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-04-37
United States General Accounting Office
GAO
Report to Congressional Requesters
January 2004
2010 CENSUS
Cost and Design Issues Need to Be Addressed Soon
a
GAO-04-37
Highlights of GAO-04-37, a report to congressional requesters
The key to a successful census is meticulous planning as it helps ensure
greater cost-effectiveness. However, the 2000 and previous censuses have
been marked by poor planning, which unnecessarily raised the costs and
risks of those efforts. GAO was asked to (1) review the U.S. Census
Bureau's (Bureau) current plans for 2010 and whether they might address
shortcomings of the 2000 Census, (2) analyze the Bureau's cost estimates,
and (3) review the rigor of the Bureau's 2010 planning process.
GAO recommends that the Secretary of Commerce direct the Bureau to combine
information on goals, costs, and other key planning elements into a single
integrated project plan that is updated as needed. The Bureau should also
study the feasibility of procedures that could allow it to adjust census
results for those purposes where it is both legal and appropriate to do
so.
Further, the Director of the Office of Management and Budget should ensure
that the Bureau take steps to improve the transparency of census life
cycle costs. In commenting on a draft of this report, both agencies
disagreed with our interpretation of key findings. They also disagreed
with our recommendations. We believe our findings and recommendations
still apply.
www.gao.gov/cgi-bin/getrpt?GAO-04-37.
To view the full product, including the scope and methodology, click on
the link above. For more information, contact Patricia Dalton at (202)
512-6806 or [email protected].
January 2004
2010 CENSUS
Cost and Design Issues Need to Be Addressed Soon
While preparations for the 2010 Census appear to be further along compared
to a similar point prior to the 2000 Census, cost and design information
had to be pieced together from various documents. The Bureau's plans also
lack a substantial amount of supporting analysis, budgetary transparency,
and other information that made it difficult to verify the Bureau's
assertions concerning the costs and benefits of its proposed approach.
Further, unlike in previous censuses, the Bureau does not intend to
develop coverage measurement procedures that would allow it to adjust
census data for certain purposes. Although its experience in 2000 shows
that its coverage measurement methodology needs improvement, GAO believes
the Bureau should have researched alternative approaches more thoroughly
and disclosed the results of its research before making a decision. In
designing the 2010 Census, the Bureau hoped to address several
shortcomings of the 2000 enumeration, namely to (1) increase the relevance
and timeliness of data, (2) reduce operational risk, (3) increase coverage
and accuracy, and (4) contain costs. To achieve these goals, three
components-all new operations-are key to the Bureau's plans for 2010. They
include enhancing procedures for building the census address list and
associated maps, replacing the census long-form questionnaire with a more
frequent sample survey, and conducting a short-form-only census.
Census Costs Are Increasing (Average Cost per Housing Unit in Constant
Fiscal 2000 Dollars)
The Bureau's approach has the potential to achieve the first three goals,
but reducing operational risk could prove to be more difficult as each of
the three components actually introduces new risks. The Bureau will also
be challenged to control the cost of the 2010 Census, now estimated at
over $11 billion. The current budget reporting process masks the long-term
costs of the census, most of which will be incurred in 2010; making it
difficult for Congress to monitor the Bureau's planned expenditures.
Certain actions by the Office of Management and Budget could produce
greater fiscal transparency, and thus help inform congressional
deliberations on how to best balance the need for an accurate census, with
the need to ensure a reasonable cost for this endeavor.
Contents
Letter
Results in Brief
Background
Scope and Methodology
2010 Design Has Potential, but Introduces New Risks
Escalating Census Costs Call for Transparent Budget Data and
Better Reporting 2010 Planning Process Is Better Than Past Efforts, but
Rigor Is
Uneven Conclusions Recommendations for Executive Action Agency Comments
and Our Evaluation
1 2 5 7 8
17
25 31 33 34
Appendixes
Appendix I: Comments from the Department of Commerce 41 Appendix II:
Comments from the Office of Management and Budget 45 Appendix III: 2010
Census Planning Organization 47
Tables Table 1: The Three-Legged Stool Strategy Could Help the Bureau
Progress toward Its Key Goals for the 2010 Census 10
Table 2: Projected Costs from June 2001 Estimate of
Reengineering
the 2010 Census 24
Figures Figure 1:
Figure 2:
Figure 3: Figure 4:
Figure 5:
Decennial Census Costs from 1970 through 2010
(Projected) in Constant 2000 Dollars 18
Decennial Census Average Cost per Housing Unit (in
Constant Fiscal Year 2000 Dollars) 19
2000 Census Life Cycle Costs by Fiscal Year 20
Bureau's June 2001 Estimating Methodology for Census
2010 Life Cycle Costs 23
Decennial Management Division's Organizational
Structure for Planning the 2010 Census 48
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
A
United States General Accounting Office Washington, D.C. 20548
January 15, 2004
The Honorable Wm. Lacy Clay
Ranking Minority Member
Subcommittee on Technology, Information Policy, Intergovernmental
Relations and the Census
Committee on Government Reform
House of Representatives
The Honorable Danny K. Davis
The Honorable Charles A. Gonzales
The Honorable Carolyn B. Maloney
House of Representatives
In designing the 2010 Census, the U.S. Census Bureau (Bureau) faces the
daunting challenge of cost effectively counting a population that is
growing
steadily larger, more diverse, and increasingly difficult to enumerate,
with a
design that meets the sometimes competing requirements and interests of a
vast spectrum of stakeholders that include Congress, government agencies,
the statistical community, advocacy groups, and the public.
Planning a successful census-one that is acceptable to Congress and
other stakeholders and achieves desired outcomesrequires the
Bureau to
assess the lessons learned from past enumerations, identify initiatives
that
show promise for producing a better census while controlling costs, test
the feasibility of these initiatives, and convince stakeholders of the
value of
the proposed plans.1 However, the Bureau's planning efforts for the 2000
and earlier censuses contained a number of weaknesses that unnecessarily
increased the cost and risk of those efforts.
In January 2003, we named the 2010 Census a major performance and
accountability challenge because of our growing concern over the
numerous obstacles to a cost-effective head count. Already, current
estimates of the cost of the 2010 Census exceed $11 billion. You asked us
to examine the decision-making process for the 2010 Census. As agreed
with your offices, our objectives for this report were to review the
Bureau's
current plans for the 2010 Census and the extent to which they might
1 See, for example, U.S. General Accounting Office, 2000 Census: Progress
Made on Design, but Risks Remain, GAO/GGD-97-142 (Washington, D.C.: July
14, 1997).
address shortcomings with the 2000 Census, analyze the Bureau's cost
estimates, and assess the rigor of the Bureau's 2010 planning process.
Results in Brief The Bureau's preparations for the 2010 Census appear to
be further along than at a similar point during the planning cycle for the
2000 Census. Still, considerable risks and uncertainties lie ahead. This
does not necessarily mean that the Bureau's design is flawed. Indeed, the
difficulties in conducting a successful head count call for the Bureau to
consider bold and innovative initiatives, and these are not risk free.
However, the uncertainties could jeopardize a cost-effective head count if
not addressed soon.
In designing the 2010 Census, the Bureau had four goals in mind, based on
its desire to address shortcomings with the 2000 enumeration: (1) increase
the relevance and timeliness of data, (2) reduce operational risk, (3)
increase coverage and accuracy, and (4) contain costs. To achieve these
goals, three components-all new operations-are key to the Bureau's plans
for 2010:
o enhancing procedures for building its address list, known as the Master
Address File (MAF), and its associated geographic information system,
called the Topologically Integrated Geographic Encoding and Referencing
(TIGER) database;2
o replacing the census long-form questionnaire with the American
Community Survey (ACS); and
o conducting a short-form-only decennial census supported by early
research and testing.
The Bureau's approach has the potential to achieve all but its goal to
reduce operational risk. However, this is no minor shortcoming as the
Bureau's 2010 design actually introduces new risks as the Bureau will be
challenged to (1) secure early congressional agreement on its design, (2)
resolve methodological and other uncertainties surrounding ACS, and (3)
successfully implement all three components of the Bureau's design.
Because the three components are interrelated (e.g., the ACS is critical
to a
2 TIGER is a registered trademark of the U.S. Census Bureau.
short-form-only census), failure to successfully carry out any one
component of the plan could doom the success of the other two, thereby
reducing the overall cost-effectiveness of the census.
A fourth challenge will be slowing the mushrooming cost of the 2010
Census, now estimated at more than $11.3 billion in current dollars. Put
another way, it will cost the nation an estimated $72 to count each
household in 2010, compared to $56 in 2000 and $13 in 1970 (in constant
2000 dollars), in large part because it is getting increasingly difficult
to locate people and get them to participate in the census. The growing
cost of the head count, at a time when the nation is facing historic
budget deficits, highlights the importance of congressional deliberations
on the extent to which each additional dollar spent on the census results
in better data, as well as how best to balance the need for a complete
count, with the need to ensure the cost of a complete count does not
become unreasonable.
In a departure from previous decennial censuses, the Bureau has decided
against developing coverage measurement procedures that would allow it to
adjust 2010 Census data for certain purposes. Although its experience
during the 2000 Census shows that its approach to measuring coverage needs
to be improved, we believe the Bureau should have researched alternative
methods more thoroughly and disclosed the results of its research before
making its decision.
Current budget reporting masks the long-term cost implications of the
census, and this could hamper or delay discussions concerning cost
containment alternatives because annual budget requests and information
provided to Congress early in the decennial life cycle do not reflect the
full cash consequences of the annual spending. Most of the early spending
will be on planning and development costs, but most of the expenses will
be incurred in 2010. As Congress begins funding the 2010 Census at
relatively low levels, it will have implicitly accepted a future spike in
costs without requiring the Bureau to define more clearly what those costs
might be, why they are justified, and what alternatives might exist.
Greater fiscal transparency, including more robust information on the
Bureau's underlying budget assumptions, could help policymakers control
escalating census costs.
In some respects, the Bureau's planning for the 2010 Census appears to be
more rigorous than that for the 2000 Census. For example, the Bureau has
involved experienced staff in the design process, organized multidivision
planning groups, assigned roles and responsibilities, and taken steps to
address issues such as the attrition of key experienced staff. Still,
other aspects of a rigorous planning process were lacking. They include
(1) a comprehensive project plan with information on operational
milestones, measurable performance goals, and costs and (2) productivity
and other data on the results of key operations from the 2000 Census to
inform lessons learned for 2010.
To improve the 2010 Census planning process, we recommend that the
Secretary of Commerce direct the Bureau to consolidate information on
goals, performance measures, costs, and other key planning elements into a
single project plan that would be updated as needed. Further, to help
ensure the nation has at its disposal the best possible data should there
be problems with the quality of the census numbers, the Secretary should
also direct the Bureau, with input from both majority and minority parties
in the Senate and House of Representatives, to research the feasibility of
procedures that could allow the Bureau to adjust census results for those
purposes for which it is both legal and appropriate to do so and, if found
to be feasible, test those procedures during the 2006 census test and 2008
census dress rehearsal.
In addition, to highlight for Congress the long-term fiscal exposure
associated with the census, signal emerging cost issues, and prompt
congressional deliberations on affordability and trade-offs, we recommend
that the Director of the Office of Management and Budget (OMB) ensure that
the Bureau take steps to increase the transparency of census life cycle
costs. Specific steps include displaying such information as updated
estimates of census life cycle costs and the amount of money the Bureau
expects to spend in each year of the cycle as a note in the Program and
Financing schedule of the President's budget. The information should also
explain any material changes from previous budgets, the sensitivity of the
cost figures to specific assumptions, and the likelihood of those
assumptions.
The Secretary of Commerce forwarded written comments from the U.S. Census
Bureau on a draft of this report. The comments are reprinted in appendix
I. The Bureau agreed that to bring closure on issues relevant to the 2010
Census, it was important to have informed discussions with stakeholders,
especially Congress, but disagreed with the recommendations we believe are
necessary for such discussions to be fruitful. However, based on our
evaluation of the Bureau's comments, we believe all of our recommendations
still apply.
For example, the Bureau took exception to our recommendation to improve
the rigor of its planning process by developing a consolidated project
plan, noting that existing documents already provide this information. We
believe our recommendation is still pertinent because, while we agree with
the Bureau that some of this information is already available, it is
disjointed and one can only obtain it by piecing together several
different documents, such as the Bureau's budget submission and strategic
plan. And even then, it would still lack such important information as
performance goals.
The Director of OMB also prepared written comments on a draft of this
report that are reprinted in appendix II. OMB shared our view that the
costs and risks associated with the 2010 Census must be carefully
monitored and evaluated throughout the decade. OMB also agreed that it is
essential to understand the key cost drivers and said that it is working
with the Census Bureau to ensure the Bureau develops high-quality,
transparent life cycle cost estimates. However, OMB disagreed with our
recommendation that it highlight for Congress the long-term fiscal
exposure associated with the census, noting that its existing internal
procedures are sufficient. Nevertheless, we do not believe the information
OMB currently reports to Congress is sufficiently timely or detailed to
provide the level of transparency needed for effective congressional
oversight and cost control, and thus stand by our recommendation.
Background Thorough and comprehensive planning and preparation are crucial
to the ultimate cost-effectiveness of any large, long-term project,
particularly one with the scope, magnitude, and immutable deadlines of the
decennial census. Indeed, the Bureau's past experience has shown that the
lack of proper planning can increase the costs and risks of downstream
operations. Moreover, sound planning is critical to obtaining
congressional support and funding because it helps demonstrate that the
Bureau has chosen the optimal design given various trade-offs and
constraints and that it will effectively manage operations and control
costs.
However, Congress, GAO, the Department of Commerce Inspector General, and
even the Bureau itself have noted how the 2000 Census was marked by poor
planning, which unnecessarily added to the cost, risk, and controversy of
the last national head count. For example, our earlier work, and that of
the Department of Commerce Inspector General, reported that in planning
the 2000 Census, the Bureau, among other shortcomings,
o did not involve key operations staff in the initial design phases;
o did not translate key performance goals into operational, measurable
terms that could be used as a basis for planning;
o did not develop and document a design until mid-decade; and
o initially failed to provide sufficient data to stakeholders on the
likely effects of its initiatives for addressing the key goals for the
census- reduced costs and improved accuracy and equity.
Planning weaknesses were not limited to the 2000 Census. In fact, a
variety of problems plagued the planning of the 1990 Census.
To help prevent the Bureau from repeating those mistakes as it plans the
2010 Census, in our October 2002 report, we recommended that the Secretary
of Commerce direct the Bureau to provide comprehensive information backed
by supporting documentation in its future funding requests for planning
and development activities, including, but not limited to,
o specific performance goals for the 2010 Census and information on how
the Bureau's programs would contribute to those goals;
o detailed information on program feasibility, priorities, and potential
risks;
o key implementation issues and decision milestones; and
o performance measures.3
The consequences of a poorly planned census are high given the billions of
dollars spent on the enterprise and the importance of collecting quality
data. The Constitution requires a census as a basis for apportioning seats
in the House of Representatives. Census data are also used to redraw
congressional districts, allocate billions of dollars in federal
assistance to state and local governments, and provide information for
many other public and private sector purposes.
3 U.S. General Accounting Office, 2000 Census: Lessons Learned for
Planning a More Cost-Effective 2010 Census, GAO-03-40 (Washington, D.C.:
Oct. 31, 2002).
Scope and Methodology
As agreed with your offices, our objectives for this report were to review
the Bureau's current plans for the 2010 Census and the extent to which
they might address shortcomings with the 2000 Census, analyze the Bureau's
cost estimates, and assess the rigor of the Bureau's 2010 planning
process.
To achieve these three objectives, we interviewed officials from the
Bureau's Decennial Management Division and other units involved with
planning the 2010 Census. We also reviewed relevant design and budget
documents as well as our prior work and that of the Department of Commerce
Inspector General, on planning the 2000 and earlier censuses.4 We also
reviewed reports by the National Academy of Sciences on planning the 2010
Census.5 We did not independently verify the cost information the Bureau
provided. To help determine the key elements for successful project
planning, we reviewed a number of guides to project management and
business process reengineering. 6
Our work was performed from January through September 2003 in accordance
with generally accepted government auditing standards. We requested
comments on a draft of this report from the Secretary of Commerce and the
Director of the Office of Management and Budget. On November 6, 2003, we
received the Secretary's written comments on the draft (see app. I). On
October 14, 2003, the Director of OMB forwarded OMB's comments on the
draft (see app. II). They are addressed in the Agency Comments and Our
Evaluation section of this report.
4 See, for example, U.S. General Accounting Office, 2000 Census: Progress
Made on Design, but Risks Remain, GAO/GGD-97-142 (Washington, D.C.: July
14, 1997).
5 National Research Council, Planning the 2010 Census: Second Interim
Report (Washington, D.C.: July 22, 2003).
6 The guides include: A Guide to the Project Management Body of Knowledge,
Project Management Institute Standards Committee, 1996.
Project Management for Mission Critical Systems, a Handbook for Government
Executives, Information Technology Resource Board, April 5, 2001.
Capability, Maturity Model Integration Project Planning Guide, Carnegie
Mellon Software Engineering Institute, March 2001.
U.S. General Accounting Office, Business Process Reengineering Assessment
Guide, GAO/AIMD-10.1.15 (Washington, D.C.: May 1997).
2010 Design Has In designing the 2010 Census, Bureau officials had four
principal goals in
mind: (1) increase the relevance and timeliness of census long-form
data,Potential, but (2) reduce operational risk, (3) increase the coverage
and accuracy of the Introduces New Risks census, and (4) contain costs.
The goals were a direct response to
problems the Bureau experienced in conducting the 2000 Census, such as
untimely long-form data; inaccurate maps; coverage difficulties; and
expensive, labor-intensive, and paper-laden field data collection.
Census Reforms Have Three Key Components
Modernizing and Enhancing the MAF/TIGER Address File and Geographic
Database
The Bureau recognized that its traditional approach to counting the
population was insufficient for meeting these four objectives. In its
place, the Bureau developed what it believes is a paradigm shift to taking
the census, basing its reform efforts on three interrelated components the
Bureau refers to as a "three-legged stool."
The mainstay of a successful census is an accurate address list and its
associated maps. The Bureau uses MAF and TIGER to provide (1) maps for
field operations and data reference, (2) the geographic location of every
structure, (3) address lists for the decennial census, and (4) names and
codes of entities for data tabulation and data for use by the commercial
geographic information systems industry.
The Bureau's experience in conducting the 2000 Census highlighted the need
to update and modernize MAF/TIGER prior to 2010. For example, the
centerlines of streets in TIGER did not accurately reflect their true
geographic locations, which could cause houses to be placed in the wrong
census blocks. Also, according to the Bureau, the 1980s software used to
develop TIGER is now outdated and cumbersome to update. To fix these and
other problems, the Bureau launched the MAF/TIGER Enhancements Program
(MTEP) as part of the 2010 Census modernization efforts. Its objectives
include
o correcting the locations of each MAF address, street, and other map
features;
o developing and deploying a new MAF/TIGER processing environment;
o expanding and encouraging geographic intergovernmental partnership
programs;
Implementing ACS to Collect Long-Form Data on an Ongoing, Annual Basis
Developing, Testing, and Implementing a Short-Form-Only Census; Taking
Advantage of the Benefits Gained by the First Two Efforts
o implementing the Community Address Updating System (CAUS), an
initiative to partner with local governments to update MAF data; and
o initiating quality assurance evaluation activities.
The Bureau estimates the total cost for these five objectives to be $536
million. According to Bureau officials, while some elements of MAF will be
improved as part of the overall MAF/TIGER enhancements program, the
primary focus of the effort is on TIGER modernization and data correction.
This modernization program will not reengineer the MAF process.
ACS is intended to be a monthly survey of 250,000 households that would
replace the long form used in past decennial censuses. According to the
Bureau, the benefits of ACS include (1) more timely long-form data at
detailed geographic levels that would be as accurate as subnational annual
data from existing surveys, such as the Current Population and American
Housing Surveys, and (2) the ability to improve the accuracy of the
decennial census population counts by eliminating the long-form
questionnaire.
The ACS data will be published annually for geographic areas with
populations over 65,000; as a 3-year average for geographic areas with
populations of 20,000 to 65,000; and as a 5-year average for geographic
areas with populations under 20,000. According to the Bureau, because of
the larger sample size associated with long-form data, the annual and
3year average data will be significantly less accurate than the long-form
data. The 5-year data would be about as accurate as the long form.
The Bureau believes that eliminating the long form will result in a number
of benefits to decennial data collection and general field operations. For
example, according to the Bureau, the reduction in paper will enable it to
process data with three data capture centers instead of the four centers
used during Census 2000. The Bureau also would not need as many local
census offices, thereby allowing it to reduce the rolls of clerical and
administrative staff.
According to the Bureau, a short-form-only census also allows the Bureau
to use such technology as handheld mobile computing devices so that
enumerators can locate and update data on housing units, help conduct
interviews, transmit data directly to the data capture centers, and
receive regularly updated field assignments. The devices will be linked to
the
satellite-based Global Positioning System (GPS) to enable field workers to
locate addresses more precisely and efficiently. The Bureau also plans to
incorporate changes that are not dependent on ACS and MAF/TIGER. For
example, the Bureau plans to expand the respondents' ability to complete
their questionnaires via the Internet.
2010 Design Is Linked to As shown in table 1, the Bureau's three-legged
stool strategy is generally Goals, but Challenges aligned with three of
its four key goals for the 2010 Census and, if Remain successful, could
put the Bureau on track toward achieving them.
Table 1: The Three-Legged Stool Strategy Could Help the Bureau Progress
toward Its Key Goals for the 2010 Census
Goal Strategy for achieving goal Rationale
Increase the relevance Implement ACS ACS, if implemented nationwide, is to
and replace the long form and provide
timeliness of census annual data for areas with
long-form populations over 65,000 and multiyear
data averages for smaller geographic
areas. This compares to the long
form's once-every-10-year data.
Reduce operational Early planning and The Bureau believes that testing
risk testing, a design early in the decade will
identify design flaws and allow
short-form-only the Bureau to correct them early
in the
process. Also, a short-form-only
census questionnaire census should be easier to
implement
since Bureau staff will only have
to plan systems for one type of
form
and handle much less paper.
Increase coverage The Bureau believes that a
and Short-form-only census, short-form-only census will
increase the
mail response rate by 8
MAF/TIGER enhancements percentage points because,
accuracy historically, more
people have responded to the
short form than the long form,
and the
Bureau has found that the
quality of the data is better
when supplied
directly by respondents. The
Bureau also plans to do a
targeted
second mailing to nonresponders
that should also boost the
response
rate. These efforts combined
with other planned improvements,
such
as the MAF/TIGER enhancements,
would improve overall accuracy
and coverage.
Contain costs Short-form-only census The higher response rates produced by
the short form will reduce the Bureau's nonresponse follow-up workload,
one of the largest drivers of census costs. The Bureau also believes that
a short-form-only census will result in less paper, and thus enable the
Bureau to get by with three rather than four data capture centers (as was
the case in 2000) and fewer local census offices.
Source: GAO analysis of U.S. Census Bureau data.
Less clear is how the Bureau will achieve its goal of reducing operational
risk using its current plan. Although the Bureau's position that early
testing will enable it to identify and correct flaws is both a common
sense business practice and supported by its past experience (assuming its
testing
program is adequately designed), as described below, the operational and
other hurdles associated with successfully implementing the three-legged
stool actually introduce new risks and challenges. This does not
necessarily mean that the Bureau's design is flawed. To the contrary, the
obstacles to a cost-effective head count call for the Bureau to consider
bold and innovative initiatives, and these are not risk free.
At the same time, given the enormity of the census and all of its
complexities, the three-legged stool by itself will not automatically
guarantee the successful accomplishment of the Bureau's goals. Our work on
transforming agencies into high-performing organizations has underscored
the importance of an agency's leadership and core business practices.
Critical success factors include, among others, effective communication
strategies to ensure coordination, synergy, and integration; strategic
planning; aligning the agency's organization to be consistent with the
goals and objectives established in the strategic plan; and effective
performance, financial, acquisition, and information technology
management. In all, the Bureau faces at least three key challenges.
Challenge #1: Securing Among the more significant challenges the Bureau
faces is securing
Congressional Approval congressional approval for its proposed approach.
As we noted in our January 2003 performance and accountability report,
congressional support for the 2010 design is necessary to ensure adequate
planning, testing, and funding levels.7 Conversely, the lack of an
agreed-upon design raises the risk that basic design elements might change
in the years ahead, while the opportunities to test those changes and
integrate them with other operations will diminish. In other words, in
order for the Bureau to conduct proper planning and development
activities, the basic design of the 2010 Census needs to be stable.
According to the Bureau, a go/no-go decision on key aspects of the
design-a short-form-only census and replacing the long form with ACS- will
need to be made around 2006. Bureau officials told us that if ACS were
dropped after 2007, the Bureau would not be able to reinstate the long
form with the short form in 2010 because of logistical obstacles. They
noted that the Bureau is already testing the short-form-only census and,
in late-2005 or early-2006, expects to sign a contract for data capture
operations. If the
7 U.S. General Accounting Office, Major Management Challenges and Program
Risks: Department of Commerce, GAO-03-97 (Washington, D.C.: January 2003).
Bureau had to revert to a long-form census at that point, it would add
significant risks and costs to the 2010 Census.
During the 2000 Census, the lack of an agreement between the
administration and Congress on the fundamental design-and particularly,
the Bureau's planned use of sampling-increased the likelihood of an
unsuccessful head count and was one of the principal reasons why, in 1997,
we designated the 2000 Census a high-risk area. Members of Congress
questioned the use of sampling and estimation for legal and methodological
reasons. Contributing to Congress's skepticism was the Bureau's failure to
provide sufficiently detailed data on the effects of its proposed
approach. Although the U.S. Supreme Court settled the dispute in January
1999,8 the Bureau ultimately wound up having to plan for both a
"traditional" census and one involving sampling, which added to the costs
and risks of the 2000 decennial census.
To help secure congressional support for its 2010 reform efforts, it will
be important for the Bureau to convincingly demonstrate that it has chosen
the optimum design given various resource and other constraints and that
it will effectively manage operations and costs. A critical first step in
this regard is to have comprehensive and transparent information that lays
out the specifics of the Bureau's plans, explains their benefits, and
supports assumptions. However, as discussed more fully in the next
section, while the Bureau's planning for the 2010 Census has improved
compared with its efforts for the 2000 Census, certain information gaps
remain. For example, the Bureau's most recent budget submissions have not
included complete life cycle cost estimates that could enable Congress to
make more informed decisions about the cost implications of the
three-legged stool design, including ACS.
Challenge #2: Resolving Most of the reforms, savings, and improvements in
accuracy the Bureau
Uncertainties That Surround anticipates will not be possible unless it
conducts a short-form-only census.
ACS However, the Bureau's planned replacement for the long form, ACS,
faces methodological and other questions that need to be resolved soon.9
Consequently, the Bureau is taking a significant risk by pinning the
success
8 Department of Commerce vs. United States House of Representatives, 525
U.S. 316 (1999).
9 For a more complete discussion of ACS, see U.S. General Accounting
Office, The American Community Survey: Accuracy and Timeliness Issues,
GAO-02-956R (Washington, D.C.: Sept. 30, 2002).
of its reform efforts largely on a survey that may not be an adequate
replacement for the long form. The Bureau believes that without ACS, it
will need to repeat the Census 2000 design.
One methodological question is whether to administer ACS as a mandatory or
voluntary survey. Under the Bureau's current approach, survey recipients
will be legally required to respond to ACS. However, in response to
congressional concerns that a mandatory survey is intrusive, the Bureau
tested conducting ACS as a voluntary survey. Based on the results of the
test, the Bureau estimates that a voluntary survey could produce a
response rate around 4.2 percentage points lower than a mandatory survey.
The Bureau estimates that costs would increase by $59.2 million per year
to maintain the same level of reliability achieved from a mandatory
survey.
Moreover, the Bureau's efforts to ensure that ACS data will serve as a
satisfactory replacement for the long-form data are not yet complete.
Among the remaining issues, most of which are critical to the reliability
of the small geographic area ACS data, are the following:
1.
2.
3.
4.
Benchmarking ACS data or small geographic areas to the population counts
and characteristics from the 2010 short form.
Inconsistency of ACS residency rules-which determine the geographic area
in which a person is supposed to be counted-with those used for the
census.
Consistency of ACS data with long-form data from the 2000 Census.
For multiyear averages of ACS data for small areas, consistency with
annual ACS data for larger areas andutilizationas a measure of change.
Challenge #3: Successfully Each leg of the Bureau's three-legged stool is
dependent on the other; that
Implementing Each Leg of the is, the implementation of one leg allows the
other two legs to operate
Bureau's Design successfully. For example, ACS is facilitated by first
updating the MAF/TIGER database for the ACS sample. Similarly, as noted
above, the Bureau's plan to conduct a short-form-only enumeration depends
on ACS. Consequently, the Bureau's design assumes that by 2008, (1) ACS
will be in place nationwide and producing data, (2) a GPS-aligned and
modernized MAF/TIGER will be available, and (3) all reengineering efforts
will be complete to allow for a true dress rehearsal. Completing any one
of these tasks would be a considerable undertaking; for 2010, the Bureau
plans to develop, refine, and integrate all three in the space of just a
few years.
Moreover, the Bureau has no contingency plans other than to revert to a
"traditional" census. According to the Bureau, while the failure of any
one leg would not doom the census, it could jeopardize the Bureau's goals.
For example, if the MAF/TIGER modernization is not completed on schedule,
the Bureau would be unable to employ the GPS-enabled handheld mobile
computing devices that enumerators are to use when conducting nonresponse
follow-up. This in turn could affect the efficiency of the effort and the
quality of the data collected. In addition, the Bureau would not have time
to conduct the research and testing necessary to improve the long form
based on lessons learned in the 2000 Census.
The Bureau's Plans for Measuring Accuracy and Coverage in 2010
Because of limitations in census taking methods, some degree of error in
the form of persons missed or counted more than once is inevitable. Since
1980, the Bureau has used statistical methods to generate detailed
measures of the undercounts of particular ethnic, racial, and other
population groups. To assess the quality of population data for the 2000
Census and to possibly adjust for any errors, the Bureau conducted the
Accuracy and Coverage Evaluation (A.C.E.) program. Although the U.S.
Supreme Court ruled in 1999 that the Census Act prohibited the use of
statistical sampling for purposes of apportioning seats in the House of
Representatives, the Court did not specifically address the use of
statistical sampling for other purposes.10
In March 2001, the Acting Director of the U.S. Census Bureau recommended
to the Secretary of Commerce that only unadjusted data be released for
purposes of congressional redistricting. The Acting Director made this
recommendation when, after considerable research, the Bureau was unable to
conclude that the adjusted data were more accurate for use in
redistricting. Specifically, the Acting Director cited the apparent
inconsistency in population growth over the decade as estimated by the
A.C.E., and demographic analysis, which estimated the population using
birth, death, and similar records. He noted that the inconsistency raised
the possibility of an unidentified error in either the A.C.E. estimates or
the census numbers, and the inconsistency could not be resolved prior to
April 1, 2001, the legally mandated deadline for releasing redistricting
data. Later that year, following additional research, the Acting Director
decided against using adjusted census data for nonredistricting purposes,
such as
10 Department of Commerce v. United States House of Representatives, 525
U.S. 316 (1999).
allocating federal aid, because A.C.E. estimates failed to identify a
significant number of people erroneously included in the census. The
Acting Director noted that "this finding of substantial error, in
conjunction with remaining uncertainties, necessitates that revisions,
based on additional review and analysis, be made to the A.C.E. estimates
before any potential uses of these data can be considered."
As the Bureau turned toward the 2010 Census, it needed to decide whether
it would have a coverage measurement program and how the results would be
used. Because of the 1999 U.S. Supreme Court ruling noted earlier, the
Bureau could not use coverage measurement results to adjust census data
for purposes of congressional apportionment. However, adjusting census
data for other purposes remained an open question.
In our January 2003 report on the objectives and results of the 2000
A.C.E. program, we noted that an evaluation of the accuracy and
completeness of the census is critical given the many uses of census data,
the importance of identifying the magnitude and characteristics of any
undercounts and overcounts, and the cost of the census overall.11 We
cautioned that the longer the 2010 planning process continues without a
firm decision on the role of coverage measurement, the greater the risk of
wasted resources and disappointing results.
Consequently, we recommended that the Bureau, in conjunction with Congress
and other stakeholders, come to an early decision on whether and how
coverage measurement will be used in the 2010 Census. In reaching this
decision, we recommended that the Bureau (1) demonstrate both the
operational and technical feasibility of its coverage measurement methods,
(2) determine the level of geography at which coverage can be reliably
measured, (3) keep Congress and other stakeholders informed of its plans,
and (4) adequately test its coverage measurement methodology prior to full
implementation.
The Bureau agreed with our recommendations, noting that we had identified
important steps that should be followed in developing a coverage
measurement methodology for the 2010 Census. While certain aspects of the
Bureau's coverage measurement plans are still being developed, the
11 U.S. General Accounting Office, 2000 Census: Coverage Measurement
Programs' Results, Costs, and Lessons Learned, GAO-03-287 (Washington,
D.C.: Jan. 29, 2003).
Bureau is not currently planning to develop a procedure that would allow
it to adjust census numbers for purposes of redistricting.
According to the Director of the U.S. Census Bureau, although the Bureau
plans to evaluate the accuracy of the coverage it achieves in the 2010
Census, its experience during the 2000 Census demonstrated "that the
science is insufficiently advanced to allow making statistical adjustment
to population counts of a successful decennial census in which the
percentage of error is presumed to be so small that adjustment would
introduce as much or more error than it was designed to correct."
Furthermore, irrespective of whether it is both legal and appropriate to
do so, the Bureau does not believe that it can both collect coverage
measurement data and complete the analysis of those data's accuracy in
time to deliver the information to the states to meet their redistricting
deadlines.
Although the Bureau's experience during the 2000 Census shows that its
approach to measuring coverage needs to be improved if it is to be used to
adjust census numbers, the Bureau has not yet determined the feasibility
of refinements to the 2000 approach or alternative methodologies.
Consequently, the Bureau's decision on how coverage evaluation data will
be used in 2010 appears to be premature. Indeed, while the Bureau has
reported that the 2000 Census had better coverage compared to the 1990
Census, as noted below, the U.S. population is becoming increasingly
difficult to count, a factor that could affect the quality of the 2010
Census.
More generally, the decennial census is an inherently fragile endeavor,
where an accurate population count requires the near-perfect alignment of
a myriad of factors ranging from the successful execution of dozens of
census-taking operations to the public's willingness to cooperate with
enumerators. External factors such as the state of the economy and world
events might also affect the outcome of the census. The bottom line is
that while the census is under way, the tolerance for any breakdowns is
quite small.
Therefore, the Bureau's ability to maintain the level of quality reported
for the 2000 Census is far from guaranteed. Thus, to ensure that the
nation uses the best available census data, it will be important for the
Bureau to research procedures that depending on what the results of the
coverage evaluation say about the quality of the census data, would allow
adjustment, if necessary, for those purposes for which it is both legal
and
appropriate to do so. The Bureau should conduct this effort on a timetable
that allows it to adequately test and refine those procedures, as well as
obtain input from both majority and minority parties in the Senate and
House of Representatives.
Escalating Census Costs Call for Transparent Budget Data and Better
Reporting
In June 2001, the Bureau estimated that the reengineered 2010 Census would
cost $11.3 billion in current dollars. This would make the 2010 head count
the most expensive in the nation's history, even after adjusting for
inflation. According to the Bureau estimates in June 2001, a repeat of the
2000 approach would cost even more, over $11.7 billion. This estimate of
repeating the 2000 approach was revised to approximately $12.2 billion in
April 2003. Moreover, the actual cost of the census could end up
considerably higher as the Bureau's initial cost projections for previous
censuses proved to be too low because of such factors as unforeseen
operational problems or changes to the fundamental design. For example,
while the Bureau estimated that the 2000 Census would cost around $4
billion using sampling, and that a traditional census without sampling
would cost around $5 billion, the final price tag for the 2000 Census
(without sampling) was over $6.5 billion.
Census Costs Continue to The Bureau's cost projections for the 2010
decennial census continue an
Grow escalating trend. As shown in figure 1, in constant 2000 dollars, the
estimated $9.3 billion cost of the 2010 Census represents a tenfold jump
over the $920 million spent on the 1970 Census (as noted above, the Bureau
estimates the 2010 Census will cost $11.3 billion in current dollars).
Figure 1: Decennial Census Costs from 1970 through 2010 (Projected) in
Constant
2000 Dollars
10 Cost in billions of dollars
9.3
8
6
4
2
0 1970 1980 1990 2000 2010 (projected)
Census year
Source: GAO analysis of U.S. Census Bureau figures.
Although some cost growth can be expected in part because the number of
housing units-and hence the Bureau's workload-has gotten larger, the cost
growth has far exceeded the housing unit increase. The Bureau estimates
that the number of housing units for the 2010 Census will increase by 10
percent over 2000 Census levels. Meanwhile, the average cost per housing
unit for 2010 is expected to increase by approximately 29 percent from
2000 levels (from $56 to $72), nearly five and a half times greater than
the $13 it cost to count each household in 1970 (see fig. 2).12
12 These figures include the 10-year costs for ACS replacement for the
census long form and the costs of MAF/TIGER.
Figure 2: Decennial Census Average Cost per Housing Unit (in Constant
Fiscal Year
2000 Dollars)
80 Dollars per housing unit
72
0
1970 1980 1990 2000 2010 (projected)Census year
Source: GAO analysis of U.S. Census Bureau figures.
As for previous censuses, the major cost for the 2010 Census is what the
Bureau calls "field data collection and support systems." Over half of
decennial census life cycle costs are attributed to this area. Specific
components include the costly and labor-intensive nonresponse follow-up
operation as well as support activities such as the opening and staffing
of hundreds of temporary local census offices.
One reason why field data collection is so expensive is because the Bureau
is finding it increasingly difficult to locate people and get them to
participate in the census. According to Bureau officials, societal trends,
such as the increasing number of respondents who do not speak English, the
growing difficulty of finding respondents at home, and the general
increase of privacy concerns, impede a cost-effective head count. Further,
the legal requirement to count everyone leads the Bureau to employ
operations that only marginally improve coverage but that are relatively
expensive to conduct.
Societal changes have also reduced the cost-effectiveness of the census,
and it has become increasingly difficult to simply stay on par with the
results of previous enumerations. For example, during the 1990 Census, the
Bureau spent $0.88 per housing unit (in 2000 dollars) to market the
census and encourage participation and achieved a response rate of 65
percent. During the 2000 Census, the Bureau spent $3.19 per housing unit
(in 2000 dollars) to promote participation, but the response rate was 64
percent.
Cost Estimates Need Greater Transparency
The constitutional mandate to count the nation's population explicitly
commits or "exposes" the government to spending money on the census each
decade. In this way, the census is similar to other fiscal exposures such
as retirement benefits, environmental cleanup costs, and the payment of
Social Security benefits in that the government is obligated to a certain
level of future outlays.
Early in each census cycle, expenditures are relatively low as the Bureau
plans the census and conducts various tests. As the decade continues,
spending increases, spiking during the decennial year when costly data
collection activities take place. As shown in figure 3, during the 2000
Census, $4.1 billion-almost two-thirds of the money spent on the entire
census-was spent in fiscal year 2000 alone.
Figure 3: 2000 Census Life Cycle Costs by Fiscal Year 5,000 Dollars in
millions
4,000 4,117
3,000
2,000
1,000
0 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003
(estimate)
Fiscal year
Source: GAO.
Current budget reporting, however, does not always fully capture or
require explicit consideration of some future fiscal exposures. In fact,
this is particularly true with the census, as annual budget requests and
reports
provided to Congress early in the decennial census life cycle do not
reflect the full cash consequences of the spending and policy decisions.
Thus, as it begins funding the 2010 Census early in the decade at
relatively low levels, Congress will have implicitly accepted a future
spike in costs-essentially a balloon payment in 2010-without requiring the
Bureau to clearly define what those costs might be, why they are
justified, and what alternatives might exist.
As we noted in our January 2003 report on improving the budgetary focus on
long-term costs and uncertainties, information on the existence and
estimated cost of fiscal exposures needs to be considered along with other
factors when making policy decisions.13 With respect to the census, not
capturing the long-term costs of annual spending decisions limits
Congress's ability to control the government's exposure at the time
decisions are made, consider trade-offs with other national priorities,
and curtail the growth in census costs. Consequently, fiscal transparency
is critical to better reflect the magnitude of the government's long-term
spending on the census and signal unanticipated cost growth. Greater
fiscal transparency can also facilitate an independent review and provide
an opportunity to improve stakeholder confidence and commitment to the
Bureau's reengineered decennial census design.
Our January 2003 report noted that increased supplemental reporting could
help improve fiscal transparency and described several options for how to
accomplish this. Although that report recommended that OMB consider
implementing these options governmentwide, at least two options could be
adapted specifically for the Bureau and its parent agency, the Department
of Commerce. The two options are (1) annually reporting the planned life
cycle cash flow and explaining any material changes from previous plans
(currently, the Bureau does not make this information available) and (2)
setting triggers to signal when the amount of money expected to be spent
in any one year exceeded a predetermined amount. Combined, these actions
could prompt more explicit deliberations on the cost and affordability of
the census and help inform specific cost control measures by Congress, if
warranted.
13 U.S. General Accounting Office, Fiscal Exposures: Improving the
Budgetary Focus on Long-Term Costs and Uncertainties, GAO-03-213
(Washington, D.C.: Jan. 24, 2003).
More Information Needed on Underlying Budget Assumptions
The assumptions the Bureau used to develop the life cycle cost estimate
could also benefit from greater transparency.14 More robust information on
the likelihood that the values the Bureau assigned to key cost drivers
might differ from those initially assumed, and the impact that any
differences would have on the total life cycle cost, could provide
Congress with better information on the range and probability of the
fiscal exposure the nation faces from the upcoming census.
As shown in figure 4, the Bureau derived the baseline for its 2010 cost
estimate using the actual cost of the 2000 Census and assumptions about
certain cost drivers, estimating the cost of the 2010 Census as if the
Bureau were to repeat its 2000 design. The key assumptions include
o a 35 percent decrease in enumerator productivity,
o a pay rate increase for census workers from 2000 levels,
o a mail-back response rate lower than Census 2000 levels,15 and
o inflation.
14 OMB Circular A-94 provides guidelines for cost-benefit analysis of
federal programs and recommends that agencies develop a sensitivity
analysis for major projects with significant uncertainty. The circular
provides a method for determining how sensitive outcomes are to changes in
assumptions.
15 A lower mail-back response rate means that more costly nonresponse
follow-up activities are needed to obtain census data from a household.
Figure 4: Bureau's June 2001 Estimating Methodology for Census 2010 Life
Cycle Costs
The projected costs and savings of repeating the 2000 design versus the
Bureau's approach based on the three-legged stool, are shown in table 2.
Table 2: Projected Costs from June 2001 Estimate of Reengineering the 2010
Census
Dollars in millions
Budget frameworks
Repeated 2000 Census design (baseline) Reengineered 2010 Census
(three-legged stool)
Savings (increases)
Program Development and
Management 183 247 (64)
Data Content and Products 765 705
Field Data Collection and Systems 7,880 5,640 2,240
Address List Compilation 610 455 155
Automated Data Processing 1,265 840 425
Testing, Evaluation, and Rehearsal 391 467 (76)
Island Areas 163 153
Marketing and Communications 468 518 (50)
ACS 1,720 (1,720)
MAF/TIGER 535 (535)
Total 11,725 11,280a 445
Source: U.S. Census Bureau.
Note: ACS and MAF/TIGER modernization spending is not included under the
scenario of repeating the 2000 Census design for the 2010 Census.
aThe Bureau revised its estimates in April 2003, but we did not have the
breakdown by budget framework to revise this table.
Transparent information is especially important since decennial cost
estimates are sensitive to many key assumptions. In fact, for the 2000
Census, the Bureau's supplemental funding request for $1.7 billion in
fiscal year 2000 primarily involved changes in assumptions related to
increased workload, reduced employee productivity, and increased
advertising.
Given the cost estimates' sensitivity to key assumptions, greater
transparency could be obtained by showing the range and likelihood of how
true cost drivers could differ from those assumed and how those
differences would affect estimates of total cost. Thus, if early research
and testing show that response rates may be higher than originally
anticipated, or that enumerator productivity could be better than
expected, the Bureau can report on the nature of its changing assumptions
and its effect on life cycle costs. Also important, by providing
information on the likely accuracy of assumptions concerning cost drivers,
the Bureau would better enable Congress to consider funding levels in an
uncertain environment.
Other key areas in which changes in assumptions can greatly affect costs
are salary rates for enumerators, the future price of handheld mobile
computing devices, and inflation.
Our prior work has described how agencies provide supporting information
when developing budget assumptions. For example, the Nuclear Regulatory
Commission identifies a basis and a certainty level for its budget
assumptions used for internal reporting. A basis summarizes the facts that
were evaluated to justify the assumption, while a certainty level depicts
the likelihood of its occurrence as high, medium, or low.16
Finally, it is important to have timely cost information for congressional
decision making. The Bureau's life cycle estimates were updated in April
2003 after being first presented in June 2001-nearly a 2-year interval. In
addition, when we requested additional information on life cycle costs the
Bureau took several months to provide information on its life cycle cost
estimates and assumptions, ultimately revising its total cost estimates
before providing us with the data.
2010 Planning Process The Bureau has taken several positive steps to
correct problems
encountered planning past censuses, and the Bureau appears to be furtherIs
Better Than Past along in planning the 2010 Census than at this same point
during the 2000Efforts, but Rigor Is Census cycle. Although an improvement
over past efforts, the Bureau's Uneven 2010 planning process still
contains certain weak points that if not
addressed could undermine a cost-effective head count and make it more
difficult to obtain the support of Congress and other stakeholders.
The Bureau Improved Its Planning Process, but Key Elements Are Still
Lacking
The characteristics of the census-long-term, large-scale, high-risk,
costly, and politically sensitive-together make a cost-effective
enumeration a monumental project management challenge, one that demands
meticulous planning. To help determine the principal ingredients of
successful project planning, we reviewed a number of guides to project
management and business process reengineering. Although there is no one
best approach to
16 U.S. General Accounting Office, Managing for Results: Efforts to
Strengthen the Link Between Resources and Results at the Nuclear
Regulatory Commission, GAO-03-258 (Washington, D.C.: Dec. 10, 2002).
project planning, the guides we reviewed contained many elements in
common, including the following:
o Developing a project plan. The plan should consider all phases of the
project and should have clear and measurable goals; all assumptions,
schedules, and deadlines clearly stated; and needed skills and resources
identified.
o Evaluating human resource implications. This includes assessing needed
competencies and how they will be acquired and retained.
o Involving stakeholders and incorporating lessons learned.
Stakeholders-both internal and external to an organization-have skills and
knowledge that could contribute to a project and should be involved in the
decision-making process. An organization should focus on the highest
priority stakeholder needs and mission goals. Evaluating past performance
and capitalizing on lessons learned is also important for improving
performance.
o Analyzing and mitigating risks. This involves identifying, analyzing,
prioritizing, and documenting risks. Ideally, more than one alternative
should be assessed.
o Monitoring progress. Measurable performance goals should be identified
and performance data should be gathered to determine how well the goals
are being achieved.
The Bureau has made considerable progress in planning the 2010 Census, and
some of the positive steps taken to date include the following efforts.
o Early in the decade, senior Bureau staff considered various goals for
the 2010 Census and articulated a design strategy to achieve those goals.
Senior Bureau officials collaborated on this initial design plan to set
the stage for further refinements during later field testing and research
activities.17
17 Details are documented in U.S. Census Bureau, Decennial Management
Division, Reengineering the Decennial Census: The Vision of a Baseline
Design for 2010
(Washington, D.C.: January 2003).
o The Bureau has involved experienced staff in the design process through
cross-divisional planning groups. Staff involved in these planning groups
will ultimately be responsible for implementing the 2010 Census. According
to Bureau officials, this is a departure from the 2000 Census planning
effort when Bureau staff with little experience in conducting the census
played a key role in designing the decennial census, which resulted in
impractical reforms that could not be implemented.
o The Bureau has recognized the importance of strategically managing its
human capital to meet future requirements. The planning and development of
the 2010 Census will take place at a time when the Bureau could find
itself experiencing substantial employee turnover (three senior Bureau
managers left the agency in 2002, and according to a report by the
Department of Commerce Inspector General, the Bureau could lose through
retirement around half of the senior staff who carried out the 2000
Census).18 The Bureau, as part of a broader risk assessment, plans to
provide less experienced staff the opportunity to obtain operational
experience prior to the actual 2010 Census. In addition, the Bureau has
provided training in project management and has encouraged staff to take
training courses in management and planning.
However, other aspects of the Bureau's 2010 planning process could be
improved. A more rigorous plan would better position the Bureau to fulfill
its key objectives for the 2010 Census and help demonstrate to Congress
and other stakeholders that it can effectively design and manage
operations as well as control costs.
18 U.S. Department of Commerce, Office of Inspector General, Improving Our
Measure of America: What Census 2000 Can Teach Us in Planning for 2010,
OIG-14431 (Washington, D.C.: Spring 2002).
Developing a Project Plan Although the Bureau has developed project plans
for some of the key components of its 2010 strategy, the Bureau has not
yet crafted an overall project plan that (1) includes milestones for
completing key activities; (2) itemizes the estimated cost of each
component; (3) articulates a clear system of coordination among project
components; and (4) translates key goals into measurable, operational
terms to provide meaningful guidance for planning and measuring progress.
OMB Circular A-1119 specifies that an agency's general goals should be
sufficiently precise to direct and guide agency staff in actions that
carry out the agency's mission and aid the agency in developing annual
performance goals.
The importance of this information for improving accountability and
performance can be seen, for example, in the Bureau's principal goal to
increase coverage and accuracy. Though laudable, the Bureau has yet to
assign any numbers to this goal. This makes it difficult to evaluate the
costs and benefits of alternative designs, determine the level of
resources needed to achieve this goal, measure the Bureau's progress, or
hold managers accountable for results.
Bureau managers provided us with several documents that pieced together
present 2010 Census goals and strategies, life cycle costs, and schedules,
but no single, comprehensive document exists that integrates this
information. For example, the Bureau's life cycle cost estimates and
information on its performance goals were contained in two separate
documents, making it hard to see the connection between cost and the
Bureau's objectives. Likewise, a draft document, entitled 2010
Reengineered Census Milestone Schedule, included various milestones by
fiscal quarter, but did not contain information on dependencies and
interrelationships among the various aspects of the project.
In its 2001 letter to the Bureau's acting director, the National Academy
of Sciences' (NAS) Panel on Research on Future Census Methods raised
similar concerns about the need for a coherent project plan. The panel
noted that it wanted "to see a clearer case for components of the 2010
census strategy, itemizing the goals, costs, and benefits of each
initiative and indicating how they integrate and contribute to a high
quality census." To that end, NAS recommended that the Bureau develop what
it called a business plan for 2010.
19 OMB Circular A-11, as revised July 2003, provides guidance to agencies
on budget preparation, performance reporting, and capital asset
acquisition.
Incorporating Lessons Learned The Bureau is making an effort to develop
and incorporate the lessons learned from the 2000 Census and, in fact,
created an elaborate evaluation program to help inform this effort.
Moreover, the Bureau chartered 11 planning groups consisting of
knowledgeable census staff (see app. III for the 2010 planning
organization). However, the Bureau's ability to build on the results of
2000 could be hampered by the fact that while the evaluation program
assessed numerous aspects of the census, the Bureau still lacks data and
information on the performance of key census activities, as well as on how
specific census operations contributed to two of the Bureau's key goals
for 2000: improved accuracy and cost-effectiveness.
For example, as noted earlier, the cost of the 2010 Census is increasing
relative to 2000 partly because the Bureau expects nonresponse follow-up
enumerators will be less productive in 2010. Because of various societal
factors, it will simply take enumerators more time to complete their work.
And yet, despite the importance of accurate productivity data to inform
the Bureau's planning and budgeting processes for 2010, the Bureau had
trouble obtaining quality productivity data following the 2000 Census.
Although the Bureau later committed additional resources to refine the
numbers, the adjustment was coarse and addressed just one of the two known
problems. Moreover, because of differences in the way the Bureau measured
staffing levels and hours worked from census to census, none of the
productivity data from the last few censuses are comparable.20
Another area in which the Bureau lacks useful performance information is
in the extent to which the dozen or so separate activities used to build
MAF in 2000 contributed to its overall accuracy relative to one another.
Without this information, the Bureau has limited data with which to guide
investment and trade-off questions for 2010, such as which activity
provided the biggest "bang for the buck" and should thus be repeated, or
whether it would be more effective for the Bureau to improve accuracy and
coverage by putting more resources into MAF-building activities or some
other operation altogether, such as marketing.
20 For more information on the difficulties the Bureau encountered
generating reliable productivity data, see U.S. General Accounting Office,
2000 Census: Better Productivity Data Needed for Future Planning and
Budgeting, GAO-02-4 (Washington, D.C.: Oct. 4, 2001).
To date, the Bureau's planning groups have incorporated a variety of
lessons learned from the evaluations of the 2000 Census. As an example,
the Coverage Improvement Planning Group observed an increase in
inconsistent responses from 2000 Census compared to the previous census
(e.g., some questionnaires were marked "uninhabited," but individuals were
enumerated at the sites). According to a Bureau official, one hypothesis
for the higher rate of inconsistent responses was that enumerators were
encouraged to fill in information even when not all of the relevant
information was known. The Bureau plans to address this issue by building
in "edits" to its planned handheld mobile computing devices so that
inconsistent data cannot be entered.21 In addition, the Coverage
Improvement Planning Group also looked at the 2000 Census experience to
provide recommendations for the Bureau's 2004 test.
Analyzing and Mitigating Risks Risk management is important for preparing
for contingencies or changes in the external operating environment. At the
time of our review, the Bureau had completed a risk assessment of some
aspects of its operations as part of its OMB Circular A-11, Exhibit 300
submission,22 and for certain aspects of the reengineering efforts.23
However, the Bureau had not developed a risk assessment that addressed the
entire 2010 Census, including ACS and the MAF/TIGER modernization.
The risk assessment for the reengineering effort uses a consistent scoring
system to assess the severity of the risks identified and addresses
various contingencies and mitigation strategies, such as preparing for the
retirement of key personnel and using succession planning to offset the
attrition. The scoring system and how it was applied is clearly described
in the plan, making it easy to evaluate the way it was used.
However, the assessment does not provide extensive detail on the
mitigation actions proposed. Also, it does not indicate how risks were
identified and whether any risks were excluded. A notable exclusion was
that it did not address the risks if ACS or MAF/TIGER fail or are not
funded and the impact this might have on the census as a whole.
21 For example, if an address is noted to be vacant, an error message will
appear if the enumerator attempts to count people at that address.
22 Exhibit 300 is a capital asset plan required under OMB Circular A-11,
Section 300.
23 U.S. Census Bureau, Decennial Management Division, 2010 Decennial
Census Risk Management Plan (Washington, D.C.: Mar. 4, 2003).
As mentioned earlier, the Bureau's three-legged stool strategy assumes
that all three legs must work together to achieve its goals. One of the
reasons for doing a risk analysis is to prepare to make trade-offs when
faced with inevitable budgetary pressures, operational delays, or other
risks. Lacking information on trade-offs, the Bureau maintains that its
only alternative to the reengineering is to repeat the 2000 Census design,
an approach that Bureau officials believe will be extremely expensive.
Conclusions The obstacles to conducting a cost-effective census have grown
with each decade, and as the Bureau looks toward 2010, it confronts its
biggest challenge yet. Consequently, the Bureau will need to balance the
growing cost, complexity, and political sensitivity of the census with
meticulous planning. As the Bureau's past experience has shown, early
investments in planning can help reduce the costs and risks of its
downstream operations.
Moreover, a rigorous plan is essential for securing early agreement
between the Bureau and Congress on the Bureau's fundamental strategy for
2010. Congressional support-regardless of whether the Bureau's current
approach or an alternative is ultimately selected-is crucial for creating
a stable environment in which to prepare for the census and avoiding a
repeat of the 2000 Census when disagreement over the Bureau's methodology
led to late design changes and additional costs and risks. The Bureau has
laid out an ambitious schedule of planning, testing, and evaluation for
the coming years, culminating with a "dress rehearsal" in 2008. While
midcourse corrections are to be expected as a result of these efforts, it
will be important for the Bureau to proceed with as few alterations to its
fundamental strategy as possible so that all of the operations used in
2010 have been thoroughly road tested.
The Bureau appears to be further along in planning the 2010 Census
compared to a similar point during the 2000 Census cycle, and its efforts
to enhance past planning practices are commendable. Focusing its
activities on early design, research, and testing and organizing its
reengineering activities around cross-divisional planning groups, are just
some of the noteworthy improvements the Bureau has made.
However, the Bureau's plans for 2010, while not unreasonable on the
surface, lack a substantial amount of supporting analysis, budgetary
transparency, and other information, making it difficult for us, Congress,
and other stakeholders to properly assess the feasibility of the Bureau's
design and the extent to which it could lead to greater cost-effectiveness
compared to alternative approaches.
Questions surrounding the Bureau's underlying budget assumptions;
uncertainties over ACS; the failure to translate key goals into
measurable, operational terms; and the lack of important performance data
from the 2000 Census to inform 2010 decision making are just some of the
problematic aspects of the 2010 planning process. More than simply
paperwork or documentation issues, this information is essential for
improving the performance and accountability of the Bureau and of the
decennial census in particular.
To be sure, some challenges are to be expected in an endeavor as demanding
as counting a population that is mobile and demographically complex and
whose members reside under a multitude of living arrangements. Further,
shortcomings with prior censuses call for the Bureau to consider bold
initiatives for 2010 that entail some risk. However, if Congress is to
accept and fund the Bureau's approach-now estimated at more than $11
billion-then the Bureau needs to more effectively demonstrate that it has
(1) selected a design that will lead to the most costeffective results and
(2) establish a rigorous capacity to manage risks, control costs, and
deliver a successful head count. Moreover, to ensure the nation uses the
best available data, it will be important for the Bureau to research
procedures that would allow it to adjust census results for purposes for
which it is both legal and appropriate to do so, if it is determined that
the adjusted figures would provide greater accuracy than the enumeration
data. Such procedures could function as a safety net should there be
problems with the initial census count.
It will also be important for policymakers to consider, early in the
decade, the long-term costs associated with the census and finding the
right balance between controlling mushrooming costs and improving
accuracy. Although initial spending on the census is relatively low, it
will accelerate in the years ahead, culminating with a balloon payment in
2010 when data collection and other costly operations take place. Greater
fiscal transparency prior to getting locked into a particular level of
spending could help inform deliberations on the extent to which (1) the
cost of the census is reasonable, (2) trade-offs will need to be made with
competing national priorities, and (3) additional dollars spent on the
census yield better quality data.
Just over 6 years remain until Census Day 2010. While this might seem like
an ample amount of time to shore up the Bureau's planning process and take
steps to control costs, past experience has shown that the chain of
interrelated preparations that need to occur at specific times and in the
right sequence leave little room for delay or missteps.
Recommendations for Executive Action
To help control the cost of the 2010 Census and inform deliberations on
the acceptability of those costs, we recommend that the Director of the
Office of Management and Budget take steps to ensure that the Bureau
improves the transparency of the fiscal exposure associated with the
census. Specifically, OMB should ensure that the Bureau, in a notational
item in the Program and Financing schedule of the President's budget,
include an updated estimate of the life cycle costs of the census and the
amount of money the Bureau expects to spend in each year of the cycle, as
well as an explanation of any material changes from previous plans. The
information should also contain an analysis of the sensitivity of the cost
figures to specific assumptions, including a range of values for key cost
assumptions, their impact on total cost estimates of the census, the
likelihood associated with those ranges, and their impact on the total
estimated cost of the census.
As part of this process, OMB should establish triggers that would signal
when the yearly 2010 Census costs, total 2010 Census costs, or both
exceeded some predetermined amount. In such instances, the Bureau should
then be required to prepare a special report to Congress and OMB
justifying why the additional costs were necessary and what alternatives
were considered.
Further, to enhance the Bureau's performance and accountability, as well
as to help convince Congress and other stakeholders that the Bureau has
chosen an optimum design and will manage operations and control costs
effectively, we recommend that the Secretary of Commerce direct the Bureau
to improve the rigor of its planning process by developing an operational
plan that consolidates budget, methodological, and other relevant
information about the 2010 Census into a single, comprehensive project
plan that would be updated as needed. Individual elements could include
o specific performance goals, how the Bureau's efforts, procedures, and
projects would contribute to those goals, and what performance measures
would be used;
o risk and mitigation plans that fully address all significant potential
risks;
o detailed milestone estimates that identify all significant
interrelationships; and
o annually updated life cycle cost estimates, including a sensitivity
analysis, and an explanation of significant changes in the assumptions on
which these costs are based.
Moreover, to help ensure that the nation has at its disposal the best
possible data should there be problems with the quality of 2010 Census,
the Bureau, with input from both majority and minority parties in the
Senate and House of Representatives, should research the feasibility of
procedures that could allow it to adjust census results for those purposes
for which it is both legal and appropriate to do so and, if found to be
feasible, test those procedures during the 2006 census test and 2008
census dress rehearsal.
Agency Comments and Our Evaluation
The Secretary of Commerce forwarded written comments from the U.S. Census
Bureau on a draft of this report that we received November 6, 2003. The
comments are reprinted in appendix I. The Bureau generally disagreed with
many of our key findings, conclusions, and recommendations.
The Bureau believes that the report, in its discussion of escalating
census costs, ignores the fact that a key cost driver is stakeholders'
demand for better accuracy. We agree with the Bureau that its mandate to
count each and every resident in the face of countervailing societal
trends is an important reason for the cost increases. As we noted in the
report, societal changes have reduced the cost-effectiveness of the
census, and it has become more and more difficult to stay on par with the
results of previous enumerations. Similarly, we stated that "the legal
requirement to count everyone leads the Bureau to employ operations that
only marginally improve coverage but that are relatively expensive to
conduct."
Further, we do not, as the Bureau asserts, treat the cost issue in a
vacuum, and agree with the Bureau that little would be gained by focusing
on the cost of the 2010 Census alone. Rather, any deliberations on the
2010 Census need to focus on how changes in spending on the census might
affect the quality of the count. Our draft report emphasized this exact
point noting that "The growing cost of the head count, at a time when the
nation
is facing historic budget deficits, highlights the importance of
congressional deliberations on the extent to which each additional dollar
spent on the census results in better data, as well as how best to balance
the need for a complete count, with the need to ensure the cost of a
complete count does not become unreasonable." Similarly, we concluded that
"it will also be important for policymakers to consider, early in the
decade, the long-term costs associated with the census and finding the
right balance between controlling mushrooming costs and improving
accuracy."
The Bureau also believes the report implies that the cost increases are
caused by the reengineering effort. Our draft report did not state, nor
did we intend to imply, that the reengineering effort would cause most of
the projected cost increases for the 2010 Census. In fact, our report even
notes that the Bureau's reengineering strategy has the potential to reduce
costs relative to a design that would repeat the Census 2000 approach. To
help clarify this point, we added text that describes how a repeat of the
2000 approach would be more costly than the reengineered design, according
to Bureau estimates.
The Bureau disagreed with our recommendation to OMB regarding the need for
greater budgetary transparency, noting that the real reason for the
vagueness of out-year cost estimates stems from a fundamental difference
of opinion between the administration and Congress over the appropriate
time to share that information. We believe that it is important for the
administration to provide details of out-year cost projections for the
decennial census for the reason stated in our draft report: annual budget
requests and reports provided to Congress early in the decennial census
life cycle do not reflect the full cash consequences of the spending in
later years of the decade. We understand that the Bureau has followed the
administration's guidance on providing out-year cost estimates; this is
also why we directed our recommendation for greater fiscal transparency to
OMB, which we discuss in greater detail below.
The Bureau disagreed with our recommendation to improve the rigor of its
planning process by developing an operational plan that consolidates
budget, methodological, and other relevant information into a single,
comprehensive project plan. The Bureau noted that these documents already
exist and are widely available, and the Bureau already shares them with
Congress, us, the National Academy of Sciences (NAS) Panel on Research on
Future Census Methods-the panel responsible for reviewing the census, and
other stakeholders.
While we agree with the Bureau that some of this information is available
(and we noted this fact in our draft report), it is piecemeal-one can only
obtain it by cobbling together the Bureau's budget submission, its
strategic plan, and several other documents, and even then, key
information such as performance goals would still be lacking. Further,
although the Bureau notes that it has provided this information to the NAS
panel, as we stated in our report, NAS, like us, also found the
information wanting. As we described in the report, the panel shared our
concerns over the need for a coherent project plan, and called on the
Bureau to develop a business plan that among other things, itemized the
goals, costs, and benefits of each census component and a describes how
they contributed to a high-quality census. Whether it is called a business
plan or a project plan, such information is not, as the Bureau maintains,
simply "more process." Quite the contrary, this information is essential
for improving performance; facilitating a thorough, independent review of
the Bureau's plans; and demonstrating to Congress and other stakeholders
that the Bureau can effectively design and manage operations and control
costs.
The Bureau incorrectly asserts that our report criticizes it for not
completing the evaluations of the 2000 Census in a timely manner. Our
report did not address this matter, although NAS's Second Interim Report
on Planning the 2010 Census urged the Bureau to "give high priority to
evaluation studies" and complete them as expeditiously as possible. We
agree that the Bureau's planning staff do have access to the draft
evaluations, and in fact, we noted in the report that they are using them
in planning for the 2010 Census.
The key point, however, is the Bureau's ability to build on the results of
the 2000 Census. This could be hampered by the fact that while the
evaluation program assessed numerous aspects of the census, the Bureau
still lacks data on the performance of key census activities as well as
how specific census operations contributed to two of the Bureau's key
goals for 2000: improved accuracy and cost-effectiveness.
The Bureau agreed with us that it is important to bring closure to the
discussion on whether and how coverage measurement will be used in the
2010 Census. However, the Bureau believes that the approach used for the
2000 Census proved that it was not feasible to produce a final analysis of
coverage measurement in time to meet redistricting requirements. We agree
with the Bureau's assessment that the coverage measurement approach used
in the 2000 Census needs to be reworked. However, this should not preclude
it from researching alternative approaches for the 2010
Census in light of the fact that the Bureau's ability to maintain the
level of quality reported for the 2000 Census is less than certain.
Finally, the Bureau questioned our assessment that the only contingency
plan for conducting the 2010 Census, if the reengineered effort fails, was
to fall back on the Census 2000 methods. The Bureau maintains that the
2000 Census was by most accounts a very successful census and,
accordingly, the Bureau already has available the methods and procedures
for taking an excellent census. Our report does not advocate the
development of another set of census methods. Rather, we were trying to
illustrate the challenge the Bureau faces in implementing its
reengineering plans, where the failure of any one leg could compromise the
other two, thereby requiring the Bureau to rely on the approach it used
for the 2000 Census. According to Bureau officials, this in turn could
make it difficult for the Bureau to accomplish its goals for the 2010
Census, which include cost containment and better quality data.
On October 14, 2003, the Associate Director for General Government
Programs, OMB, provided written comments on a draft of this report, which
are reprinted in appendix II. OMB shared our view that the costs and risks
associated with the 2010 Census must be carefully monitored and evaluated
throughout the decade. OMB also agreed that it is essential to understand
the key cost drivers and said that it is working with the U.S. Census
Bureau to ensure that the Bureau develops high-quality, transparent life
cycle cost estimates.
However, OMB disagreed with our recommendation that it ensure that the
Bureau, include a notational item in the Program and Financing (P&F)
schedule of the President's Budget with an updated estimate of the life
cycle costs of the census and the amount of money the Bureau expects to
spend in each year of the cycle, as well as an explanation of any
significant changes from previous plans. OMB believes that the Bureau's
report on the life cycle costs, which is updated regularly, is the best
mechanism to present estimates of the total life cycle costs and
explanations for any material changes from previous plans. Further, OMB
noted that presenting this information in the P&F schedule is cumbersome
and unnecessary because the Analytical Perspectives volume of the
President's Budget currently shows out-year estimates that incorporate
anticipated programmatic changes of the Decennial Census within the
Periodic Censuses and Program account.
As noted in our report, we do not believe the information OMB currently
reports to Congress is sufficiently timely or detailed to provide the
level of transparency needed for effective congressional oversight and
cost control. Indeed, while OMB cites the Bureau's life cycle report, the
document that we reviewed for this report took the Bureau nearly 2 years
to revise. Moreover, the revised estimates, like the original estimates,
overstated the life cycle cost estimate by $300 million because the Bureau
did not take into account a surplus of that amount that it identified near
the end of fiscal year 2000. Although the Bureau is to reissue the 2010
life cycle cost estimates early in calendar year 2004, the incorrect
estimates will have been in circulation for more than 2 years.
Additionally, the information contained in the Analytical Perspectives
volume of the President's Budget is limited. For example, it only provides
information on out-year estimates for 5 years. As a result, the volume
will not include estimates for the high-cost year of 2010 until the
release of the President's fiscal year 2006 budget. Further, the
Analytical Perspectives volume lacks information on the sensitivity of
cost figures to specific assumptions and the likelihood of these
estimates. It also does not contain any explanations of changes in cost
estimates from year to year. Complete and transparent information on
out-year costs is important to inform deliberations on the acceptability
of these costs and to ensure that Congress understands the possible range
of census life cycle costs.
OMB also disagreed with our recommendation to establish triggers to signal
when the yearly 2010 Census costs, total 2010 Census costs, or both
exceeded some predetermined amount. OMB noted that it has established
internal procedures within its budget reviews to monitor 2010 Census costs
and believes they are sufficient for ensuring that estimates are not
exceeded without clear justification. OMB added that this justification
could be included in the Bureau's updates to its life cycle cost
estimates.
Although OMB's internal procedures might be sufficient for OMB's
requirements, they do little to address the fundamental need for greater
fiscal transparency. Continued reliance on these procedures would inhibit
independent review by congressional and other external stakeholders, as
well as limit informed discussion of the trade-offs of dollars versus
accuracy and what cost control measures, if any, might be needed to make
the 2010 Census more affordable.
In closing, OMB commented that the Bureau's reengineering plan is being
reviewed by NAS as well as seven advisory committees. OMB stated that
the analyses stemming from these reviews, such as NAS's recently issued
report on census planning, enhance the Bureau's accountability and help
ensure "that the ultimate 2010 Census design is optimal."
We agree with OMB that NAS and the advisory committees are important for
reviewing the Bureau's plans and holding the Bureau accountable for a
cost-effective census in 2010. And this is precisely why we made the
recommendations that we did. Without a transparent budgeting and planning
process, a thorough, independent review by these and other external groups
would be difficult to impossible. That greater transparency is needed in
both these areas is highlighted not just in our report, but in the very
NAS study that OMB cites. Indeed, NAS found that "a major conclusion of
the panel is that discussion of the 2010 Census design needs to be more
fully informed by the evaluation of various tradeoffs-the costs and
benefits of various reasonable approaches in order to make wise
decisions."24
As agreed with your offices, unless you release its contents earlier, we
plan no further distribution of this report until 30 days from its issue
date. At that time we will send copies to other interested congressional
committees, the Secretary of Commerce, the Director of the U.S. Census
Bureau, and the Director of the Office of Management and Budget. Copies
will be made available to others upon request. This report will also be
available at no charge on GAO's Web site at http://www.gao.gov.
24 National Research Council, Panel on Research on Future Census Methods,
Committee on National Statistics, Planning the 2010 Census: Second Interim
Report (Washington, D.C.: 2003).
If you or your staff have any questions concerning this report, please
contact me at (202) 512-6806 or [email protected] or Robert Goldenkoff,
Assistant Director, at (202) 512-2757 or [email protected]. Key
contributors to this report were Richard Donaldson, Ty Mitchell, Robert
Yetvin, and Christine Bonham.
Patricia A. Dalton Director Strategic Issues
Appendix I
Comments from the Department of Commerce
Appendix I
Comments from the Department of
Commerce
Appendix I
Comments from the Department of
Commerce
Appendix I
Comments from the Department of
Commerce
Appendix II
Comments from the Office of Management and Budget
Appendix II
Comments from the Office of Management
and Budget
Appendix III
2010 Census Planning Organization
The U.S. Census Bureau (Bureau) has given its Decennial Management
Division responsibility for planning the 2010 Census, including the
American Community Survey (ACS) and the Master Address File/Topologically
Integrated Geographic Encoding and Referencing (MAF/TIGER) enhancements.
The division established an executive steering committee, shown in figure
5, for this purpose.
Appendix III 2010 Census Planning Organization
Figure 5: Decennial Management Division's Organizational Structure for
Planning the 2010 Census
Source: U.S. Census Bureau.
Appendix III 2010 Census Planning Organization
Each group under the 2010 Census planning organization has specific
activities that it was charged with studying. Listed below are the
research and development planning groups chartered at the time of our
review and a list of these activities.
2010 Census Reengineering: Research and Development Planning Groups
The Content Planning Group
o determines alternatives for question wording and order;
o develops consistent questions across multiple response modes;
o develops consistent forms design across multiple response modes;
o researches use of administrative records for missing data;
o researches and documents analysis of tabulated items' quality,
including item response rate for misreported, underreported, or generally
nonreported questions;
o researches other methods of imputing characteristics;
o researches information on administrative records and aggregate census
tract modeling;
o explores attitudinal trends on the mailing package and questionnaire;
o develops alternative formats for race and Hispanic origin question
layout;
o researches printing alternatives and new technologies;
o researches United States Postal Service requirements and alternative
services related to addressing, barcodes, and delivery;
o coordinates content development and congressional approval of ACS;
o assesses appropriate questionnaire wording to ensure conformance with
the enumeration and residence rules; and
o defines and assesses appropriate question wording for "usual home
elsewhere" on any self-response and enumerator instrument(s).
Appendix III 2010 Census Planning Organization
The Special Place/Group Quarters Planning Group
o reviews definitions of housing units and group quarters, including
transient locations, and where necessary, modifies the definitions to
reflect current living arrangements to reduce census classification
errors;
o determines the best methods for implementing housing unit and group
quarters definitions in 2010 Census operations that reduce decennial
census classification errors;
o determines what operations are required to improve the accuracy of the
current Census 2000 inventory file and the geographic reference base,
including geographic assignment;
o determines what coverage improvement operations are required to ensure
complete coverage of special places and group quarters;
o develops a process that ensures the integration of special place/group
quarters addresses with housing unit addresses and subsequent updates into
the MAF/TIGER system;
o determines the appropriate enumeration methods for the group quarters
population;
o determines the appropriate enumeration method for embedded housing
units within special place/group quarters that will reduce 2010 Census
error;
o determines improvements to the enumeration process, including
appropriate requirements regarding the implementation of mobile computing
devices;
o determines questionnaire requirements;
o assesses the quality of administrative records for the group quarters
population and determines how and if administrative records should be
incorporated into the enumeration process; and
o determines privacy issues, legal issues, or both and solutions and
identifies a process to implement these findings into operational
procedures.
Appendix III 2010 Census Planning Organization
The Self Response Options Planning Group
o identifies the optimal mix of options, including mail, Internet, and
telephone, by which the public can respond to the census;
o identifies options for accommodating responses by people who believe
they have not been counted by the traditional methods;
o identifies messages and methods to inform, advertise, and motivate
census questionnaire response in a multimode self-response environment;
o determines the optimal mailing strategy;
o identifies methods and options for delivering a replacement
questionnaire;
o examines methods for making the identification number shorter and
easier to handle;
o develops an integrated self-response that optimizes requirements from
other program areas, such as language and self-initiated response
requirements; and
o develops strategies for accommodating response by people with
disabilities.
The Overseas Enumeration Planning Group
o makes recommendations to the Bureau's executive staff on key policy
issues, such as applying residence rules overseas, goals of counting
Americans living overseas, the definition of this universe, and conducting
a count that would take a different form than the stateside enumeration;
o determines possible methods for enumerating Americans overseas in
accordance with decisions on the policy issues;
o determines the type of evaluations that will be conducted to assess the
quality of data captured during this operation; and
o defines a process and plan for consultations with stakeholders.
Appendix III 2010 Census Planning Organization
The Field Activities Planning Group
o determines the optimal fit of new technologies, such as mobile
computing devices, within the enumeration methods and processes;
o determines the optimal and most cost-effective field infrastructure to
support field activities;
o determines the possible enumeration methods, such as Update/Leave,
Urban Update/Leave, and Update/Enumerate and their feasibility for the
2010 short-form-only census;
o determines how to apply the criteria established for the areas in which
each type of enumeration method will be used;
o determines how best to target for special methods areas where mail
delivery methods are not optimal, for example, areas where residents use
post office boxes;
o determines the most effective special methods or tool kit approaches
for enumerating housing units in hard to enumerate areas, such as gated
communities;
o determines the most efficient use of the telephone data collection
methodology to reduce personal field visits;
o determines the most effective management and implementation approach
that will ensure integrated quality assurance in field activities;
o determines the best methodology for delineating assignment areas; and
o determines data requirements for field instruments relating to
management and quality control needs.
The Coverage Planning Group
o proposes content rules for identifying which addresses to include in
the 2010 Census;
o proposes consistent and accurate approaches for improving matching
rules to reduce duplication of addresses;
Appendix III 2010 Census Planning Organization
o determines the appropriate use of administrative records for MAF
activities, such as targeting and/or updating MAF;
o determines areas in which MAF and TIGER need improvement;
o determines best methods for targeting areas that need updating;
o develops requirements for the positional accuracy of MAF/TIGER in
coordination with the Global Positioning System Applications Strategy
Group;
o defines and assesses methods to improve the address list and coverage
of the 2010 Census after mail-out address file is final;
o defines criteria for determining the areas in which each type of
enumeration method will be used;
o in coordination with the Special Place/Group Quarters Planning Group,
develops a process that ensures the integration of special place/group
quarters addresses with housing unit addresses and subsequent updates of
the MAF/TIGER system;
o reviews the enumeration and residence rules, taking into consideration
implementation issues, and recommends appropriate revisions;
o defines and assesses appropriate methods for dealing with usual home
elsewhere situations on all questionnaires and instruments;
o determines methods for avoiding multiple listings of persons within and
between households;
o determines methods for improving within household coverage, including
the Be Counted program;
o develops thresholds reasonable for count imputations; and
o determines best count imputation method, including the use of spatial
analysis tools, to determine locational relationships.
Appendix III 2010 Census Planning Organization
The Software and Systems Planning Group
o determines acceptable industry standards and practices for the software
process;
o identifies and assesses alternative standards and practices for
decennial applications for requirements development, software analysis and
design, testing methods, quality assurance, configuration management,
documentation of all components, and risk management;
o recommends standards and practices for the software process;
o identifies and assesses the need for a common data dictionary,
including integrating it with the corporate meta-data project;
o recommends whether to develop a common data dictionary solution, and if
adopted, develops a rationale for its use and defines its functional
requirements;
o reviews deliverables and recommends requirements for the Technical
Architecture and Infrastructure contract in preparation for development of
the 2010 Census systems architecture;
o evaluates potential systems components;
o recommends approach for developing and selecting the 2010 Census
physical architecture;
o recommends physical architecture solution;
o for mobile computing devices (MCD),
o determines which operations will test the use of the MCDs,
o determines what functionality will be employed in MCDs for testing,
o determines which specific type(s) of MCDs and which software
applications the Bureau will test, and
o considers the implications for field infrastructure, particularly with
respect to the technology needed to support MCD applications;
Appendix III 2010 Census Planning Organization
o for Internet response mode,
o tests alternatives for validating/confirming receipt of census
questionnaire/response,
o researches and tests mechanisms for respondents to generate e-mail to
the Bureau,
o identifies options for e-mail to respondents with "census message,"
questionnaire, or both,
o researches and tests alternatives for serving multiple forms (such as
those needed for the language program and overseas enumeration),
o researches and tests use of census housing unit identification,
o determines effective ways to maintain security,
o researches and tests alternative interfaces, and
o identifies alternatives for meeting accessibility needs;
o for high technology data collection modes,
o researches ways to utilize Web television,
o researches ways to utilize kiosks, and
o identifies alternatives for meeting accessibility needs; and
o for telephone response modes,
o researches and tests alternatives for initial response by voice
recognition/IVR/touch tone/agent assisted,
o tests alternatives for automated dial-up "census message,"
o researches and tests alternatives for languages,
o determines effective ways to maintain security, and
o identifies alternatives for meeting accessibility needs.
Appendix III 2010 Census Planning Organization
The Communications Planning Group
o determines how the public will be divided into groups in order to
target specific messages;
o determines the most appropriate messages that will resonate with the
Bureau's audiences and motivate them to respond and the strategies for
creating a brand (name/image) for the Bureau;
o develops innovative strategies using 2000 Census and ACS data to reach
the
o hard to count population,
o emerging populations,
o rural communities, and
o non-English-speaking populations;
o develops media strategies and messages for media outlets, operations
that need specific advertising or publicity, and crisis communications
management;
o considers 2010 Census cost implications; and
o determines the best methods for evaluating the effectiveness of the
components of the integrated marketing strategy.
The Language Programs Planning Group
o researches the availability, design, and use of bilingual forms;
o researches and develops alternative operational testing requirements
for
o alternatives for an English/Spanish mail-out questionnaire,
o the languages the 2010 Census will support,
o questionnaire assembly,
Appendix III 2010 Census Planning Organization
o process for requesting a questionnaire in a specific language,
o questionnaire/guide distribution,
o validation of a bilingual questionnaire,
o alternatives for a bilingual enumerator instrument,
o questionnaire data capture,
o data entry for all collection modes,
o accessibility for in-language respondents (Braille, audio, etc.), and
o partnership responsibilities; and
o considers Advisory Committee lessons learned and recommendations. The
Race and Ethnicity Planning Group
o determines if revised instructions to the questions on race and
Hispanic origin convey the federal government's distinction of race and
Hispanic origin as two separate concepts that require responses that
reflect the intent of each question;
o determines alternative wording to the questions on race and ethnicity;
o determines the type of analyses and evaluations needed to assess the
quality of race and ethnicity data from the 2000 Census and ACS;
o determines if the exclusion of a "Some other race" response category in
the question on race will affect race reporting;
o determines if changing the order of the race response categories
affects race reporting;
o determines what effect the inclusion of examples for "Other Asian" and
"Other Pacific Islander" have on race reporting;
o determines if the use of shortened response categories during telephone
and other data collection modes will affect respondents' understanding of
the response options;
Appendix III 2010 Census Planning Organization
o determines whether alternative terminology (e.g., "Print race or races"
instead of "Print category or categories") will affect race reporting;
o determines whether a revised Hispanic question that does not include
"Spanish" as an identifier will increase reporting of specific Hispanic
origin responses in the "Other Hispanic" response category; and
o determines whether the inclusion of examples for detailed Hispanic
groups will increase specified responses in the "Other Hispanic" response
category.
The American Indian and Alaska Native (AIAN) Planning Group
o determines what enumeration methods will be tested in AIAN areas;
o determines outreach and partnership strategies for testing in AIAN
areas;
o determines strategies and explores methodologies for improving
coverage; and
o assesses developments or innovations needed for mapping and enumeration
activities.
GAO's Mission The General Accounting Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting its
constitutional responsibilities and to help improve the performance and
accountability of the federal government for the American people. GAO
examines the use of public funds; evaluates federal programs and policies;
and provides analyses, recommendations, and other assistance to help
Congress make informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony
The fastest and easiest way to obtain copies of GAO documents at no cost
is through the Internet. GAO's Web site (www.gao.gov) contains abstracts
and fulltext files of current reports and testimony and an expanding
archive of older products. The Web site features a search engine to help
you locate documents using key words and phrases. You can print these
documents in their entirety, including charts and other graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as "Today's Reports," on its
Web site daily. The list contains links to the full-text document files.
To have GAO e-mail this list to you every afternoon, go to www.gao.gov and
select "Subscribe to e-mail alerts" under the "Order GAO Products"
heading.
Order by Mail or Phone The first copy of each printed report is free.
Additional copies are $2 each. A check or money order should be made out
to the Superintendent of Documents. GAO also accepts VISA and Mastercard.
Orders for 100 or more copies mailed to a single address are discounted 25
percent. Orders should be sent to:
U.S. General Accounting Office 441 G Street NW, Room LM Washington, D.C.
20548
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061
To Report Fraud, Contact: Web site: www.gao.gov/fraudnet/fraudnet.htm
Waste, and Abuse in E-mail: [email protected]
Federal Programs Automated answering system: (800) 424-5454 or (202)
512-7470
Public Affairs Jeff Nelligan, Managing Director, [email protected] (202)
512-4800 U.S. General Accounting Office, 441 G Street NW, Room 7149
Washington, D.C. 20548
Presorted Standard
Postage & Fees Paid
GAO
Permit No. GI00
United States
General Accounting Office
Washington, D.C. 20548-0001
Official Business
Penalty for Private Use $300
Address Service Requested
*** End of document. ***