2000 Census: Preparations for Dress Rehearsal Leave Many Unanswered
Questions (Chapter Report, 03/26/98, GAO/GGD-98-74).
Pursuant to a congressional request, GAO reviewed the status of key
activities that the Bureau of the Census plans to test as part of its
dress rehearsal for the 2000 Census.
GAO noted that: (1) address list development, local outreach and
promotion, staffing, and statistical sampling are all still facing the
developmental and implementation changes that led GAO in 1997 to raise
concerns about the high risk of a failed census in 2000; (2) this
situation has arisen in part because Congress and the administration
have yet to reach agreement on key aspects of the census design; (3) an
accurate address list and associated maps are essential for ensuring
that households receive census questionnaires and for establishing the
universe of housing units for sampling and statistical estimation
procedures; (4) accurate maps are critical for locating households for
congressional reapportionment and redistricting; (5) the Bureau's
initial plan for creating the address list and maps for the 2000 Census
called for: (a) combining addresses provided by the Postal Service with
the Bureau's 1990 Census address file and mapping database; and (b)
submitting these addresses and maps to local governments for their
review; (6) the Bureau has since concluded that the resulting address
file might not meet its goal of being 99 percent complete; (7) in
September 1997, the Bureau announced plans to reengineer its process for
creating the address file for the 2000 census, in part by hiring
temporary staff to physically verify the accuracy of all addresses by
canvassing streets; (8) this reengineered approach has not been tested
because of a lack of resources, and will not be tested because the
Bureau does not have sufficient time to do so; (9) as part of its
reengineering effort, the Bureau plans to seek earlier local government
review of addresses from the Postal Service and the Bureau's 1990 Census
file; (10) the Bureau's approach still will not address some of the
problems encountered by local officials in their reviews; (11) to boost
the questionnaire mail response rate and thus reduce its costly
nonresponse follow-up workload, the Bureau plans to partner with local
governments, schools and other community organizations to promote the
census and has initiated a paid advertising campaign that is being
developed by a private contractor; (12) the Bureau estimates that it
will need to recruit over 2.6 million applicants to fill about 295,000
office and field positions; (13) the dress rehearsal does not and cannot
provide an adequate test of the effectiveness of the Bureau's staffing
strategy for the 2000 Census; and (14) several missteps by the Bureau in
drawing a sample for the Integrated Coverage Measurement have raised
GAO's concerns about the Bureau's quality control procedures.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: GGD-98-74
TITLE: 2000 Census: Preparations for Dress Rehearsal Leave Many
Unanswered Questions
DATE: 03/26/98
SUBJECT: Census
Data collection
Population statistics
Statistical methods
Mailing lists
Surveys
Reengineering (management)
Data integrity
Human resources utilization
Quality assurance
IDENTIFIER: Census Integrated Coverage Measurement
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO report. Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved. Major **
** divisions and subdivisions of the text, such as Chapters, **
** Sections, and Appendixes, are identified by double and **
** single lines. The numbers on the right end of these lines **
** indicate the position of each of the subsections in the **
** document outline. These numbers do NOT correspond with the **
** page numbers of the printed product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
** A printed copy of this report may be obtained from the GAO **
** Document Distribution Center. For further details, please **
** send an e-mail message to: **
** **
** **
** **
** with the message 'info' in the body. **
******************************************************************
Cover
================================================================ COVER
Report to the Committee on Governmental Affairs,
U.S. Senate
March 1998
2000 CENSUS - PREPARATIONS FOR
DRESS REHEARSAL LEAVE MANY
UNANSWERED QUESTIONS
GAO/GGD-98-74
Preparations for Dress Rehearsal
(410220)
Abbreviations
=============================================================== ABBREV
ACF - Address Control File
CCC - Complete Count Committee
DSF - Delivery Sequence File
FTE - full-time equivalent
ICM - Integrated Coverage Measurement
LUCA - Local Update of Census Addresses
MAF - Master Address File
PES - Post-Enumeration Survey
Y&R - Young & Rubicam
Letter
=============================================================== LETTER
B-278755
March 26, 1998
The Honorable Fred Thompson
Chairman
The Honorable John Glenn
Ranking Minority Member
Committee on Governmental Affairs
United States Senate
This report, which was prepared at your request, reviews the status
of key activities that the U.S. Bureau of the Census plans to test
as part of its dress rehearsal for the 2000 Census. To the extent
that it mirrors the decennial, the dress rehearsal could foreshadow
how well these activities might work in 2000, and thus indicate where
additional congressional and Bureau attention is needed now to ensure
a successful census.
As agreed with the Committee, this report reviews the results to date
of the Bureau's efforts to create a complete and accurate address
list; increase the mail response rate through outreach and promotion;
staff its operations with an adequate workforce; and reduce costs and
improve accuracy through sampling and statistical estimation methods.
We are sending copies of this letter to the Chairman and Ranking
Minority Member, Subcommittee on the Census, House Committee on
Government Reform and Oversight; other appropriate congressional
committees; the Secretary of Commerce; the Director, Bureau of the
Census; and other interested parties. Copies will be made available
to others on request.
Please call me or J. Christopher Mihm, Associate Director, on (202)
512-8676 if you have any questions concerning this report. The major
contributors to this report are listed in appendix II.
L. Nye Stevens
Director, Federal Management
and Workforce Issues
EXECUTIVE SUMMARY
============================================================ Chapter 0
PURPOSE
---------------------------------------------------------- Chapter 0:1
In February 1997, GAO designated the 2000 Decennial Census as being
at high risk for wasted expenditures and unsatisfactory results.\1 In
the months that followed, the U.S. Bureau of the Census continued
with its preparations for the 2000 Census, presenting its latest plan
in a legislatively mandated report to Congress in July 1997.\2
Separately, GAO in the same month reported that risks of a failed
census in 2000 had increased.\3
Because of their interest in determining what progress, if any, the
Bureau has made in reducing these risks since mid-1997, the Chairman
and Ranking Minority Member, Senate Committee on Governmental
Affairs, asked GAO to review the status of several census-taking
activities that are key to the success of the 2000 Census. The 1998
dress rehearsal is the Bureau's last opportunity for an operational
test of its overall design of the 2000 Census and to demonstrate to
Congress and other stakeholders the feasibility of its plans.
Similar to the 2000 Census, the dress rehearsal consists of a number
of activities spanning a broad period of time.
As agreed with the Committee, the objective of this report is to
review the status of key activities that the Bureau plans to test as
part of its dress rehearsal for the decennial census: address list
development, local outreach and promotion, staffing, and statistical
sampling.
--------------------
\1 High-Risk Series (GAO/HR-97-2, Feb. 1997).
\2 Census 2000 Operational Plan, Bureau of the Census, July 1997.
\3 2000 Census: Progress Made on Design, but Risks Remain
(GAO/GGD-97-142, July 14, 1997).
BACKGROUND
---------------------------------------------------------- Chapter 0:2
The decennial census is the nation's most comprehensive and expensive
statistical data-gathering program. Accurate results are critical
because, as required by the Constitution, decennial census data are
used to reapportion seats in the House of Representatives. Countless
decisions affecting governments, businesses, and private citizens
also depend on census data.
The 1990 Census was the most costly in history, and it produced data
that were less accurate than those from the 1980 Census, leaving
millions of Americans uncounted. GAO, Congress, the Bureau, and
others concluded that the established approach used for taking the
census in 1990 had exhausted its potential for counting the
population cost-effectively, and that fundamental design changes were
needed to reduce census costs and improve the quality of the data
collected. In response, the Bureau redesigned key features, such as
instituting procedures for developing a complete and accurate address
list, increasing the mail response rate through outreach and
promotion, staffing census-taking operations with a capable
workforce, and reducing costs and improving accuracy through sampling
and statistical estimation. However, Congress has not endorsed the
Bureau's overall design because of its concerns over the validity and
legality of the Bureau's statistical sampling and estimation
procedures.
In concept, the 1998 dress rehearsal for the 2000 Census should test
almost all of the various operations and procedures planned for the
decennial census under as close to census-like conditions as
possible. The dress rehearsal sites include Sacramento, CA; 11
counties in the Columbia, SC, area; and Menominee County in
Wisconsin, including the Menominee American Indian Reservation.
RESULTS IN BRIEF
---------------------------------------------------------- Chapter 0:3
The key activities that GAO examined for this report, including
address list development, local outreach and promotion, staffing, and
statistical sampling, are all still facing the developmental and/or
implementation challenges that led GAO in 1997 to raise concerns
about the high risk of a failed census in 2000. For example, the
dress rehearsal, which originally was intended to be a full-scale
demonstration of a well-defined census design for 2000, because of
these unresolved issues, will leave a number of design questions
unanswered. This situation has arisen in part because Congress and
the administration have yet to reach agreement on key aspects of the
census design. And, the longer this disagreement continues and key
desion issues remain unresolved, the greater the risk of a failed
2000 Census.
An accurate address list and associated maps are essential for
ensuring that households receive census questionnaires and for
establishing the universe of housing units for sampling and
statistical estimation procedures. Accurate maps are critical for
locating households for congressional reapportionment and
redistricting. The Bureau's initial plan for creating the address
list and maps for the 2000 Census called for (1) combining addresses
provided by the U.S. Postal Service with the Bureau's 1990 Census
address file and mapping database and (2) submitting these addresses
and maps to local governments for their review. However, the Bureau
has since concluded that the resulting address file might not meet
its goal of being 99 percent complete. Because of this, in September
1997, the Bureau announced plans to "reengineer" its process for
creating the address file for the 2000 Census, in part by hiring
temporary staff to physically verify the accuracy of all addresses by
canvassing streets. However, this reengineered approach has not been
tested because of a lack of resources, and will not be tested because
the Bureau does not have sufficient time to do so. Thus, it is not
known if the Bureau's reengineered procedures will allow it to
achieve its goal of having a 99 percent complete address list for the
2000 Census.
Moreover, local review of address list information did not go
smoothly at the dress rehearsal sites. Many jurisdictions did not
participate because they said they lacked the resources, while
participating jurisdictions encountered problems, such as the level
of Bureau assistance, that impeded their review. As part of its
reengineering effort, the Bureau plans to seek earlier local
government review of addresses from the Postal Service and the
Bureau's 1990 Census address file. However, the Bureau's approach
still will not address some of the problems encountered by local
officials in their reviews.
To boost the questionnaire mail response rate and thus reduce its
costly nonresponse follow-up workload, the Bureau plans to partner
with local governments, schools, and other community organizations to
promote the census and has initiated a paid-advertising campaign that
is being developed by a private contractor. However, during the
dress rehearsal, inconsistent local participation in grassroots
promotion efforts, and limited local and Bureau resources, raise the
risk that the Bureau may not be able to rely on these initiatives to
play a significant role in motivating public response and achieving
the Bureau's goal of a 66.9 percent mail response rate in 2000.
Furthermore, the paid-advertising campaign faces the dual challenge
of building census awareness and encouraging people to return their
questionnaires in spite of a number of demographic, cultural,
attitudinal, and other barriers.
The Bureau estimates that it will need to recruit over 2.6 million
applicants to fill about 295,000 office and field positions.
However, several uncertainties surround the Bureau's ability to
accomplish this goal. In addition to the sheer number of people
needed, most census jobs are part-time and temporary, provide few
benefits, and thus may not be as attractive to potential applicants
as other types of work. Although the Bureau has taken steps to
address these staffing obstacles, such as basing pay on local wage
rates, if current employment trends continue, the Bureau could find
itself competing for workers in a tight labor market. Furthermore,
the Bureau's decision to focus its recruitment efforts on individuals
who are currently employed and are seeking additional work
("moonlighters") and retirees is based on limited data that may not
be comparable nationwide. Likewise, the dress rehearsal does not and
cannot provide an adequate test of the effectiveness of the Bureau's
staffing strategy for the 2000 Census. Although Bureau officials
have noted that the dress rehearsal operations that have occurred to
date have been adequately staffed, employment conditions at the dress
rehearsal sites may not be representative of what the Bureau will
experience nationally in 2000.
The Bureau's plans for the 2000 Census include using sampling and
statistical estimation techniques to reduce the nonresponse follow-up
workload and using the Integrated Coverage Measurement (ICM), which
is a procedure designed to improve the accuracy of the census
population count by reconciling the original census counts with data
obtained from an independent sample of housing units, and using the
results to adjust the census. However, the Bureau's plans face a
variety of operational challenges. These challenges include
methodological and technological issues such as conducting the census
under a more compressed schedule compared to that of the 1990 Census,
obtaining a high completion rate for the sample-based data collection
activities, and overcoming limitations in the software used to match
data during the ICM. Several missteps by the Bureau in drawing a
sample for the ICM have raised GAO's concerns about the Bureau's
quality control procedures.
PRINCIPAL FINDINGS
---------------------------------------------------------- Chapter 0:4
THE ACCURACY OF THE BUREAU'S
ADDRESS LISTS AND MAPS IS
UNCERTAIN, AND LOCAL REVIEWS
MAY BE TOO SPORADIC TO
GREATLY IMPROVE THEM
-------------------------------------------------------- Chapter 0:4.1
To build its address list, which is known as the Master Address File,
the Bureau planned, in part, to (1) use addresses provided by the
Postal Service, (2) merge these addresses with the address file the
Bureau created during the 1990 Census, (3) conduct limited checks of
the accuracy of selected addresses, and (4) send the addresses to
local governments for verification.
However, the Bureau's analyses of the completeness of the Postal
Service's addresses, when combined with the Bureau's 1990 addresses
for a selected number of locations, have shown that the resulting
address list was not always complete. For example, address lists
created in 1995 for two test locations did not include from 3.6 to
6.4 percent of the addresses identified through other Bureau
operations, such as field verification. Following these and similar
analyses for lists created in 1996, in September 1997, the Bureau
concluded that primary reliance on the Postal Service's and the
Bureau's 1990 address files was not sufficient, and that it needed to
reengineer its process to achieve an address file for the 2000 Census
that, as a whole, is 99 percent complete. Under the new procedures,
the Bureau now plans to canvass neighborhoods across the nation to
physically verify the completeness and accuracy of the file for the
2000 Census. The reengineered approach is estimated to cost an
additional $108.7 million. While the components of the new approach
have been used and tested in the past, the Bureau has not used or
tested them in concert with each other, nor in the sequence as
presently designed for use in the 2000 Census, and does not plan to
do so in the dress rehearsal. Consequently, it will not be known
until the 2000 Census whether the Bureau's redesigned procedures will
allow it to meet its goal.
The dress rehearsal results to date suggest that local participation
may be too inconsistent and face too many obstacles to consistently
verify or increase the accuracy of the address file and maps. For
example, despite the Bureau's efforts to encourage all local
jurisdictions to participate, just 34 of the 60 local jurisdictions
involved with the dress rehearsal participated in the verification
effort. Reasons for the low participation rate included the lack of
resources and/or information to review address lists or maps at the
local level. Jurisdictions that did participate in this effort
identified problems with the level of Bureau assistance as well as
with the accuracy and completeness of the address lists and maps,
which impeded their review efforts. The Bureau's reengineered
address development procedures call for obtaining earlier assistance
from local governments to review addresses and maps. However, this
does not address all of the problems encountered by local officials
in reviewing address lists during the dress rehearsal, such as the
availability of assistance and the quality of the address list and
maps. (See ch. 2.)
THE BUREAU'S OUTREACH AND
PROMOTION EFFORTS FACE
OBSTACLES THAT COULD IMPEDE
ITS ABILITY TO ACHIEVE ITS
MAIL RESPONSE RATE OBJECTIVE
-------------------------------------------------------- Chapter 0:4.2
To help increase the mail response rate and thus reduce its costly
nonresponse follow-up workload, the Bureau plans to partner with
local governments and other organizations to raise public awareness
of the census. However, implementation difficulties could adversely
affect the Bureau's ability to achieve its goal of a 66.9 percent
mail response rate in 2000.
Complete Count Committees, which are critical components of the
Bureau's local outreach and promotion efforts, are to consist of
elected, business, community, social service, religious, and other
local leaders who are to help mobilize grassroots promotion efforts.
However, GAO found that not all of the dress rehearsal jurisdictions
that the Bureau hoped would establish committees had done so. For
example, in South Carolina, of the 11 counties and the City of
Columbia participating in the dress rehearsal, just 3 counties and
Columbia had active committees at the time of GAO's review. The
eight remaining counties either had not started committees or had
formed committees that subsequently became inactive.
According to local officials, vague guidance and expectations from
the Bureau played a role in the effort's mixed results. GAO also
found that local and Bureau resources may be too limited to organize
an effective local outreach campaign through the Complete Count
Committees. Officials from five South Carolina counties and the City
of Columbia indicated that resources to carry out outreach and
promotion activities were limited, while Bureau personnel might be
stretched too thinly. Bureau officials noted that the Bureau made a
conscious policy decision not to fund local governments' outreach and
promotion activities because the Bureau did not see itself as a
funding organization. Instead, the Bureau hopes to assist local
governments through its partnership specialists.
Moreover, although the Bureau has contracted with a private firm to
conduct a multimedia, national and locally targeted advertising
campaign, the firm faces not only the familiar task of developing
public awareness of the census, but also the greater challenge of
motivating people to return their questionnaires in spite of a
long-term decline in the mail response rate. While the Bureau found
that 93 percent of the public was aware of the census in 1990, the
mail response rate was only 65 percent, 10 percentage points lower
than it was in 1980. (See ch. 3).
THE BUREAU COULD ENCOUNTER
DIFFICULTIES STAFFING THE
2000 CENSUS
-------------------------------------------------------- Chapter 0:4.3
For the 2000 Census, the Bureau estimates that it will need to
recruit over 2.6 million applicants to fill about 295,000 positions.
Aside from the large numbers of people needed, hiring census workers
could be difficult because most census jobs are part-time and
temporary and do not come with such benefits as health insurance.
Consequently, potential applicants may not find census jobs as
attractive as alternative work opportunities.
To address this staffing challenge, the Bureau plans to focus its
recruitment efforts on moonlighters and retirees. The Bureau also
plans to pay employees a wage that is based on local rates and offer
productivity incentives. However, if current employment trends
continue, the Bureau could find itself recruiting workers in a
tighter labor market than prevailed in 1990. Furthermore, the
Bureau's decision to focus its recruitment efforts on moonlighters
and retirees is based on informal discussions with census workers
during the 1995 Census Test, the hiring practices of private survey
research firms, and census workforce studies that may not necessarily
be comparable to the 2000 Census.
According to Bureau officials, the Bureau has been able to adequately
staff the dress rehearsal operations that have occurred to date.
However, the dress rehearsal is not a good test of the Bureau's
staffing strategy for 2000. The labor market at the dress rehearsal
sites is not necessarily representative of the one the Bureau will
encounter nationwide in 2000. Also, while the Bureau can compensate
for any staffing shortages at the dress rehearsal sites by shifting
employees or recruiting outside of the dress rehearsal jurisdictions,
the Bureau's ability to take such measures in 2000 will be more
limited because the demands of taking the census nationally will
reduce the availability of such individuals. Finally, if the current
debate over the use of sampling is resolved by deciding to follow up
on all nonresponding households, the Bureau estimates that an
additional 59,000 enumerators would be needed to do this follow up,
and an additional 25,000 to 30,000 enumerators would be needed to
verify vacant housing units--which would add to potential recruiting
difficulties. (See ch. 4.)
THE BUREAU'S SAMPLING AND
STATISTICAL ESTIMATION
DESIGN FACES METHODOLOGICAL,
TECHNOLOGICAL, AND QUALITY
CONTROL CHALLENGES
-------------------------------------------------------- Chapter 0:4.4
For the 2000 Census, the Bureau anticipates that the proportion of
households returning census questionnaires will be about 66.9
percent--about the same as in the 1990 Census. Rather than following
up on all remaining households as it did in 1990, the Bureau plans to
conduct a sampling of nonresponding households. In addition, the
Bureau's ICM procedures are designed to measure and adjust for any
inaccuracies in the population count.
GAO's July 1997 report noted that sampling for nonresponse follow-up
could reduce costs and save time, while the ICM could improve the
accuracy of the population totals.\4 However, GAO also noted that to
achieve these objectives, sampling and the ICM would need to be
effectively implemented. Unfortunately, the Bureau's sampling and
statistical estimation plans face a number of unresolved operational
challenges that could jeopardize their success in 2000. It is
uncertain whether the Bureau will complete its nonresponse follow-up
and ICM operations in the time allotted, considering that in 1990
similar processes took longer even though the amount of work was
less. Uncertainties also surround the Bureau's ability to achieve
its target of a 98 percent completion rate for ICM operations. If
the ICM completion rate falls below 98 percent, there is increased
risk that ICM-based adjustments to census data may not accurately
reflect different segments of the population. The need to obtain a
high response rate in a short time could require the Bureau to
collect data from neighbors and other individuals outside of the
nonresponding households, a method that has been shown in the past to
be less accurate.
Technologically, the software the Bureau plans to use in properly
identifying and matching ICM data to those data collected from the
same household during the census enumeration phase has limitations
and could preclude a match between individuals counted in the census
enumeration and those counted in the ICM. These limitations could
affect the precision of the adjusted population counts.
The Bureau has made several missteps in drawing an ICM sample for the
dress rehearsal, such as inadvertently including commercial
addresses. Because these errors went undetected until relatively
late in the sample selection process, GAO is concerned about the
Bureau's ability to catch and correct problems during this critical
phase of the 2000 Census in a timely manner. (See ch. 5.)
--------------------
\4 GAO/GGD-97-142.
RECOMMENDATIONS
---------------------------------------------------------- Chapter 0:5
Since plans for the dress rehearsal operations are generally
complete, GAO is not making any recommendations in this report.
AGENCY COMMENTS AND GAO'S
EVALUATION
---------------------------------------------------------- Chapter 0:6
GAO received written comments on a draft of this report from the
Acting Director of the Bureau of the Census. While agreeing that the
2000 Census still faces operational challenges--building an accurate
address list, motivating people to participate, staffing, and
refining its sampling plans under tight schedule constraints--the
Acting Bureau Director expressed confidence that these activities are
well under control and that, through the adjustments under way, the
challenges will be met. GAO remains concerned, however, that
important questions remain unanswered at a disturbingly late stage in
the census cycle when the emphasis should be on fine tuning rather
than basic revisions to operational plans.
The Bureau's comments are discussed further at the end of chapter 6
and reprinted in full in appendix I.
INTRODUCTION
============================================================ Chapter 1
The decennial census is the nation's most comprehensive and expensive
statistical data-gathering program. An accurate population count is
critical because, as required by the Constitution, census results are
used for allocating seats in the House of Representatives. Census
data are also used for redistricting within states, distributing
billions of dollars in federal funding, guiding investment decisions
of the public and private sectors for the ensuing decade, and
providing a baseline for countless other statistical measurement
programs.
Unfortunately, as we noted in a previous report, the nation runs the
risk of a very expensive and seriously flawed census in 2000.\5 To
assess the Bureau of the Census' progress in minimizing risks, we
reviewed the status of key activities that the Bureau plans to test
as part of its dress rehearsal for the next decennial census:
address list development, local outreach and promotion, staffing, and
statistical sampling. To the extent that the dress rehearsal
accurately mirrors the actual census, the results of the dress
rehearsal could indicate what might occur in 2000.
--------------------
\5 GAO/HR-97-2.
THE CENSUS DRESS REHEARSAL IS
TO PROVIDE A FULL-SCALE
DEMONSTRATION OF MOST
PROCEDURES PLANNED FOR 2000
---------------------------------------------------------- Chapter 1:1
According to the Census Bureau, the dress rehearsal for the 2000
Census is designed to test the various operations, procedures, and
questions that are planned for the decennial under as near
census-like conditions as possible. In July 1996, the Bureau
announced that Sacramento, CA; 11 counties in the Columbia, SC, area;
and Menominee County, WI, including the Menominee American Indian
Reservation, had been selected as sites for the 2000 Census dress
rehearsal.
According to the Bureau, Sacramento was selected because its diverse
population will allow the Bureau to test estimation methods designed
to reduce differential undercounts. Sacramento is also a primary
media market, thus enabling the Bureau to test its paid-advertising
campaign. The Columbia site was selected, among other reasons, for
its mixed rural and urban features and because its relatively high
proportion of African-Americans will give the Bureau an opportunity
to test methods of reducing the differential undercount for this
population group. The Menominee site was included to allow the
Bureau to test measures aimed at reducing the undercount of American
Indians.
Administratively, the Bureau's Seattle Regional Office is responsible
for conducting the Sacramento dress rehearsal, while its Charlotte,
NC, and Chicago, IL, Regional Offices, respectively, are responsible
for the South Carolina and Menominee dress rehearsals.
An important aspect of the dress rehearsal activities is Dress
Rehearsal Census Day, which is scheduled for April 18, 1998.
However, as is the case with the actual census, the Bureau's dress
rehearsal activities span a much wider period of time than 1 day.
Following the selection of the dress rehearsal sites in July 1996,
the Bureau developed preliminary mailing lists and materials for
these locations, contacted local governments at the three sites, and
conducted staffing activities to hire temporary census employees in
those locations. Similarly, after April 18, the Bureau is to develop
its census count by conducting the necessary follow-up activities at
nonresponding households and completing other fieldwork.
A DEMONSTRATION OF THE BUREAU'S
PLANS IS CRITICAL GIVEN PAST
PROBLEMS
---------------------------------------------------------- Chapter 1:2
A well-planned and -executed dress rehearsal should provide the
Bureau with an opportunity to demonstrate the feasibility of its
design for the 2000 Census. This demonstration is particularly
important considering the problems the Bureau has experienced to date
and the controversy that surrounds its design.
Since 1970, when measured in constant dollars, census costs have
increased faster than inflation, even after allowing for population
growth. In 1990 constant dollars, total census cycle costs were $0.7
billion in 1970, $1.8 billion in 1980, and $2.6 billion in 1990.\6
The Bureau expects that the escalating cost of conducting the census
will continue; in March 1997, the Bureau estimated that the 2000
Census will cost at least $4.0 billion in 2000 dollars.
Unfortunately, the nation's growing investment in the census has not
generated more accurate results. Although the 1990 Census was the
most expensive in history, the results were less accurate than those
of the 1980 Census. The Bureau estimated that about 6 million
persons were counted twice in the 1990 Census, while 10 million
persons were missed. These errors were particularly worrisome
because the Bureau's evaluation showed a persistent differential
undercount of minority groups and renters, particularly those living
in rural areas.
The cost of a census is largely influenced by the relative need for
the Bureau to follow up on nonresponding housing units. Since the
1970 Census, the Bureau has used a "mail-out/mail-back" approach for
counting the vast majority of the population whereby the Bureau
develops an address list of the nation's housing units and mails a
questionnaire to each one. Although households are to mail back
completed forms, many millions do not. This nonresponse requires the
Bureau to hire hundreds of thousands of temporary employees, called
enumerators, to follow up on each nonresponding housing unit by
visiting each unit and gathering census information. The mail
response rate to the census questionnaire has dropped from 78 percent
in 1970 to 65 percent in 1990, thus increasing the Bureau's costly
nonresponse follow-up workload.
--------------------
\6 Constant-dollar value is measured in terms of prices for a base
period to remove the influence of inflation. The resulting
constant-dollar value is the value that would exist if prices had
remained the same as in the base period.
INCREASING CONGRESSIONAL
CONCERN HAS LED TO CHANGES TO
THE DRESS REHEARSAL
---------------------------------------------------------- Chapter 1:3
The Bureau's plans for the 2000 Census include the use of sampling
and statistical estimation. Initially, the Bureau planned to use
these methods at the three dress rehearsal sites just as it plans to
do nationwide in 2000. Congressional concern over the validity and
legality of the Bureau's sampling and statistical estimation methods
have led to recent changes to the dress rehearsal. According to
Department of Commerce officials, as part of a compromise between the
administration and Congress over the Bureau's fiscal year 1998
appropriation,\7 it was agreed that the Bureau would use sampling and
statistical estimation methods only in the Sacramento site for
nonresponse follow-up and to improve the accuracy of the population
count. In the Columbia site, the Bureau is to hire enumerators to
follow up on all nonresponding households, just as it did nationally
in the 1990 Census. At the Menominee dress rehearsal site, the
Bureau is to do a 100 percent nonresponse follow-up but is to use
sampling to improve the accuracy of the population count. Although
use of the different methods at the dress rehearsal sites invites a
comparison of the results, geographic, demographic, and possibly
other differences between the dress rehearsal locations preclude such
a comparison.
The Bureau's fiscal year 1998 appropriation act also allows for
expedited judicial review of sampling and created an eight-member
Census Monitoring Board that is to oversee all aspects of the
preparation and implementation of the 2000 Census and report its
findings to Congress.\8
--------------------
\7 Departments of Commerce, Justice, and State, the Judiciary, and
Related Agencies Appropriations Act, 1998, P.L. 105-119, Nov. 26,
1997.
\8 In February 1998, House Republicans and a public interest law firm
each filed a lawsuit challenging the Bureau's planned use of
sampling.
OBJECTIVES, SCOPE, AND
METHODOLOGY
---------------------------------------------------------- Chapter 1:4
Concerned about the risk of a failed census, the Chairman and Ranking
Minority Member, Senate Committee on Governmental Affairs, asked us
to review the status of key census-taking activities that the Bureau
is to test during the 1998 dress rehearsal. In response, we examined
the Bureau's efforts to (1) create a complete and accurate address
list, (2) increase the mail response rate through local outreach and
promotion, (3) staff census-taking operations with an adequate
workforce, and (4) implement its sampling and statistical estimation
methodology. If designed and implemented as much like the actual
census as possible, the dress rehearsal could yield results
indicative of what might occur in 2000.
To examine the status of address list development, local outreach and
promotion, and hiring, we obtained documents that describe the
Bureau's plans, procedures, and progress for each activity and
interviewed Bureau headquarters officials and, where applicable,
regional office officials. We also reviewed the Bureau's analysis of
missed or duplicated housing units in the 1990 census; the Bureau's
analyses of missed or duplicate housing units in its 1995 Census Test
conducted in Paterson, NJ, and Oakland, CA; and its analysis of the
completeness of address lists for seven locations around the nation.
This was done to review the accuracy and completeness of the Bureau's
address list.
To help verify the Bureau's information and also obtain a local
perspective on the dress rehearsal activities, we (1) made site
visits to the Sacramento and South Carolina dress rehearsal
jurisdictions; (2) conducted in-person and telephone interviews with
local officials on their experiences in reviewing address lists,
promoting the census, and recruiting and hiring census workers; and
(3) where applicable, obtained relevant documents on these
activities. Menominee officials were contacted by telephone. To
obtain an indication of the task the advertising contractor faces in
developing an effective promotional campaign, we observed focus group
sessions the contractor scheduled with different population groups as
part of its market research at the dress rehearsal sites.
To examine the Bureau's efforts to implement its sampling and
statistical estimation methodology, we reviewed Bureau documents
related to sampling and estimation, interviewed Bureau officials who
are responsible for drawing the sample for the dress rehearsal, and
reviewed the Bureau's methodology for drawing the sample.
We conducted our audit work at the South Carolina and Sacramento
dress rehearsal sites; the Bureau's Charlotte and Seattle regional
offices; as well as in Washington, D.C., and Dallas, TX, between
September 1997 and February 1998 in accordance with generally
accepted government auditing standards.
We requested comments on a draft of this report from the Secretary of
Commerce or his designee. On March 18, 1998, the Acting Director,
Bureau of the Census, provided us with comments on the draft report.
These comments are presented and evaluated in chapter 6 and reprinted
in appendix I.
THE QUALITY OF THE ADDRESS LIST
HAS SIGNIFICANT IMPLICATIONS FOR
THE 2000 CENSUS
============================================================ Chapter 2
A quality address list--one that is complete and accurate--and
precise maps are the fundamental building blocks of a successful
census. Accurate addresses are essential for delivering
questionnaires, avoiding unnecessary and expensive follow-up efforts
at vacant or nonexistent residences, and establishing a universe of
households for sampling and statistical estimation. Accurate maps
are critical for counting the correct portions of the population in
their correct locations--the cornerstone of congressional
reapportionment and redistricting.
For the 2000 Census, the Bureau estimates that it needs to identify
and locate 118 million housing units and to create a database of
addresses, known as the Master Address File (MAF). To build the MAF,
the Bureau originally planned, in part, to (1) use addresses provided
by the Postal Service, (2) merge these addresses with the address
file the Bureau created during the 1990 Census, (3) conduct limited
checks of the accuracy of selected addresses, and (4) send the
addresses to local governments for verification. However, the
Bureau's experience during the dress rehearsal shows that its
procedures for developing an accurate MAF and maps have weaknesses
that, despite the Bureau's recent improvement efforts, may not
necessarily result in a MAF or maps of sufficient accuracy to ensure
all households receive a census questionnaire. Because of these
procedural weaknesses, the Bureau is "reengineering" how it builds
the MAF by physically verifying all addresses and by encouraging
earlier local government reviews of address lists and maps created
from the Postal Service and the Bureau's 1990 address file. Although
this reengineered approach may address several problems, the Bureau
will not be testing this new approach before the 2000 Census.
THE POSTAL SERVICE'S ADDRESS
FILE MAY NOT BE SUFFICIENTLY
ACCURATE OR COMPLETE TO MEET
THE BUREAU'S NEEDS
---------------------------------------------------------- Chapter 2:1
To reduce cost and improve the accuracy of the 2000 Census, the
Bureau, in developing the MAF for city-style addresses for the
census,\9 originally planned to combine the Postal Service's Delivery
Sequence File (DSF), which includes all mail delivery points
recognized by the Postal Service, with the address file created
during the 1990 Census. The Bureau also planned to conduct a limited
amount of canvassing in each city to ensure that certain areas, such
as those that have experienced growth, had accurate and complete
addresses or those that have multiunit (e.g., apartment) buildings
had consistent counts of apartment units. In areas with mostly
non-city-style mailing addresses, such as rural areas that have rural
delivery or post office mailing addresses, the Bureau plans to create
a MAF as it did for the 1990 Census by having census enumerators
visit all housing units in the designated areas to identify their
physical locations, and to obtain other data such as each occupant's
name, mailing address, and telephone number.
On the basis of its analyses of the DSF, the Bureau has concluded
that the DSF, combined with targeted canvassing in city-style address
areas, cannot be relied upon to achieve its goal of having a MAF for
the 2000 Census that has no more than 1 percent net housing unit
undercoverage and no more than 1 percent duplicate housing coverage.
One of the major problems associated with having an inaccurate
address list is the cost incurred in following up on questionnaires
that are not mailed back. In the 1990 Census, we reported that the
cost to follow up on questionnaires accounted for 57 percent, or $730
million, of the $1.3 billion spent on data collection.\10 A portion
of that amount was caused by inaccuracies in the address list. For
example, according to the Bureau, the cost of unnecessary follow-up
work at housing units that were either vacant or nonexistent was
about $317 million in 1990, a total that included the direct cost of
hiring enumerators and providing training and travel expenses, as
well as the indirect cost of office space and equipment, recruiting,
and administration. To estimate the potential quality of the MAF for
the 2000 Census, the Bureau has analyzed address lists created for a
few selected locations around the nation. Following the 1995 Census
Test in Paterson, NJ, and Oakland, CA, the Bureau found that
field-verifying addresses (e.g., precanvassing), local government
review of the MAF, and a final Postal Service check added 6.4 percent
and 3.6 percent of the total addresses on the MAF for those
locations, respectively. Precanvassing and local government review
of the MAF also deleted 9 percent and 6 percent of the addresses on
the MAF for Paterson and Oakland, respectively. The Bureau also
examined the quality of address files created for the American
Community Survey test sites in seven locations.\11 The Bureau's
estimates of missed housing units ranged from a low of 0.0 percent in
Rockland County, NY, to a high of 2.4 percent in Harris County, TX.
The Bureau's estimates of duplicate or nonexistent housing units
ranged from a low of 0.0 percent in Multnomah County, OR, to a high
of 11.7 percent in Rockland County.
According to the Bureau, incidents of missed housing units occur
because the DSF is not (1) completely up-to-date in terms of new
housing construction; and (2) uniformly current because the DSF
reflects address changes in certain areas faster than in other areas.
Missed housing units can also occur because the Postal Service may
define the delivery points for mail differently than the Bureau
defines separate housing units. This could occur, for example, when
the Postal Service uses one delivery point for all mail delivered to
a multiunit apartment building. Further, erroneous addresses can be
introduced into the MAF when local governments provide the Bureau
with addresses generated from locally developed addressing systems
(e.g., emergency services) that may differ from the addresses
contained in the DSF and used for mail delivery.
According to Bureau officials, in June 1997, the Bureau initiated an
effort to determine whether it could improve the process for building
the MAF for the 2000 Census, and, if so, how. As part of that
effort, the Bureau assembled a team that included tribal, state, and
local government representatives to obtain outside reaction to the
Bureau's original plan to develop its address list by relying
primarily on the DSF and targeted canvassing, and to discuss new
approaches to developing the address list. The team developed a
modified, "reengineered" plan for address list development that
includes, for areas that contain predominantly city-style addresses,
verifying all addresses (100 percent block canvassing) through field
visits, as was done during the 1990 Census. The Bureau estimates
that the reengineered plan will add $108.7 million to its address
list development procedures, which are already budgeted at $357.2
million.
According to the Bureau, the reengineered plan will allow it to
achieve its goal of a MAF that has no more than 1 percent net housing
unit undercoverage and no more than 1 percent duplicate housing
coverage. The Bureau's initial plan for developing the MAF for
city-style addresses included combining the DSF and the 1990 Census
Address Control File (ACF); conducting limited, targeted canvassing;
and local government review. Its decision to reengineer the MAF
building process was based on Bureau officials' judgment, following
their review of existing data, that the initial plan would not result
in the high coverage rate desired. It is not known whether the
reengineered plan will result in the high accuracy rate the Bureau
wants to attain because the Bureau has not tested this plan to
determine the number and nature of missed or erroneously included
housing units that would occur by using this process. For city-style
addresses in the Bureau's 1995 Census Test, the Bureau combined the
DSF, the ACF, 100-percent canvassing, local address review, and a
Postal Service address accuracy check, to compile the MAF. However,
Bureau officials stated that they lacked the resources to determine
the final housing unit undercount and the duplicate housing unit
rates in the 1995 Census Test MAF as was done following the 1990
Census.
--------------------
\9 A city-style address includes a street name and house number
(e.g., 22 Oak Street). A significant characteristic of this type of
address is that it is part of an address system, which means that it
usually appears in numeric sequence along a street and reflects
parity conventions (i.e., odd numbers appear on one side of the
street and even numbers appear on the other side).
\10 Decennial Census: 1990 Results Show Need for Fundamental Reform
(GAO/GGD-92-94, June 9, 1992).
\11 The American Community Survey is a large monthly household survey
that uses mailed questionnaires, telephone interviews, and visits
from Bureau field representatives. It is to be conducted at
additional sites in future years to supplement information obtained
in the 2000 Census.
LOCAL REVIEWS MAY BE TOO
SPORADIC AND CONTAIN TOO MANY
OBSTACLES TO CONSISTENTLY
CONFIRM OR IMPROVE THE MAF'S
ACCURACY
---------------------------------------------------------- Chapter 2:2
Local address review has not progressed smoothly at the dress
rehearsal sites. Many governments did not participate in local
review, while some governments that did participate cited time and
resource constraints and limited assistance from the Bureau as
impediments to their reviews.
Under the Bureau's Local Update of Census Addresses (LUCA) program,
local governments and Indian tribes owning lands are to be given an
opportunity to verify address list information. In 1990, local
governments were given the opportunity to review Bureau counts of
housing units, as well as boundary maps to identify any major
discrepancies. The 1994 Census Address List Improvement Act extended
the Bureau's confidentiality provisions to local governments, and
beginning with the 2000 Census, the Bureau plans to provide local
government officials with detailed address information.\12 The
importance of LUCA in improving the quality of the Bureau's MAF was
demonstrated during the 1995 Census Test. During that test, the
Bureau found that LUCA alone added, deleted, corrected, or moved 7.1
percent of the addresses in Paterson and 1.6 percent of the addresses
in Oakland.
--------------------
\12 P.L. 103-430, Oct. 31, 1994.
TIME AND RESOURCE
CONSTRAINTS LIMITED LOCAL
GOVERNMENT PARTICIPATION
-------------------------------------------------------- Chapter 2:2.1
The dress rehearsal locations include 62 jurisdictions--Sacramento,
Menominee County, and 60 jurisdictions (11 counties and 49 towns and
cities) in the 11-county South Carolina area. Sacramento and
Columbia as well as Menominee County and the state of South Carolina
all were involved with the LUCA program, although participation was
far from universal among the South Carolina jurisdictions. In South
Carolina, the Bureau's Charlotte Regional Office officials (1)
contacted the mayors of the 49 municipalities and the chairs of the
11 county councils, (2) made presentations to county councils and the
City Manager for Columbia about the importance of reviewing the
census address list, and (3) mailed numerous letters to the
jurisdictions. However, despite the Bureau's efforts, it received
responses to the LUCA review, in terms of annotated maps and/or
address changes, for only 34 of the 60 jurisdictions. South
Carolina's Office of Research and Statistics, which maintains
geodetic data for accurate large-scale mapping, provided assistance
to or conducted LUCA for an additional 10 of 60 the jurisdictions for
a total participation rate of 44 jurisdictions. Due to resource
constraints, this office was unable to assist two other
jurisdictions.
According to Bureau regional officials, reasons jurisdictions gave
for not participating in the LUCA review included lack of resources
and/or lack of information to update address lists or maps. Our
discussions with officials from 5 of the 16 South Carolina area
jurisdictions that did not participate in LUCA and that responded to
our inquiries, confirmed that the jurisdictions' primary reason for
not participating was a lack of resources. For example, an official
from one South Carolina town said that the town would have liked to
participate, but that there was no one locally who would agree to
work on or help out with LUCA. Similarly, an official from another
South Carolina town said that it too would have liked to participate.
However, she noted that she would have been tasked to do the job, and
that there was insufficient time to perform her full-time job with
the town as well as review the Bureau's address list and maps.
Our discussions with officials from 12 South Carolina area
jurisdictions that did participate in LUCA and that responded to our
inquiries, indicated that the potential cost of participating in LUCA
can be high. For example, an official in one South Carolina county
estimated the cost to be $5,000, with staff having to perform LUCA as
well as their normally assigned duties in the tax assessor's office.
An official from another city was unable to provide a cost estimate,
but indicated that the city's Geographic Information System
specialist had spent about 4 weeks full-time on the effort, and that
several others had spent part of their time on the effort. An
official in another city said that the city committed 4-1/2 weeks to
LUCA. This official said that, during that time, two people worked
full-time, 9 hours a day, driving down each street in the city.
After that initial canvassing, two people worked full-time and two
other people worked part-time for the remaining 2 weeks correcting
the maps and address lists.
Sacramento officials estimated that city officials spent about 90
hours over the course of 5 weeks, while the officials from the local
area council of governments spent about 450 hours. Even with these
resources, these officials estimated that only about half of the
amount of work required for LUCA was accomplished. This work
included reconciling addresses in blocks where the city and the area
council of governments believed the Bureau missed addresses and
reviewing and correcting maps.
Even with the resources committed to conducting reviews in those
jurisdictions that participated in LUCA, officials in the six
jurisdictions we contacted and that responded to our inquiries said
that the amount of time allowed for LUCA during the dress
rehearsal--about 5 weeks--was insufficient. According to Sacramento
officials, time constraints resulted in the city only being able to
completely review 17, and partially review another 15, of the 48 maps
the Bureau provided the city; 16 maps were not reviewed at all.
Officials from one South Carolina jurisdiction said that because of
insufficient time, local officials did not focus any efforts on
reviewing addresses for possible deletions. Instead, they
concentrated their efforts on adding addresses even though they
acknowledged that these addresses may already have been located
elsewhere in the address files without their knowledge.
LIMITED BUREAU ASSISTANCE
AND POOR MAF AND MAP QUALITY
HAMPERED LOCAL REVIEW
-------------------------------------------------------- Chapter 2:2.2
Those jurisdictions participating in LUCA reported several obstacles
that made their reviews more difficult. For example, officials in
some jurisdictions stated that the Bureau was able to provide little
assistance to them while they were reviewing the address lists and
maps. Furthermore, the accuracy, completeness, and format of the
address lists and maps hindered their review.
According to Bureau officials, although the Bureau's Seattle and
Charlotte Regional Offices each designated a LUCA coordinator, the
Seattle coordinator for Sacramento was responsible for just a single
jurisdiction. In contrast, the Charlotte LUCA coordinator was
responsible for all 60 jurisdictions in the South Carolina dress
rehearsal site. This difference in workload could be responsible for
the different levels of satisfaction between the two sites with the
quality of the Bureau's assistance, and suggests that the Bureau's
workload in 2000 may significantly limit its ability to provide
needed assistance to localities.
For example, according to Sacramento officials, the Bureau's Seattle
Regional Office staff was available at all times during the LUCA
review, and the level and quality of Bureau support was sufficient.
However, a South Carolina county official said that, while early
contact with the Bureau was open and forthcoming, during the LUCA
review Bureau staff provided little to no help. According to this
official, when the county contacted the Bureau about problems it was
experiencing with the Bureau's electronic files and maps, it was told
that the Bureau was aware of these problems but that it could not
provide any guidance or help. Similarly, officials from one of the
South Carolina cities said that the Bureau was of limited help in
providing guidance and assistance during their review of the files
and maps. According to these officials, they were told that because
of the Bureau's limited resources, local governments were relatively
"on their own" when conducting LUCA.
The Bureau's assistance to local governments in 2000 is expected to
be significantly less than that provided to South Carolina during the
dress rehearsal. According to a Charlotte Regional Office official,
while the Bureau had one staff person available to assist the 60
local jurisdictions during the dress rehearsal, this level of support
will greatly diminish for the 2000 Census. According to Bureau
officials, on average, each region will have about 11 partnership
specialists for the 2000 Census, of which about half are to focus
primarily on LUCA. Therefore, the LUCA coordinators in the Charlotte
office would be responsible for about 2,200 communities in that
5-state region, or about 200 to 400 communities each.
The accuracy, completeness, and format of the address lists and maps
were also an issue to participating jurisdictions, and an issue that
could reduce local governments' ability to improve the accuracy of
the MAF and maps in 2000. According to Sacramento officials, the
Bureau's address file for the city contained about 198,000 addresses,
which were more addresses than were contained in Sacramento's
database of addresses. Sacramento officials found that the Bureau's
address file included addresses outside the city limits, addresses
not assigned to any block or tract number, and commercial
addresses.\13 This significantly contributed to problems in reviewing
the Bureau's address file. Sacramento officials also said that the
electronic address file provided by the Bureau did not include
information on where each address was located on the maps, which
created additional work for the city.
Similar concerns were voiced by South Carolina officials. These
officials said that the maps did not include street corrections,
street additions, and block adjustments made to the Bureau maps about
1-1/2 years earlier as part of the Bureau's Block Boundary Suggestion
Program. Furthermore, the maps did not include changes in city
boundaries due to annexations that are provided annually to the
Bureau under the Bureau's Boundary and Annexation Survey. As a
result, according to officials from the city of Sacramento and the
Sacramento Area Council of Governments, they expended considerable
time reviewing the LUCA maps to make corrections that they had
provided earlier to the Bureau. Similar problems were voiced by
South Carolina officials. For example, according to officials in one
South Carolina county, the same errors were on the LUCA maps that the
county has been trying to correct since the 1990 Census.
According to Bureau officials, maps may contain errors for a variety
of reasons, including the Bureau's inability to make corrections on
some Bureau maps when local governments provide corrections and
revisions on their own maps, as opposed to the Bureau's maps. The
Bureau cannot make some of the local government's corrections and
revisions because Bureau maps are not always spatially accurate and
thus Bureau staff may be unable to make the fine corrections
necessary to accurately reflect changes on locally produced,
spatially correct maps.
--------------------
\13 A tract is a neighborhood of about 4,000 people and 1,700 housing
units, designed to have homogeneous population characteristics,
economic status, and living conditions.
THE BUREAU'S REENGINEERING
PLANS ADDRESS SOME, BUT NOT
ALL, CONCERNS CITED BY LOCAL
OFFICIALS ABOUT LUCA
---------------------------------------------------------- Chapter 2:3
According to the Bureau, the reengineered approach to developing its
address list addresses several problems cited by jurisdictions that
participated in LUCA during the dress rehearsal. However, while the
revised approach does address some problems, it does not fully
address them all. Furthermore, the feasibility of the Bureau's plan
remains questionable.
To remedy the problem of lack of sufficient time to perform LUCA, the
Bureau plans to begin the LUCA process earlier and allow more time
for local review. Rather than have LUCA begin in late 1998 and
continue until the second quarter of 1999, the Bureau plans to begin
LUCA in early 1998 and continue it through the second quarter of
1999.
To address the lack of participation in LUCA, the Bureau plans to
establish stronger relationships with state data centers,
metropolitan and regional planning organizations, and councils of
government, especially in areas where local governments are unable to
participate. However, based on the Bureau's dress rehearsal
experience, it is questionable whether such organizations can provide
the assistance envisioned by the Bureau. For example, according to
officials from the South Carolina Office of Research and Statistics,
the organization lacked sufficient resources to provide assistance to
all of the jurisdictions in the SC dress rehearsal site that
requested help. Should assistance be required on a statewide basis,
these officials said that the research and statistics office would
not have sufficient resources to provide it.
The Bureau's reengineered approach does not address the quality of
the address lists or maps cited by officials from some of the dress
rehearsal jurisdictions. Although Bureau canvassing could improve
the quality of the address lists and maps to a certain extent,
because the Bureau's reengineered approach provides for an earlier
LUCA, canvassing will not occur in areas having city-style addresses
until after local governments have completed their review of
addresses and maps. The Bureau's reengineered approach also does not
address the level of Bureau assistance available to local governments
that choose to participate in LUCA, even though it does call for
providing more training. Further, given the Bureau's decision to
reengineer its address list development efforts, the Bureau will not
be able to test its revised approach before implementing it for the
2000 Census.
THE BUREAU'S MAIL RESPONSE RATE
OBJECTIVE COULD BE DIFFICULT TO
ACHIEVE BECAUSE OF OBSTACLES TO
IMPLEMENTING ITS OUTREACH AND
PROMOTION STRATEGY
============================================================ Chapter 3
To increase awareness of the 2000 Census and boost mail response
rates, the Bureau plans an extensive outreach and promotion campaign.
Effective outreach and promotion are essential because they have the
potential to boost the mail response rate, thereby reducing the
Bureau's nonresponse follow-up workload and, thus, the cost of the
census. Through targeting, outreach and promotion can also help
reduce any undercount of traditionally hard-to-enumerate populations.
The Bureau expects that its outreach and promotion efforts, combined
with other initiatives, such as simplified census questionnaires,
should produce a mail response rate of 66.9 percent for the 2000
Census. This is 12 percentage points higher than the 55-percent
response rate that the Bureau expects it would achieve without these
activities and slightly higher than the 65-percent response rate
achieved in the 1990 Census. Nevertheless, the Bureau's experience
thus far during the dress rehearsal suggests that, in 2000, this goal
might be difficult to achieve.
THE BUREAU HAS INITIATED
SEVERAL OUTREACH AND PROMOTION
ACTIVITIES
---------------------------------------------------------- Chapter 3:1
According to the Bureau, the success of its outreach and promotion
efforts will depend heavily on the effectiveness of the partnerships
it hopes to build with state, local, and tribal governments;
nongovernmental organizations; business; and the media. As a matter
of long-standing policy, the Bureau has maintained that it cannot
seek to shoulder the major financial burden of local census outreach
and promotion efforts. Thus, the Bureau is placing a priority on
working with partners because they can, among other things, help
spread the word about the census, encourage active participation,
dispel myths, and issue public statements of endorsement. Among the
major activities we examined for this report were the Bureau's
efforts to form Complete Count Committees (CCC), implement its Census
in Schools Project, and contract-out for a paid-advertising campaign.
One of the primary goals of the CCCs is to heighten public awareness
of the census through community outreach activities. Bureau guidance
recommends that local governments form CCC's consisting of elected
officials, business leaders, minority groups, religious
organizations, community-based organizations, and other individuals
and groups. Such partnerships could provide the Bureau with local
knowledge, experience, and expertise to help it take a more accurate
census and ensure productive use of the Bureau's limited resources.
For example, CCCs could assist the Bureau by alerting Bureau
officials to the existence of hard-to-count population groups within
their communities. These groups could then be targeted for more
intensive and/or specific types of advertising.
Bureau guidance provides potential CCCs with specific projects that
they could undertake to reach individuals in their communities. Such
projects could include working with businesses to (1) obtain their
commitment to display promotional flyers about the census, (2) insert
reminders about the census in customers' billing statements or
employees' paychecks, or (3) set up a census assistance table where
volunteers would assist people in completing their census forms.
Other projects could involve obtaining commitments from local radio
stations or newspapers to provide media coverage and publicity. The
Sacramento and Menominee County CCCs, as well as CCCs in some of the
South Carolina counties, have initiated these and/or similar
activities.
Through its second partnership activity, the Census in Schools
Project, the Bureau hopes to increase response rates by using
students to encourage and, where necessary, help their parents or
guardians complete census forms. The schools project is also
intended to (1) organize millions of teachers, students, and families
to support the census: (2) build a database of interested teachers
who may be recruited as census workers; and (3) improve students'
understanding of the census process and why it is important to the
country. As currently planned, the schools project is to include
grades kindergarten through 12 as well as classes for adult English
as a second language, citizenship, and high school equivalency
degrees. The project also is to focus on five types of schools:
public, private, parochial, Bureau of Indian Affairs, and Department
of Defense. College and university students will be not be included
in the schools project because the Bureau believes that these
students can be better reached through traditional advertising.
A third partnership activity is the Bureau's paid-advertising
campaign. According to the Bureau, the campaign is intended to
increase mail returns from the general public, targeted audiences,
and historically undercounted populations. In the 1990 Census, the
Bureau relied on pro bono public service advertising to get its
message across. However, such advertising had its limitations. For
example, the Bureau has noted that television announcements were
often aired at non-prime-time hours when audiences were relatively
small. For the 2000 Census, the Bureau concluded that a
paid-advertising campaign would allow it to better choose where and
when to place advertisements, more effectively reach hard-to-count
populations, and present a more uniform message.
On October 10, 1997, the Bureau announced that it had awarded its
2000 Census paid-advertising contract to Young & Rubicam (Y&R), which
is a private advertising agency. A Y&R subcontractor, Scholastic,
Inc., is to develop a package of educational materials and a
marketing plan for the Bureau's schools project.
According to Bureau officials, the Bureau has budgeted about $230
million for 2000 Census outreach and promotion activities. Of that
amount, about $100 million is for the Y&R paid-advertising campaign.
Of the $100 million, about 80 percent has been earmarked for buying
advertising in print and the broadcast media. The remaining 20
percent is to be used for such activities as contract administration
and overhead. Of that 20 percent, a portion is to be used for an
independent evaluation of the effectiveness of the paid-advertising
campaign.
Another $100 million of the $230 million outreach and promotion
budget is to be used to hire as many as 320 full-time equivalent
(FTE) partnership specialists, including government, community, and
media specialists.\14 The remaining $30 million is to fund (1) per
diem and travel expenses for members of census advisory committees,
(2) the Census in Schools Project, and (3) certain local promotional
activities, such as printing.
--------------------
\14 An FTE or work year generally includes 260 compensable days or
2,080 hours. According to the Bureau, government specialists will be
responsible for working with local governments to review local
address files; establish CCCs; and obtain space to recruit and train
census workers, among other activities. Community specialists are to
work with local groups to raise awareness of the census, help recruit
census workers, and obtain testing space. Media specialists are to
generate publicity about the census by working with print and
electronic media.
EFFECTIVELY IMPLEMENTING THE
BUREAU'S OUTREACH AND PROMOTION
ACTIVITIES COULD BE PROBLEMATIC
---------------------------------------------------------- Chapter 3:2
Although the Bureau's plan to publicize the 2000 Census was designed
to address the outreach and promotion problems encountered during the
1990 Census, the results to date of the dress rehearsal suggest that
several obstacles may diminish its effectiveness. These obstacles
include inconsistent levels of participation by local communities, a
lack of resources at both the Bureau and local level, and resistance
to the Bureau's message that the census is important. As a result,
the Bureau may have difficulty achieving its goal of a 66.9 percent
mail response rate in 2000.
PARTICIPATION IN OUTREACH
AND PROMOTION ACTIVITIES AT
THE DRESS REHEARSAL SITES
HAS BEEN INCONSISTENT
-------------------------------------------------------- Chapter 3:2.1
Although the Sacramento and Menominee County CCCs appear to be
active, efforts to establish CCCs in South Carolina have had mixed
results. Of the 11 counties and City of Columbia included in the
South Carolina dress rehearsal, just 3 counties and Columbia had
active CCCs at the time of our review. The remaining eight counties
either had not formed CCCs or those that were formed had become
inactive.
Reasons for the nonfunctioning CCCs in South Carolina varied and were
at times beyond the Bureau's control. For example, according to a
Bureau official, one county's CCC disbanded after a key member left.
In at least two instances, however, it appears that the Bureau was at
least partly responsible for the operational problems that caused the
CCCs to suspend their activities. For example, one local official
said that CCC members became frustrated and dissatisfied with the
lack of organization and leadership from representatives of the
Bureau's regional office. An official from another CCC said that
vague communications between the Bureau and the CCC played a role in
the CCC's cessation of activities. The official said the Bureau did
not set clear expectations for what the CCC was supposed to do, and
that the Bureau had supplied only minimal guidance and support.
Communication difficulties appeared to be a problem with several
active South Carolina CCCs as well. Four of the six active CCCs we
contacted at the three dress rehearsal sites indicated that the
Bureau did not set clear expectations for their CCCs, especially when
they were first initiated, and/or Bureau guidance and literature has
been minimal.
LOCAL AND BUREAU RESOURCES
AFFECTED RESULTS TO DATE OF
OUTREACH AND PROMOTION
EFFORTS AT THE DRESS
REHEARSAL SITES
-------------------------------------------------------- Chapter 3:2.2
Local and Bureau resources may not be adequate for effective outreach
and promotion. While the Bureau generally plans to provide
communities with in-kind services (e.g., copies of flyers), local
communities are to provide critical funding during the 2000 Census.
In practice, however, this arrangement has led to some operational
problems with the CCCs at the dress rehearsal locations. For
example, officials from five South Carolina counties and the City of
Columbia indicated that resources to carry out promotional activities
were limited. In Sacramento, a city official said that much of what
the city's CCC proposed to do in the way of media advertising will
depend on the funding available from the Bureau's advertising
contractor and any in-kind services provided by the Bureau.
Moreover, although local officials said that some local businesses
have agreed to undertake such activities as displaying promotional
flyers, as a rule, local businesses have not provided and likely will
not provide funding for promoting the dress rehearsal. For example,
in South Carolina, officials from five counties said that funding and
in-kind services from local businesses would probably not be
available. The Bureau's partnership specialist for the South
Carolina site acknowledged that local business support has been
sporadic and that business partnerships the Bureau had hoped for have
not yet occurred. In Sacramento, a local official noted that while
businesses would likely be enthusiastic about certain in-kind
services, such as displaying posters and including flyers in
mailings, they probably would not provide any funding. At the
Menominee County dress rehearsal site, a tribal official noted that
the reservation contained few businesses, virtually all of the
businesses were small, and none had the resources to contribute to
promoting the census.
The costs of supporting CCC activities typically were absorbed by
local governments. An official from one South Carolina county
estimated that the county's CCC has spent 100 staff hours on CCC
initiatives, while an official on the City of Columbia's CCC said its
CCC members have spent about 240 staff hours on outreach and
promotion activities as of early December 1997 and will probably
spend an equal amount before the dress rehearsal is finished. An
official from another South Carolina county estimated that its CCC
has spent $2,000 in cash and in-kind services for outreach and
promotion efforts and that it will spend about $5,500 more before the
dress rehearsal is completed. An official from another South
Carolina CCC told us that outreach and promotion activities have cost
about $8,000 to date, and the official anticipated that the CCC would
spend an additional $27,000 before the dress rehearsal is over.
As previously noted, the Bureau ultimately plans to fill as many as
320 FTE partnership specialist positions for the decennial census in
its 12 regional offices. Given the approximately 3,100 counties and
county equivalents that comprise the nation, each full-time
partnership specialist's caseload would average 10 counties. This is
only slightly less than the workload of the Bureau's South Carolina
partnership specialist, who, as previously described, was responsible
for mobilizing outreach and promotion efforts in 11 counties and the
City of Columbia and achieved only mixed results. (The South
Carolina partnership specialist's workload did not go unnoticed;
several local officials from five of the South Carolina dress
rehearsal sites observed that the partnership specialist might have
been stretched too thin.) Furthermore, in addition to outreach and
promotion efforts during the 2000 Census, the partnership specialists
will be responsible for additional activities, such as responding to
media inquiries, obtaining space for recruiting and training census
workers, and assisting with LUCA reviews.
Local resources for promoting the census was an issue during the 1995
Census Test. As we noted in a previous report, the 1995 Census Test
indicated that cooperative ventures with local governments provided a
way to promote public participation in the census. However, as we
reported, local governments in urban areas where response rates were
lowest reported that their lack of funding to promote the census was
problematic.\15
The Bureau is also implementing the Census in Schools Project at the
dress rehearsal sites and, as with the CCCs, is encountering
inconsistent participation. In South Carolina, the Bureau's
partnership specialist told us that six counties were participating
in the program at the time of our study. As was the case with the
South Carolina CCCs, county officials involved with the schools
project cited problems such as unclear expectations from the Bureau
and/or lack of resources as the reasons for their difficulties.
--------------------
\15 GAO/GGD-97-142.
THE BUREAU'S
PAID-ADVERTISING CAMPAIGN
FACES CHALLENGES
-------------------------------------------------------- Chapter 3:2.3
The Bureau's national advertising contractor, Y&R, began its efforts
for the dress rehearsal in mid-December 1997 by convening a series of
focus group interviews with different segments of the population at
the dress rehearsal sites. According to Bureau officials, the
purpose of these focus groups was to build on earlier, more extensive
research that was conducted by the Bureau on personal reactions to
the census, and to test proposed advertising themes and approaches
that could be used to reach specific, targeted audiences during the
2000 Census.
Currently, the advertising campaign is based on the theme "This is
your future--don't leave it [blank]" and stresses how responding to
the census questionnaire benefits one's community. This same general
theme was also used for the 1990 Census. However, Bureau officials
are hoping that the paid-advertising strategy will allow more
effective ad placement as well as more precise targeting of specific
messages to particular groups compared to the pro bono promotional
services that were used in previous censuses.
The focus group results give an indication of the task Y&R faces in
developing an effective campaign. While some participants in the
sessions we observed were generally aware of the census and its
purpose, others were unfamiliar with the census. Furthermore, while
some participants reacted positively to the test advertisements they
were shown, others were more cynical and remained unconvinced that
they should respond to the census. Cultural and other demographic
differences will likely reinforce the difficulties confronting the
Bureau. Moreover, as demonstrated by the results of the 1990 Census,
simply raising public awareness is insufficient. The public must
also be motivated to return their questionnaires in the face of a
long-term decline in the mail response rate. The Bureau's research
found that, although about 93 percent of the public was aware of the
1990 Census, the mail response rate was only 65 percent. This was 10
percentage points lower than the mail response rate in the 1980
Census.
THE BUREAU'S ABILITY TO STAFF THE
2000 CENSUS IS UNCLEAR
============================================================ Chapter 4
Recruiting, hiring, training, and retaining a workforce sufficient to
accomplish the decennial census is undoubtedly one of the
government's greatest human resource challenges. Although the Bureau
has a number of initiatives aimed at making it easier to recruit and
retain census workers, a number of uncertainties surround the
Bureau's efforts and their likely success in 2000.
THE BUREAU HAS DEVELOPED
SEVERAL INITIATIVES TO ADDRESS
POTENTIAL STAFFING DIFFICULTIES
BUT COULD STILL ENCOUNTER
PROBLEMS IN 2000
---------------------------------------------------------- Chapter 4:1
The Bureau estimates that, under its current design, it will need to
fill about 295,000 office and field positions to carry out various
activities for the 2000 Census. Key positions include census takers
(enumerators), crew leaders, field operations supervisors
(supervisory enumerators), and clerical office workers. To fill this
many positions, the Bureau estimates it will need to recruit some
2,632,600 applicants because, for a variety of reasons, most
applicants never make it through the employment process. Some
eventually take other jobs, others do not report for testing or
training, while still others fail background checks and employment
tests. Overall, the Bureau says that for every 10 applicants, only 1
is ultimately hired.
Aside from the sheer number of applicants needed, staffing the census
is difficult because most census jobs are temporary and do not offer
such benefits as health insurance, life insurance, sick or annual
leave, retirement plans, or child care. However, census workers are
eligible for unemployment and worker compensation. Also, the
majority of the office and field positions must be filled during the
short window when nonresponse follow-up operations are conducted. As
currently planned, for the 2000 Census, this window will occur during
a 6-week period beginning the end of April 2000 and lasting through
early June. During that time, the Bureau has set a staffing goal of
121,000 people.
As we reported in our previous work, during the 1990 Census, the
Bureau encountered a number of staffing problems that adversely
affected both the cost and quality of the census. For example,
almost one-half of the Bureau's district offices were unable to meet
the 1989 recruiting goal of four applicants for every field position
for precanvassing operations, during which census workers verified
the completeness and accuracy of address lists. As a result, 14
percent of the 109 district offices finished the operation at least a
week behind schedule.\16 Furthermore, three of the Bureau's regional
offices reported that over 70 percent of their district offices were
understaffed as of early May 1990--a peak period for following up on
nonresponding households.\17
--------------------
\16 1990 Census: Enhanced Oversight Should Strengthen Recruitment
Program (GAO/GGD-90-65, Apr. 13, 1990).
\17 Progress of the 1990 Decennial Census: Some Causes for Concern
(GAO/T-GGD-90-44, May 21, 1990).
THE BUREAU PLANS SEVERAL
INITIATIVES TO ENSURE IT
WILL HAVE AN ADEQUATE
WORKFORCE, BUT STAFFING THE
2000 CENSUS COULD STILL BE
DIFFICULT
-------------------------------------------------------- Chapter 4:1.1
To address the potential problem of recruiting, training, and
retaining almost 300,000 staff to fill office and field positions,
the Bureau plans to use several recruitment and compensation
strategies. For example, during the 1990 Census, the Bureau focused
its recruitment efforts on unemployed individuals; however, during
the 2000 Census, the Bureau plans to target its hiring toward
individuals seeking "part-time, part-time" employment (i.e.,
individuals who are already employed but who are seeking additional
work (moonlighters)). Retirees wishing to supplement their income
are also a focus of the Bureau's hiring efforts.
Bureau officials said that moonlighters and retirees are more likely
to find the temporary, intermittent weekend and evening work of an
enumerator attractive. The officials also said such individuals may
be more likely than unemployed individuals to have existing health
insurance and other benefits as well as the needed job skills to do
the work. Furthermore, they might be in a better position to work
evenings and weekends when more people are likely to be home, thus
making the Bureau's nonresponse follow-up efforts more efficient.
To further reduce recruitment and retention problems such as those
experienced in the 1990 Census, the Bureau plans to implement a
wage-setting program, key features of which include locality-based
pay and bonuses for exceeding production. The Bureau's compensation
program is based on recommendations contained in a June 1997 interim
report from Westat, which is a private consulting firm. The Bureau
asked Westat to analyze factors (pay, working conditions, demographic
and economic trends) that appeared to have contributed to the success
or failure of hiring that took place in the 1990 Census. The Bureau
plans to set the wages it pays census workers at 60 to 80 percent of
the prevailing wage rate for the local area as reported by the Bureau
of Labor Statistics. According to Bureau projections, the resulting
pay rates would range from $8 to $15 per hour.
The Bureau also plans to pay Census employees bonuses for exceeding
production and quality standards and for completing assigned work for
selected operations. For example, enumerators who obtain completed
census questionnaires at the rate of 1.5 cases per hour during
nonresponse follow-up operations are to receive a bonus of about
$2.00 per case.
To expand the potential applicant pool, the Bureau has taken steps to
make it easier to hire federal employees and individuals on public
assistance without affecting their pay and benefits. For example,
the Bureau has the authority to waive pay/retirement reduction
requirements for federal military and civilian retirees who work on
the 2000 Census. The Bureau is also seeking assistance from the
Office of Personnel Management to remove certain administrative
barriers to make it easier to hire current federal employees to work
on the census. For individuals on public assistance, for example,
the Bureau requested the Department of Housing and Urban Development
to exclude the amounts earned by census enumerators when calculating
annual income for eligibility in public and assisted housing
programs. According to the Bureau, the Department approved the
request in May 1996.
Despite these initiatives, the Bureau may find that its efforts to
adequately staff the 2000 Census could be problematic. On top of the
hiring difficulties that the Bureau has encountered during past
censuses, if current national trends continue, the Bureau could be
facing an extremely tight labor market. In 1990, the overall
unemployment rate was 5.6 percent, compared to 4.9 percent in
1997--the lowest since 1973.\18
Furthermore, if the Bureau is prevented from using statistical
sampling and estimation procedures, recruiting and retaining a
sufficient workforce for the 2000 Census could be even more
difficult. According to the Bureau, if all nonresponding households
are followed up by enumerators, an additional 59,000 enumerators
would be needed on top of the estimated 300,000 staff. Additional
effort would also be required to verify whether households were
vacant. If the Bureau is allowed to sample, the Bureau plans to
verify 10-percent of the housing units identified as vacant by the
Postal Service. According to the Bureau, verifying a 10-percent
sample of vacant housing units would require 5,000 enumerators.
However, in the absence of sampling for vacant housing units, the
Bureau estimates that 25,000 to 30,000 enumerators would be required
for this operation. These enumerators are in addition to those
required for nonresponse follow-up.
--------------------
\18 Historical unemployment rates and the estimated unemployment rate
for 1997 are published by the Bureau of Labor Statistics and
contained in monthly reports entitled Current Population Survey Data,
Civilian Labor Force, Age 16 and Older.
THE BUREAU'S RECRUITMENT
STRATEGY WAS BASED ON
LIMITED DATA
-------------------------------------------------------- Chapter 4:1.2
We are also concerned that the Bureau's decision to focus its
recruitment efforts on moonlighters and retirees was based on limited
data. Bureau officials told us that the rationale for its decision
was based, in part, on the Bureau's experience during the 1995 Census
Test, when informal briefings were held with enumerators. The
officials noted that the Bureau did not do a formal evaluation of the
composition of the workforce for the 1995 Census Test.
The Bureau noted that its recruitment strategy decision was also
based on research conducted by Westat. However, Westat's study
acknowledges the limitations of the data, which were partly based on
Westat's own hiring practices and those of similar firms. Westat
noted that most of its recruiting, training, and surveying is
conducted in the evening and on weekends using employees that have
other jobs or family responsibilities during regular business hours.
However, we believe that Westat's implicit assumptions that its
workforce (1) consists of similar kinds of employees who would be
interested in census work and (2) is comparable nationwide, in the
absence of any other data, may or may not be valid. For example, as
noted by a Westat official, unlike census enumerators who are
employed in all parts of the country to collect information by going
door-to-door, the majority of Westat's interviewers work in the
Washington, D.C., area, and conduct interviews by telephone.
Further, although Westat conducted a detailed analysis of the
nonresponse follow-up workforce used during the 1990 Census in
Pennsylvania, it lacked certain data relevant to its recommended
recruitment strategy. For example, the resulting report recommended
further analysis to determine such key relationships as how well
prior employment and earnings predict recruitment and retention and
how productivity varies as a function of employment patterns.
THE DRESS REHEARSAL WILL NOT
VALIDATE THE BUREAU'S
STAFFING INITIATIVES
-------------------------------------------------------- Chapter 4:1.3
According to Bureau officials, staffing the dress rehearsal
activities that have occurred to date has gone well. The Bureau was
able to hire enough people to adequately staff its operations.
Nevertheless, the demand for temporary census workers thus far has
been small, and more generally, the Bureau's experience gained during
the dress rehearsal does not and cannot provide an adequate test of
the effectiveness of the Bureau's staffing strategy for the 2000
Census.
According to officials in the Bureau's Seattle Regional Office, the
Bureau has been able to adequately staff its operations for the
Sacramento dress rehearsal. They noted that the quality of the
people hired, to date, has been good, and has included people with
master's degrees and retired corporate executives. Similarly,
officials in the Bureau's Charlotte Regional Office stated that they
have had no difficulty attracting and hiring the necessary staff.
Nevertheless, only a relatively small amount of hiring has taken
place. As of January 1998, Bureau workforce data indicate that, for
calendar year 1997, a total of only 488 individuals were hired at the
three dress rehearsal sites. Of the 488 individuals hired, 352, or
about 72 percent, worked at various locations at the Columbia site,
primarily conducting address listing operations. Sacramento dress
rehearsal operations accounted for 127 of the 488 hires. These
individuals were involved with address development operations. Very
limited hiring was done in calendar year 1997 to staff dress
rehearsal operations at the Menominee County site, where nine
individuals were hired to complete address listing activities at this
location. Many more individuals will be needed after Dress Rehearsal
Census Day for nonresponse follow-up operations and statistical
sampling procedures in Sacramento.
However, the dress rehearsal is not a good test of the Bureau's
staffing strategy for the 2000 Census. First, the labor market at
the dress rehearsal sites is not necessarily representative of what
the Bureau will encounter nationally in the 2000 Census. For
example, according to Bureau officials, recruiting at the Sacramento
site was made somewhat easier by the closure of a nearby air force
base, which added more people to the potential applicant pool.
Moreover, as the demand for census workers increases during the dress
rehearsal for such operations as nonresponse follow-up, the Bureau
can compensate for any local hiring difficulties by shifting
employees to where they are needed most, or even recruiting outside
of the dress rehearsal jurisdictions. In the 2000 Census, with each
community being called upon to supply the necessary resources to
carry out census activities, the Bureau will be less able to use such
measures.
THE BUREAU'S PLANS FOR SAMPLING
AND STATISTICAL ESTIMATION FACE A
NUMBER OF CHALLENGES
============================================================ Chapter 5
Declining response rates have posed problems for the Bureau since it
began its mail-out/mail-back procedure in 1970. Hundreds of
thousands of additional enumerators must be hired to collect census
information from an increasing number of nonresponding households.
To reduce its nonresponse follow-up workload, the Bureau plans to
sample nonresponding households for the 2000 Census. The Bureau has
also designed a procedure called Integrated Coverage Measurement
(ICM), which involves taking a separate sample after the nonresponse
follow-up is completed to make adjustments to the census counts.
However, these activities face several methodological, technological,
and quality control challenges. The 1998 dress rehearsal may provide
experience useful to addressing these challenges, but others may
surface as time goes on.
THE BUREAU PLANS TO USE
STATISTICAL METHODS TO REDUCE
ITS NONRESPONSE FOLLOW-UP
WORKLOAD
---------------------------------------------------------- Chapter 5:1
In 1990, the Bureau sent Census questionnaires to most households in
the United States, asking that household members complete the
questionnaire and mail it back to the Census Bureau. The Bureau
hired enumerators to visit and collect data from the 35 percent of
households that did not return a questionnaire. This follow-up
effort proved costly. About $560.3 million of the total $2.6 billion
allocated for the 1990 Census (about 50 percent of the data
collection budget) was spent on nonresponse follow-up. Furthermore,
the effort ran substantially behind schedule because a number of
housing units were particularly difficult to contact and required
repeated attempts to enumerate.
Because of the difficulty in enumerating these housing units, on
occasion, the Bureau had to rely on "last resort" and close-out data
collected from neighbors or other knowledgeable sources for those
housing units that could not be personally contacted.\19 Such data
has been shown in the past to be less accurate than data collected
directly from housing unit members because external sources cannot be
expected to have complete information about a housing unit or to know
if its members were counted elsewhere. For example, in our 1992
report, we found that these procedures may have contributed to the
decline in the quality of 1990 Census data because the Bureau
accepted less complete responses and information from nonhousehold
respondents.\20 Bureau data from the 1990 Census show that last
resort and close-out procedures were used to gather information on
6.6 percent of the persons enumerated during nonresponse follow-up.
We reported that these procedures were used most heavily in urban
areas and among minority populations. Last resort and close-out data
accounted for an average of 20 percent of the households in
large-city census offices and 4.9 percent of the non-Hispanic Black
enumerations (compared to 2.3 percent of the non-Hispanic White
enumerations).
For the 2000 Census, the Bureau anticipates that the mail response
rate will be 66.9 percent, which is essentially the same as the 65
percent mail response rate achieved in the 1990 Census.\21 Rather
than following up with visits to every nonresponding housing unit,
the Bureau has proposed a strategy to enumerate remaining housing
units on the basis of a sample of nonresponding housing units in each
census tract. After the end of the mail response phase, the Bureau
will select a sample that would result in a response rate of at least
90 percent of the housing units in each tract. Sample data will then
be used to impute the characteristics of the remaining 10 percent of
the housing units. To ensure equitable treatment of census tracts
with high response rates, all tracts with an initial response rate of
at least 85 percent in the mail-back phase will be sampled at a rate
of 1 in 3, which will result in more than 90 percent of the housing
units in these tracts being enumerated.\22 The Bureau believes its
plan will reduce costs and the time necessary to conduct follow-up of
nonresponding housing units.
For the dress rehearsal in Sacramento, 2000 Census procedures will be
used for nonresponse follow-up. A systematic sample, sorted by
address, will be selected in each census tract once the mail-back
phase of enumeration is completed. If a census form is mailed back
after the nonresponse follow-up sample has been selected, the data
from the mailed form will be used for that household, and forms sent
in by households will continue to be accepted until the end of
nonresponse follow-up data collection.
--------------------
\19 Last resort data are collected either from someone outside of the
household, such as a postal carrier, neighbor, landlord, or relative,
or from a household member who does not wish to provide complete
information. Closeout data are collected when last resort
information is unavailable and includes minimal information, such as
whether a unit is occupied or nonexistent.
\20 GAO/GGD-92-94.
\21 The Census Bureau projects that mailout of a single Census
form--the procedure used beginning in 1970--would result in a
mailback rate of 55 percent in 2000.
\22 As discussed in the following section of this report, enumerators
will attempt to contact all housing units in blocks selected for the
Integrated Coverage Measurement Survey. Sample-based nonresponse
follow-up will not be utilized in these blocks.
THE BUREAU PLANS TO ADJUST FOR
ENUMERATION ERRORS
---------------------------------------------------------- Chapter 5:2
Since the 1940 Census, the Bureau has conducted a variety of studies
to measure inaccuracies in census population counts. Methodologies
have varied over the years--Post-Enumeration Surveys (PES);
demographic analyses of official records such as birth and death
certificates; and matches to the Current Population Survey.
Regardless of the method used, a net undercount has been measured for
each census since 1940. During the 1990 Census, the Bureau conducted
a PES of 167,046 housing units to measure inaccuracies in the
population count. However, then-Secretary of Commerce Robert
Mosbacher ruled that evidence for adjusting census data, on the basis
of the 1990 PES design, was inconclusive and unconvincing.
Consequently, the Bureau used PES results only to evaluate the
quality of census data, not to revise its final 1990 census numbers.
To measure inaccuracies in the population count for the 2000 Census,
the Bureau has proposed the ICM, which corrects several problems
identified in the 1990 PES and which relies on a much larger sample.
The Bureau expects to use the ICM results to produce a "one-number
census."
Based on the PES, the net national undercount (the number of people
omitted minus the number overcounted) in 1990 is estimated to have
been 4 million people, or 1.6 percent of the population. The net
undercount reflects 10 million people who were not counted or who
were omitted from the census in their proper place of residence
(i.e., omissions) and 6 million people who were either enumerated
more than once, were ineligible to be counted, or were counted at a
locations other than their correct place of residence (i.e.,
erroneous enumerations). While the net undercount in 1990 was 4
million people, the differential undercount of minorities and other
hard-to-enumerate groups and areas was more problematic. For
example, in the 1990 census, the rate of undercount was over four
times higher for African Americans than for non-African Americans.
Evaluations of the 1990 Census have indicated that attempts to add
additional counting activities did not overcome this problem.
The ICM is designed to improve the accuracy of the population counts
for a variety of segments of the population and would be the last
phase in census data collection, following receipt of mail-out
questionnaires and participation in nonresponse follow-up. The
Bureau is to draw a sample of 750,000 housing units throughout the
United States for the ICM, based on an address list created
independently from that prepared for the census enumeration.
Enumerators are then to be sent to these housing units to interview
residents and obtain the characteristics of individuals living there.
In blocks selected for the ICM, sample-based, nonresponse follow-up
will not be used. Both the census enumeration and the ICM will
attempt to collect information from all housing units in blocks
selected for ICM. The Bureau will then compare data collected by the
ICM and by the Census enumeration in ICM blocks to determine the
degree to which individuals enumerated by one data collection were
also detected by the other. Information from this process is to be
used to estimate the extent to which people were correctly
enumerated, missed, or counted in error for the entire census. The
Bureau would estimate the correct population for entire geographic
areas, as well as for specific subpopulations as defined by
characteristics such as age, sex, tenure, race, and ethnic origin.
SAMPLING AND STATISTICAL
ESTIMATION DESIGNS FACE
METHODOLOGICAL, TECHNOLOGICAL,
AND QUALITY CONTROL CHALLENGES
---------------------------------------------------------- Chapter 5:3
In our July 1997 report, we noted that sampling for nonresponse
follow-up could reduce costs and save time, while the ICM could
improve the accuracy of the population totals.\23 Nevertheless, the
Bureau faces a number of operational challenges in implementing its
sample-based, data collection activities.
First, as currently designed, the schedules for completing certain
phases of the census--including mail response, nonresponse follow-up,
and ICM--are compressed when compared to the 1990 Census. In 1990,
the Bureau allowed 4 weeks from Census Day for mail response before
beginning nonresponse follow-up. In 2000, the Bureau also plans to
allow 4 weeks from Census Day for mail response. In 1990,
nonresponse follow-up was scheduled to last 6 weeks but in some
locations lasted 14 weeks.\24 For the 2000 Census, the Bureau will
again allow 6 weeks for completion of nonresponse follow-up. In
1990, PES data collection and reconciliation were not completed until
January 4, 1991, while for the 2000 Census, the Bureau plans to
perform the same tasks for 5 times the number of households by the
end of September 2000. Furthermore, ICM activities from data
collection to completion of ICM follow-up interviewing and matching
are scheduled to last 5 months. The PES performed after the 1990
census used a much smaller sample of 167,046 housing units, yet it
required more than 6 months to complete.
Second, given these time constraints, the Bureau may have to limit
repeated call-backs to hard-to-enumerate housing units and rely more
heavily on other sources for last-resort or closeout census data to
complete the nonresponse follow-up phase on time. In the 1990
Census, the Bureau's reliance on last-resort data varied by location,
with large city census offices enumerating an average of 20 percent
of all nonresponse follow-up households by last-resort or closeout
procedures. The quality of census enumeration data can suffer when
the Bureau uses last-resort or closeout data, because other sources,
such as landlords, neighbors, or relatives, may not have complete
information on the occupants of a housing unit.
Third, the quality of the final census numbers, as adjusted by the
ICM, will be highly dependent on accurate and precise ICM measurement
of those persons who were properly enumerated and those who were
omitted. The Bureau has set a target of a 98 percent completion rate
for ICM interviews--which was the rate it was able to achieve for PES
during the 1990 Census--to provide the data quality it expects for
adjusting the population count.
As noted earlier, completion schedules for ICM activities are
compressed when compared to 1990 PES time frames. A potential
consequence of failure to complete ICM on schedule is that the
completion rate for ICM data may fall below the expected 98 percent,
which could adversely affect the quality of ICM data and therefore
the final adjusted census counts.
More specifically, a lower ICM completion rate may result in some
segments of the population being underrepresented in ICM data.
Comparisons of housing units interviewed during the 1990 PES indicate
that it was more difficult to complete interviews in housing units
composed of singles, minorities, and renters. Thus, if the ICM
completion rate is below the 98 percent target, there is increased
risk that these hard-to-count groups will be underrepresented in ICM
data, and that ICM-based adjustments may not reflect different
segments of the population accurately. Reaching the targeted
completion rate for hard-to-enumerate populations may require the
Bureau to rely on a larger proportion of last resort and close-out
data, which, as noted earlier, are typically of lower quality.
Fourth, difficulties in properly identifying and matching ICM
information back to data submitted by ICM households during the
enumeration phase of the census may affect the usefulness of ICM data
files and estimates. In December 1997, the U.S. Department of
Commerce Inspector General reported that software designed to match
an ICM sample housing unit to the same census enumeration housing
unit will only search within the block to which that housing unit has
been assigned.\25 A discrepancy of as little as one digit in the
housing unit address between ICM and the census enumeration could
result in the housing unit being located in different blocks by ICM
and census enumeration, thus precluding a match. To improve the
quality of its software development, the Bureau says it is
implementing the Inspector General's recommendations, taking such
steps as using peer review and checklists to ensure the proper
implementation of sampling and estimation methodology in software.
Housing units selected for ICM whose residents move between Census
Day and the beginning of ICM data collection may create additional
problems. The failure to locate and interview the same individuals
who lived at ICM addresses at the time of the enumeration will lower
the ICM completion rate. Tracking individuals who move is important
because such individuals may differ in important characteristics,
such as family size or marital status, from less mobile individuals.
As a fallback, data may be collected from successor households in the
housing unit or from other knowledgeable sources outside of the
housing unit, but these sources cannot be expected to always have
full or accurate information on members of the original household.
Fifth, the Bureau's implementation of the ICM for the Dress Rehearsal
has suffered from a number of quality control problems. During the
course of our work, we observed the Bureau's difficulties in drawing
a useable sample of housing units to conduct the ICM for the dress
rehearsal. The Bureau initially drew an ICM sample from a list that
included commercial addresses that should have been excluded.
Inclusion of these commercial addresses rendered the initial sample
unusable. The Bureau drew a second ICM sample from a list that
excluded these ineligible commercial addresses. However, the
Bureau's specifications for selecting blocks with different racial,
ethnic, and renter/owner characteristics for the sample contained a
number of inconsistencies. As a result, there were several errors in
the number of housing units selected for the different categories of
blocks. Because these errors were not detected until relatively late
in the sample selection process, we are concerned about the Bureau's
ability to detect and correct problems in a timely manner.
--------------------
\23 GAO/GGD-97-142.
\24 In 1990, Bureau enumerators were able to complete about 90
percent of their follow-up workload within 8 weeks, with the final 10
percent requiring more than 6 weeks to complete.
\25 2000 Decennial Census: Key Milestones and Associated Risks,
report by the U.S. Department of Commerce Inspector General to Sen.
John McCain, Chairman, Committee on Commerce, Science, and
Transportation, U.S. Senate, Dec. 30, 1997.
CONCLUSIONS
============================================================ Chapter 6
We believe that, although the Bureau has made progress in addressing
some of the problems that occurred during the 1990 Census, key
activities continue to face developmental and/or implementation
challenges. We recognize that difficulties are to be expected with
such a massive and complex undertaking as the census. Nevertheless,
with the Decennial Census Day just 2 years away, we find it troubling
that the Bureau finds itself facing several ongoing and newly
emerging operational challenges. In addition to these operational
issues, the continued lack of an agreement between Congress and the
administration over the final design of the 2000 Census contributes
to the uncertainty surrounding the Bureau's ability to conduct an
accurate and cost-effective census. The risk of a failed census in
2000 will increase as long as the impasse over its final design
persists.
The dress rehearsal is the Bureau's final opportunity for a
large-scale, operational test of its overall design. If properly
planned, executed, and evaluated, the dress rehearsal could provide
the Bureau with valuable information on the feasibility of its
current approach and on where corrective action might be needed.
With little time remaining until the 2000 Census, a window of
opportunity will open only briefly to allow the Bureau to assess the
results of the dress rehearsal and take the necessary steps to
conduct an accurate, cost-effective census in 2000.
AGENCY COMMENTS AND OUR
EVALUATION
---------------------------------------------------------- Chapter 6:1
The Acting Director of the Bureau of the Census provided written
comments on a draft of this report. These comments, along with our
response, are summarized below and printed in full in appendix I.
The Bureau agreed that the key census-taking activities we reviewed
were facing operational challenges, but expressed confidence that
these activities are well under control. The Bureau is concerned
that delays and uncertainties caused by the lack of agreement on the
Bureau's planned use of statistical methods have increased the risks
to the census. We believe that the successful implementation of key
census operations is a prerequisite for a high quality census,
regardless of whether or not the Bureau uses statistical methods.
The Bureau also commented on our characterization of specific
operational issues. The Bureau generally agreed with the facts as
presented with regard to the original design for the address list
development methodology and the reengineered design. It also
concurred with our assessment that achieving a 99 percent inclusion
rate for its address list is one of the greatest challenges for the
2000 Census. However, the Bureau disagreed with our statement that
the reengineered design has not been tested. It noted that the
methodology is proven and has been used in previous censuses and
tests. We agree that the components of the reengineered
methodology--canvassing census blocks, working with the Postal
Service, and LUCA--have been used and tested, but the Bureau has not
used or tested them in concert with each other, nor in the sequence
presently designed for use in the 2000 Census. For example, in 1990,
block canvassing was done before local government became involved.
Further, local governments were not provided address lists, but
instead just numbers of addresses in particular census blocks.
Moreover, data from the 1990 Census suggests that address list
development methodologies may not necessarily be uniform across all
jurisdictions or sufficient to meet the Bureau's goal for the
accuracy of its address list.
The Bureau also noted that, as part of its reengineering effort, it
has taken steps to improve LUCA, which it agreed did not progress
smoothly during the dress rehearsal. These steps include providing
additional time for local review, simplifying the process, and
suggesting alternative strategies requiring less intensive efforts
for jurisdictions having limited resources. However, as noted in the
report, we believe that the initiatives will address some, but not
all, of the concerns cited by local officials. For example, we
remain concerned that the quality of the address lists and maps
provided to local governments may limit the effectiveness of local
review.
The Bureau agreed with the importance of outreach and promotion
activities, but disagreed with our characterization that "results to
date of the dress rehearsal [with local outreach] suggest . . .
the Bureau may have difficulty achieving its goal of 66.9 percent
mail response rate in 2000." The Bureau stated that the 66.9 percent
response rate is derived from extensive testing of its multiple
mailing strategy, which includes such innovations as use of first
class postage and a reminder card.
Achieving a 66.9 percent response rate is the Bureau's best-case
scenario for the 2000 Census. It depends on many traditional
census-taking methods and on new initiatives developed since the 1990
Census, including the multiple mail strategy, an outreach program
that includes $100 million in paid advertising, and simplified
questionnaire forms. The Bureau's statement that the 66.9 percent
response rate depends primarily on the multiple mailing strategy is
inconsistent with its past statements and its approach to building
high levels of public cooperation with the census. Thus, we have
retained our language that achieving the 66.9 percent mail response
goal is dependent upon a successful mailing strategy and an effective
promotion and outreach program. All the census-taking methods must
work successfully in concert if the Bureau is to achieve its desired
response rate.
With regard to the Bureau's ability to staff the 2000 Census, the
Bureau agreed with our assessment concerning the magnitude of the
challenge facing the Bureau. However, it disagreed with our
characterization that "the Bureau's recruitment strategy was based on
informal and limited data." Given the nature of today's society, the
Bureau's decision to focus on evening and weekend hours for
conducting nonresponse follow-up interviews appears reasonable. And,
as a result of information subsequently provided by the Bureau that
shows the additional research on which the Bureau based its
recruitment strategy, we agree that the data were not "informal" and
have revised the report accordingly. However, our conclusion on the
limitations of the data were based on Westat's own acknowledgment.
For example, in one presentation to the Bureau, Westat suggested that
"about 847 enumerators will be needed per district office out of a
pool of 20,300 workers available in any given month." Westat stated
that this was a "back-of-an-envelope" estimate. In discussing the
pool of workers that may be available to be enumerators, Westat
stated: "At this time, our best evidence for the view that the
success of the [nonresponse follow-up] hinges on tapping into the
pool of `moonlighters' comes from Westat's own hiring practices and
those of similar firms."
The Bureau agreed its sampling and statistical estimation plans pose
certain challenges and added that a nonsampling census plan would
present many of the same challenges as well as many others. The
Bureau agreed with our concerns about the tight scheduling of its
nonresponse follow-up and ICM operations as well as about quality
control issues. The Bureau observed that it will implement a number
of initiatives, including its multiple mail strategy, that are
designed to improve census data quality. The Bureau also noted that
evaluations of the 1990 census, including ours, found that data
quality diminishes the farther the collection effort gets from Census
Day. Finally, the Bureau said that ICM is the only proven method for
reducing both undercounts and overcounts, and the only option for
conducting the census within current scheduling constraints.
The Bureau stated that it takes seriously the challenges with regard
to quality control, pointing out that it is developing extensive risk
mitigation strategies for key areas of the census. It stated that it
is improving the quality of software development and using peer
review and checklists to ensure the proper implementation of sampling
and estimation methodology. The Bureau will be sharing the results
of its efforts in the near future. It remains to be seen whether
these efforts reduce the risk to the census.
In summary, the Bureau recognized that it faces substantial
challenges in conducting an effective census in 2000, but remains
confident that it will be successful. We remain concerned, however,
that important questions remain unanswered at a disturbingly late
stage in the census cycle when the emphasis should be on fine tuning
rather than basic revisions to operational plans.
(See figure in printed edition.)Appendix I
COMMENTS FROM THE BUREAU OF THE
CENSUS
============================================================ Chapter 6
(See figure in printed edition.)
(See figure in printed edition.)
MAJOR CONTRIBUTORS TO THIS REPORT
========================================================== Appendix II
GENERAL GOVERNMENT DIVISION,
WASHINGTON, D.C.
-------------------------------------------------------- Appendix II:1
James H. Burow, Assistant Director, Federal Management and Workforce
Issues
Robert Goldenkoff, Assistant Director
Don Allison, Senior Evaluator
Victoria Miller, Senior Evaluator
Bruce Taylor, Senior Statistician
DALLAS OFFICE
-------------------------------------------------------- Appendix II:2
Marcia McWreath, Senior Evaluator
Reid Jones, Senior Evaluator
James Rose, Evaluator
*** End of document. ***