2010 Census: Counting Americans Overseas as Part of the Decennial
Census Would Not Be Cost-Effective (19-AUG-04, GAO-04-898).
The U.S. Census Bureau (Bureau) has typically counted overseas
members of the military, federal civilian employees, and their
dependents. However, it usually excluded private citizens
residing abroad. In July 2004, the Bureau completed a test of the
practicality of counting all overseas Americans. GAO was asked to
assess (1) whether the Bureau implemented the test consistent
with its design, and (2) the lessons learned from the test
results.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-04-898
ACCNO: A11698
TITLE: 2010 Census: Counting Americans Overseas as Part of the
Decennial Census Would Not Be Cost-Effective
DATE: 08/19/2004
SUBJECT: Americans abroad
Census
Cost effectiveness analysis
Data collection
Public relations
Statistical methods
Surveys
Testing
Comparative analysis
2010 Decennial Census
France
Kuwait
Mexico
2000 Decennial Census
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-04-898
United States Government Accountability Office
GAO Report to the Subcommittee on Technology, Information Policy,
Intergovernmental Relations and the Census, Committee on Government Reform,
House of Representatives
August 2004
2010 CENSUS
Counting Americans Overseas as Part of the Decennial Census Would Not Be
Cost-Effective
a
GAO-04-898
Highlights of GAO-04-898, a report to the Subcommittee on Technology,
Information Policy, Intergovernmental Relations and the Census, Committee
on Government Reform, House of Representatives
The U.S. Census Bureau (Bureau) has typically counted overseas members of
the military, federal civilian employees, and their dependents. However,
it usually excluded private citizens residing abroad. In July 2004, the
Bureau completed a test of the practicality of counting all overseas
Americans. GAO was asked to assess (1) whether the Bureau implemented the
test consistent with its design, and (2) the lessons learned from the test
results.
Congress may wish to consider eliminating funding for additional research
related to counting Americans abroad as part of the decennial census,
including funding for tests planned in 2006 and 2008. However, funding for
the evaluation of the 2004 test should continue as planned, particularly
to inform congressional decision making on this issue. Should Congress
desire better data on overseas Americans for certain policymaking and
other nonapportionment purposes, Congress may wish to consider funding
research on the feasibility of counting this group using alternatives to
the decennial census. To facilitate this, we recommend that the Bureau, in
consultation with Congress, research options such as a separate survey,
administrative records, and data exchanges with other countries'
statistical agencies. The Bureau agreed with our conclusions and
recommendations.
www.gao.gov/cgi-bin/getrpt?GAO-04-898.
To view the full product, including the scope and methodology, click on
the link above. For more information, contact Patricia A. Dalton at (202)
512-6806 or [email protected].
August 2004
2010 CENSUS
Counting Americans Overseas as Part of the Decennial Census Would Not Be
Cost-Effective
The Bureau generally implemented the overseas census test on schedule and
consistent with its research design. Still, participation was poor, with
just 5,390 questionnaires returned from the three test sites-France,
Kuwait, and Mexico. Moreover, because of the low response levels,
obtaining those questionnaires proved to be quite expensive-around $1,450
per response, which is far costlier on a unit basis than the 2000 Census.
Although the two are not directly comparable because the 2000 Census
included operations not used in the overseas test, the 2000 Census cost
around $56 per household. Further, boosting the response rate globally
might not be practical. On the domestic front, during the 2000 Census, the
Bureau spent $374 million on a months-long publicity campaign that
consisted of television and other advertising that helped yield a
72-percent return rate. Replicating this level of effort on a worldwide
basis would be difficult, and still would not produce a complete count.
Ensuring a smooth overseas count could also stretch the Bureau's
resources. For example, at each test site the Bureau encountered various
challenges that needed to be resolved such as French privacy laws.
Moreover, managing a complex operation from thousands of miles away also
proved difficult.
Enumerating Americans in Mexico and France
The approach used to count the overseas population in the 2004 test-a
voluntary survey that largely relies on marketing to secure a complete
count, lacks the basic building blocks of a successful census. The Bureau
has done some initial research on alternatives, but all require more
extensive review. Given that the Bureau already faces the difficult task
of securing a successful stateside count in 2010, having to simultaneously
count Americans abroad would only add to the challenges facing the Bureau.
Contents
Letter 1
Results in Brief 2
Background 4
Scope and Methodology 6
2004 Overseas Test Was Generally Implemented as 7
Designed
Overseas Census Test Results Were Disappointing 9
and Costly
Conclusions 20
Matters for Congressional Consideration 21
Recommendations for Executive Action 21
Agency Comments and Our Evaluation 22
Appendix
Appendix I: Comments from the Department of Commerce
Table 1: Key Components of Overseas Enumeration
Tables Were
Generally Implemented as Planned 8
Table 2: Comparison of Responses Received for 2004
Overseas
Census Test 9
Figures Figure 1: Key Decisions Facing Congress on
Enumerating Americans Abroad 6
Figure 2: Census Bureau Ads Placed in the
International Herald
Tribune 10
Figure 3: Census Materials Were Prominently
Displayed in Various
Locations in France and Mexico 14
Figure 4: Key Building Blocks of a Successful 16
Census
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
A
United States Government Accountability Office Washington, D.C. 20548
August 19, 2004
The Honorable Adam H. Putnam
Chairman
The Honorable Wm. Lacy Clay
Ranking Minority Member
Subcommittee on Technology, Information Policy, Intergovernmental
Relations and the Census
Committee on Government Reform
House of Representatives
Although more than four million American citizens are believed to reside
abroad, the precise number of overseas Americans is unknown. The U.S.
Census Bureau (Bureau), the federal agency tasked with counting the
nation's population every 10 years, has generally included in the census
overseas members of the military, federal civilian employees, and their
dependents (a group known collectively as "federally affiliated"
individuals), but has typically excluded private citizens such as
retirees, students, and business people.1
Under the Constitution and federal statutes, the Bureau has discretion
over whether to count Americans abroad. However, in recent years, the
Bureau's policy of excluding private citizens from the decennial census
has been called into question. For example, advocates of an overseas
census claim that better demographic data on this population group would
be useful for a variety of policy-making and business purposes, and would
better represent their unique interests in Congress. Moreover, in January
2001, Utah sued the Bureau's parent agency, the U.S. Department of
Commerce, claiming that it lost a congressional seat because the 2000
Census excluded the state's 11,000 Mormon missionaries and other private
citizens living abroad.2 According to the Congressional Research Service,
Utah would have gained a congressional seat had an additional 855 people
been
1 Only the 1970, 1990, and 2000 Censuses used counts of federally
affiliated personnel for purposes of apportioning Congress.
2 Utah v. Evans, 143 F. Supp. 2d 1290 (D. Utah 2001), aff'd, Utah v.
Evans, 534 U.S. 1038 (2001).
included in the state's apportionment totals.3 Utah's suit, however, was
unsuccessful.
As we noted in our May 2004 report on this issue, counting Americans
abroad as an integral part of the 2010 Census would be a monumental task
that would introduce new resource demands, risks, and uncertainties to an
endeavor that was already facing a variety of difficulties.4 Specific
challenges include policy questions such as who should be counted and how
should the data be used, as well as logistical difficulties such as
ensuring a complete count and verifying U.S. citizenship.
To assess the practicality of counting overseas Americans, the Bureau held
a test enumeration from February through July 2, 2004, in France, Kuwait,
and Mexico. As agreed with your offices, we assessed (1) whether the
Bureau implemented the test consistent with its design, and (2) the
initial lessons learned from the test results and their implications for
future overseas enumerations.
To address these objectives, we reviewed applicable planning and other
documents, and interviewed knowledgeable Bureau officials and
representatives of private organizations who helped the Bureau promote the
census at the three test sites. Further, to review how the Bureau was
implementing the census at the test locations, we made on-site inspections
in Paris, France, and Guadalajara, Mexico. We conducted our work from
March through July 2004, in accordance with generally accepted government
auditing standards.
Results in Brief The Bureau generally implemented the 2004 overseas census
test consistent with its research design. Key elements of the Bureau's
plan, such as developing a questionnaire specifically for overseas
Americans and launching a marketing campaign designed to publicize the
test, were generally carried out as planned. Moreover, the test was
conducted on schedule.
3 Congressional Research Service, House Apportionment: Could Census
Corrections Shift a House Seat?, RS21638 (Washington, D.C.: Oct. 8, 2003).
4 GAO, 2010 Census: Overseas Enumeration Test Raises Need for Clear Policy
Direction, GAO-04-470 (Washington, D.C.: May 21, 2004).
However, the response levels fell far short of what the Bureau planned for
relative to the number of questionnaires it printed. For example, although
the Bureau printed about 520,000 census forms for the three test sites-
France, Kuwait, and Mexico-the actual number of paper responses it
received only totaled 1,783, as of the end of the test in early July 2004.
Another 3,607 responses were received via the Internet. Further, because
of the low response levels, the data were expensive to obtain on a unit
cost basis-around $1,450 per return. In contrast, the unit cost of the
2000 Census was about $56 per household. Although the 2000 Census costs
are not directly comparable to the 2004 overseas test because the 2000
test included operations not used in the overseas test, the 2000 Census
was the most expensive census in our nation's history.
Further, substantially boosting overseas response levels might be
infeasible. For example, during the 2000 Census, the Bureau spent $374
million on a months-long publicity campaign that consisted of television,
radio, and other forms of advertising that helped secure a 72-percent
return rate. Replicating this level of effort on a worldwide basis would
be impractical at best, and would not produce a complete count. Indeed,
even after the Bureau's aggressive publicity effort in 2000, it still
needed to follow-up with about 42 million households that did not complete
their census forms.
Ensuring a smooth overseas count could also stretch the Bureau's
resources, and thus detract from domestic efforts. For example, at each
test site the Bureau encountered various difficulties that needed to be
worked out. The difficulties included addressing French privacy laws and
delivery problems in Kuwait. Moreover, managing a complex operation from
thousands of miles away was also hard. This was particularly evident in
the logistical challenges the Bureau had in overseeing the performance of
the private firm hired to publicize the census at the three test sites.
The Bureau's longstanding experience in counting the nation's population
has made it clear that a cost-effective census is assembled from key
building blocks that include mandatory participation, a complete and
accurate address list, and the ability to follow-up with nonrespondents.
The approach the Bureau used to count the overseas population-a voluntary
survey that relies heavily on marketing to secure a complete count-largely
for reasons of practicality, lacks these building blocks, and it is
unlikely that any refinements to this basic design would produce
substantially better results. What's more, the Bureau already faces the
difficult task of securing a successful stateside count in 2010. Having to
count Americans abroad would only add to the challenges facing the Bureau.
Given the obstacles to a cost-effective count of overseas Americans as
part of the decennial census, Congress may wish to consider eliminating
funding for the research, planning, and development activities related to
counting this population group using the Bureau's current approach,
including funding for tests planned in 2006 and 2008. However, funding for
the evaluation of the 2004 test should continue as planned because it
could provide useful data to Congress. Moreover, should Congress determine
that a count of overseas Americans might be useful for certain
policymaking and other nonapportionment purposes, Congress may wish to
consider authorizing and funding research on the feasibility of counting
Americans abroad using alternatives to the decennial census.
To facilitate congressional decision making, we recommend that the Bureau,
in consultation with Congress, research potential alternatives to the
decennial census such as conducting a separate survey, examining how
administrative records could be refined to produce a more accurate count
of overseas Americans, and exchanging data with other countries'
statistical agencies and censuses, subject to applicable confidentiality
and other provisions. Once Congress knows the tradeoffs of these various
options, it would be better positioned to provide the Bureau with the
direction it needs so that the Bureau could then develop and test an
approach that meets congressional requirements at reasonable resource
levels.
The Secretary of Commerce forwarded written comments from the U.S. Census
Bureau on a draft of this report. The comments are reprinted in the
appendix.
The Bureau agreed with our conclusions and recommendations. Furthermore,
the Bureau noted, "should Congress request and fund" further research on
counting overseas Americans, it would be equipped to do that research
itself.
Background According to the Bureau, no accurate estimate exists of the
total number of Americans living abroad. The Constitution and federal law
give the Bureau discretion to decide whether to count American citizens
living abroad. In prior censuses, the Bureau has generally included
"federally affiliated" groups-members of the military and federal
employees and their
dependents-but has excluded private citizens residing abroad from all but
the 1960 and 1970 Censuses. The 2000 Census, using administrative records,
found 576,367 federally affiliated Americans residing overseas, including
226,363 military personnel, 30,576 civilian employees, and 319,428
dependents of both groups.
In response to congressional direction and the concerns of various private
organizations, the Census Bureau launched a research and evaluation
program to assess the practicality of counting both private and federally
affiliated U.S. citizens residing abroad. The key part of this effort, the
enumeration, took place from February 2004 through July 2, 2004, in
France, Kuwait, and Mexico. To promote the overseas census test the Bureau
relied on third parties-American organizations and businesses in the three
countries-to communicate to their members and/or customers that an
overseas enumeration of Americans was taking place and to make available
to U.S. citizens either the paper questionnaire or Web site address.
Currently, the Bureau is processing and analyzing the overseas
questionnaire data and plans to complete an evaluation of the test results
in early 2005. The Bureau estimates that it will have spent approximately
$7.8 million in fiscal years 2003 through 2005 to plan, conduct, and
evaluate the 2004 test. The Bureau has requested additional funds for
fiscal year 2005 to plan for further testing scheduled for 2006. The
Bureau also plans to include overseas testing in the 2008 dress rehearsal
if it were to receive the necessary funding.5
In May 2004 we reported on the design of the overseas enumeration test and
concluded that because of various methodological limitations, the test
results will only partially answer the Bureau's key research objectives
concerning feasibility (as measured by such indicators as participation
and number of valid returns), data quality, and cost. Further, we noted
that the Bureau should not decide on its own whether or not to enumerate
Americans overseas, and in fact would need congressional guidance on how
to proceed. As shown in figure 1, the key decisions facing Congress in
this regard include, in addition to the threshold question of whether
American residing overseas should be counted, how the data should be
5 At this point, the Bureau does not have cost data beyond fiscal year
2005. Therefore, it is unknown what the costs will be for implementing and
evaluating the 2006 overseas test or the 2008 overseas dress rehearsal.
used and whether to enumerate this population group as part of the
decennial census.
Figure 1: Key Decisions Facing Congress on Enumerating Americans Abroad
Source: GAO analysis.
We also recommended that if further testing were to occur, that the Bureau
resolve the shortcomings of the design of the 2004 test and better address
the objectives of an overseas enumeration.
Scope and As agreed with your offices, our objectives for this report were
to assess (1) whether the Bureau implemented the test consistent with its
design,
Methodology and (2) the initial lessons learned from the test results and
their implications for future overseas enumerations. To assess the first
objective, we interviewed Bureau officials and compared the Bureau's test
plans with what was actually done at the three test sites. We visited
Paris, France, and Guadalajara, Mexico, to obtain the views of 12 private,
civic, and other
organizations on the implementation of the overseas census test, and/or to
confirm at 36 organizations the availability of census material. In
addition, to a more limited extent, we interviewed officials from third
party organizations in Kuwait via the telephone or e-mail. We judgmentally
selected these organizations because they had agreed to display census
promotional materials and, in some cases, had also agreed to do one or
more of the following activities: make available paper copies of the
census questionnaire, publish information in a newsletter, post a link to
a Web site, send outreach e-mail to members, and/or create speaking
opportunities to discuss the census. The results of these visits are not
necessarily representative of the larger universe of third-party
organizations.
To assess the implications of the test results on future overseas
enumerations and the 2010 census, we obtained from Bureau officials
preliminary results of the overseas census by test site and response mode
as well as cost data. We also interviewed officials from the Bureau and
third-party organizations to determine what lessons were learned from the
test and the implications on future overseas enumeration efforts.
2004 Overseas Test The Bureau's design for the 2004 overseas enumeration
test was generally
implemented as planned and completed on schedule. The Bureau's designWas
Generally had four key components: the mode of response, the questionnaire
Implemented as designed specifically for Americans living overseas, three
test sites, and an Designed outreach and promotion program designed to
communicate and educate
Americans abroad that a test census was being conducted. Table 1
describes each of these components in greater detail.
Table 1: Key Components of Overseas Enumeration Were Generally Implemented
as Planned
Generally Key overseas test implemented component What the Bureau planned
as planned? GAO observations
Census form response Census questionnaire will be available Yes Paper
version of the census form was available at
mode in paper format or via the Internet.�various locations at the
test sites. Questionnaire was available for completion on the Internet.
Paper version was also available in Spanish in Mexico.
Census test 2000 decennial short Yes Asked respondents to provide their
questions form passport
questionnaire will be numbers and social security numbers
modified for the to verify
overseas enumeration. citizenship.
Asked respondents about their
employment status-
military, federal, or other.
Asked for information on everyone
in the household
even if the person was not a U.S.
citizen.
Test sites Conduct test in three Yes Overseas test was implemented
geographically in three countries:
diverse areas with large France, Kuwait, and Mexico.
American
populations, and where
administrative
records may be available to
help verify
results.
Outreach/ Contract with a public Yes Public relations firm hired to
relations firm to develop a
Promotion develop a communications communication strategy.
strategy to
inform and motivate Strategy relied on public and
respondents living private organizations in
in the selected countries to each of the test sites.
answer the
census. Organizations displayed
promotional materials about
the test, communicated test
census to members, and
distributed the paper census
forms to American
residents.
Source: GAO.
However, while the test was generally implemented as designed, our earlier
report pointed out several methodological limitations with the design,
such as not being able to calculate response rates because the universe of
Americans is unknown or not being able to measure the quality of data
because of the impracticality of developing an address list. As we discuss
later in this report, it is these methodological limitations that impede
the Bureau's ability to implement a successful overseas enumeration.
Overseas Census Test Results Were Disappointing and Costly
Although the 2004 overseas enumeration test ended in early July 2004 and
the Bureau has just begun evaluating the results, the response levels were
poor, and very expensive to obtain on a per unit basis. The response level
to the overseas enumeration suggests that the current approach to counting
overseas Americans-a voluntary survey that relies heavily on marketing to
get people to participate-by itself cannot secure a successful head count.
Further, obtaining the additional resources needed to produce
substantially better results may not be feasible, and still not yield data
that are comparable in quality to the stateside enumeration.
Response Levels Were Disappointing and Costly to Obtain
The 5,390 responses the Bureau received for this test were far below what
the Bureau planned for when printing up materials and census forms. While
the Bureau ordered 520,000 paper forms for three test sites, only 1,783
census forms were completed and returned. Of these, 35 were Spanish
language forms that were made available in Mexico. The remaining 3,607
responses were completed via the Internet. Table 2 below shows the number
of census questionnaires that the Bureau printed for each country and the
number of responses they actually received in both the paper format and
via the Internet.
Table 2: Comparison of Responses Received for 2004 Overseas Census Test
Number of responses by mode
Test sites
Number of forms printed for each test site Paper Internet
Total number of responses received
Mexico 430,000a 869b 1,130 1,999
France 75,000 886 2,219 3,105
Kuwait 15,000 28 258
Total 520,000 1,783 3,607 5,390
Source: U.S. Census Bureau.
aThis includes 100,000 forms printed in Spanish. bThis includes 35 Spanish
forms returned.
Because of the low response levels, in early April 2004, the Bureau
reversed its decision to not use paid advertising and in May 2004
initiated a paid advertising campaign in France and Mexico. This included
print and
Internet ads in France and print and radio ads in Mexico. See figure 2 for
examples of the ads used in the paid advertising campaign.
Figure 2: Census Bureau Ads Placed in the International Herald Tribune
Source: GAO.
A Bureau official told us the ad campaign for the 2004 overseas test cost
about $206,000. This official said there were surplus funds available in
the project budget to use for this purpose due to lower than expected
processing and postage costs for the overseas test. While the Bureau saw
some increase in the number of responses after the paid advertising
campaign began, this official said the increase was slight.
Return on Investment for Overseas Enumeration is Low
Not only were response levels low, they were extremely expensive to obtain
on a unit basis-roughly $1,450 for each returned questionnaire, based on
the $7.8 million the Bureau spent preparing for, implementing, and
evaluating the 2004 overseas test. In contrast, the unit cost of the 2000
Census was about $56 per household. Although the two surveys are not
directly comparable because the 2000 Census costs covered operations not
used in the overseas test, the 2000 Census was still the most expensive
census in our nation's history.
Securing a Higher Return Rate Would Be an Enormous Challenge
The main reason for the high unit cost is the low return rate. However,
significantly boosting participation levels may not be feasible. The
Bureau's experience in the 2000 Census highlights the level of effort that
was needed to raise public awareness about the census and get people to
complete their forms. For the 2000 decennial, the Bureau spent $374
million on a comprehensive marketing, communications, and partnership
effort. The campaign consisted of a five-part strategy conducted in three
waves beginning in the fall of 1999 and continuing past Census Day (April
1, 2000). The effort helped secure a 72-percent return rate. Specific
elements included television, radio, and other mass media advertising;
promotions and special events; and a census-in-schools program. Thus, over
a period of several months, the American public was on the receiving end
of a steady drumbeat of advertising aimed at publicizing the census and
motivating them to respond.
The Bureau also filled 594 full-time partnership specialist positions.
These individuals were responsible for mobilizing support for the census
on a grassroots basis by working with governmental entities, private
companies, and religious and social service groups, and other
organizations.
Replicating this level of effort on a worldwide basis would be
impractical, and still would not produce a complete count. Indeed, even
after the Bureau's aggressive marketing effort in 2000, it still had to
follow-up with about 42 million households that did not return their
census forms. Moreover, because there are no reliable figures on the
number of Americans residing overseas, the Bureau would not have a good
measure of the number of people that did not participate, or the overall
quality of the data.
Ensuring a Smooth Overseas Enumeration Could Stretch the Bureau's Resources
Country-specific Issues Created Implementation Problems
The Bureau's experience in conducting the 2004 overseas test underscored
the difficulties of administering a complex operation from thousands of
miles away. Not surprisingly, as with any operation this complex, various
challenges and unforeseen problems arose. While the Bureau was able to
resolve them, its ability to do so should there be a full overseas
enumeration as part of the 2010 Census, would be highly questionable as
far more resources would be required. This was particularly evident in at
least two areas: grappling with country-specific issues and overseeing the
contractor responsible for raising public awareness of the census at the
three test sites.
The Bureau encountered a variety of implementation problems at each of the
test sites. In some cases the problems were known in advance, in others,
glitches developed at the last minute. Although such difficulties are to
be expected given the magnitude of the Bureau's task, a key lesson learned
from the test is that there would be no economy of scale in ramping up to
a full enumeration of Americans abroad. In fact, just the opposite would
be true. Because of the inevitability of country-specific problems, rather
than conducting a single overseas count based on a standard set of rules
and procedures (as is the case with the stateside census), the Bureau
might end up administering what amounts to dozens of separate censuses-one
for each of the countries it enumerates-each with its own set of
procedures adapted to each country's unique requirements. The time and
resources required to do this would likely be overwhelming and detract
from the Bureau's stateside efforts.
For example, during the overseas test, the Bureau found that French
privacy laws restrict the collection of personal data such as race and
ethnic information. However, these data are collected as part of the
decennial census because they are key to implementing a number of civil
rights laws such as the Voting Rights Act.
Addressing France's privacy laws took a considerable amount of negotiation
between the two countries, and was ultimately resolved after a formal
agreement was developed. The Bureau issued and posted on its Web site an
advisory that informed Americans living in France that it was not
mandatory to respond to the questionnaire, the only recipient of the
collected data is the Census Bureau, the data will be kept for one year,
and the respondent has a right to access and correct the data collected.
The Bureau was able to collect race and ethnic data-generally a prohibited
practice without the respondents' permission-only after it received
special approval from a French government agency. Initially, however, it
looked as if the Bureau might have to redesign the census form if it
wanted to use it in France.
In Kuwait, delivery of the census materials was delayed by several weeks
at the beginning of the test because they were accidentally addressed to
the wrong contractor. Ultimately, the U.S. Embassy stepped-in to accept
the boxes so that the enumeration could proceed. In Mexico, there was some
initial confusion on the part of Mexican postal workers as to whether they
could accept the postage-paid envelopes that the Bureau had provided to
return the paper questionnaires for processing in the United States.
Because of the small number of countries involved in the test, the Bureau
was able to address the various problems it encountered. Still, the
Bureau's experience indicates that it will be exceedingly difficult to
identify and resolve in advance all the various laws, rules, societal
factors, and a host of other potential glitches that could affect a full
overseas enumeration.
On-site Supervision of As noted previously, the Bureau hired a public
relations firm to develop a
Contractor Was Problematic communications strategy to inform and motivate
respondents living in the test countries to complete the census. The
firm's responsibilities included identifying private companies, religious
institutions, service organizations, and other entities that have contact
with Americans abroad and could thus help publicize the census test.
Specific activities the organizations could perform included displaying
promotional materials and paper versions of the census questionnaire,
publishing information in a newsletter, and posting information on their
Web sites. Although the public relations firm appeared to go to great
lengths to enlist the participation of these various entities-soliciting
the support of hundreds of organizations in the three countries-the test
revealed the difficulties of adequately overseeing a contractor operating
in multiple sites overseas.
For example, the public relations firm's tracking system indicated that
around 440 entities had agreed to perform one or more types of promotional
activities. However, our on-site inspections of several of these
organizations in Paris, France, and Guadalajara, Mexico, that had agreed
to display the census materials and/or distribute the questionnaires,
uncovered several glitches. Of the 36 organizations we visited that were
supposed to be displaying promotional literature, we found the information
was only available at 15. In those cases, as shown in figure 3, the
materials were generally displayed in prominent locations, typically on a
table with posters on a nearby wall.
Figure 3: Census Materials Were Prominently Displayed in Various Locations
in France and Mexico
Source: GAO.
Five of these 15 organizations were also distributing the census
questionnaire, but the forms were not readily accessible.
However, at 21 sites we visited, we found various discrepancies between
what the public relations firm indicated had occurred, and what actually
took place. For example, while the firm's tracking system indicated that
questionnaires would be available at a restaurant and an English-language
bookstore in Guadalajara, none were available. In fact, the owner of the
bookstore told us that no one from the Census Bureau or the public
relations firm had contacted her about displaying materials for the
overseas test.
At the University of Guadalajara, although the tracking system indicated
that an official had been contacted about, and agreed to help support the
census test, that official told us no one had contacted him. As a result,
when boxes of census materials were delivered to his school without any
explanatory information, he did not know what to do with them, and had to
telephone the U.S. Consulate in Guadalajara to figure out what they were
for.
Likewise, in Paris, we went to several locations where the tracking system
indicated that census information would be available. None was. In fact,
at some of these sites, not only was there no information about the
census, but there was no indication that the organization we were looking
for was at the address we had from the database.
The results of the overseas test point to the difficulties of overseeing
the contractor's performance. As census materials were made available at
scores of locations across the three test countries, it would have been
impractical for the Bureau to inspect each site. The difficulty of
supervising contractors-and any field operation for that matter-would only
be magnified in a global enumeration.
The Design of the Overseas The Bureau's experience in counting the
nation's population for the 2000 Census Lacks the Building and earlier
censuses sheds light on some of the specific operations and Blocks of a
Successful other elements that together form the building blocks of a
successful head Census count (see fig. 4).
Figure 4: Key Building Blocks of a Successful Census
Source: GAO.
While performing these activities does not necessarily guarantee a
costeffective headcount, not performing them makes a quality count far
less promising and puts the entire enterprise at risk. The current
approach to counting overseas Americans lacks these building blocks, as
most are infeasible to perform on an overseas population. Each is
discussed in greater detail below.
o Mandatory participation: Under federal law, all persons residing in the
United States regardless of citizenship status are required to respond to
the decennial census. By contrast, the overseas enumeration test was
conducted as a voluntary survey where participation was optional. The
Bureau has found that response rates to mandatory surveys are higher than
the response rates to voluntary surveys. This in turn yields more complete
data and helps hold down costs.
o Early agreement on design: Both Congress and the Bureau need to agree
on the fundamental design of an overseas census. Concurrence on the design
helps ensure adequate planning, testing and funding levels. Conversely,
the lack of an agreed-upon design raises the risk that basic design
elements might change in the years ahead, while the opportunities to test
those changes and integrate them with other operations will diminish.
Under the Bureau's current plans, after the 2006 test, the Bureau would
have just one more opportunity to test its prototype for an overseas
enumeration-a dress rehearsal in 2008. Any design changes after 2008 would
not be tested in a real-world environment.
The design of the census is driven in large part by the purposes for which
the data will be used. Currently, no decisions have been made on whether
the overseas data will be used for purposes of congressional
apportionment, redistricting, allocating federal funds, or other
applications. Some applications, such as apportionment, would require
precise population counts and a very rigorous design that parallels the
stateside count. Other applications, however, could get by with less
precision and thus, a less stringent approach.
As we noted previously, Congress will need to decide whether or not to
count overseas Americans, and how the results should be used. The basis
for these determinations needs to be sound research on the cost, quality
of data, and logistical feasibility of the range of options for counting
this population group. Possibilities include counting Americans via a
separate survey, administrative records such as passport and voter
registration forms, and/or records maintained by other countries such as
published census records and work permits.
The Bureau's initial research has shown that each of these options has
coverage, accuracy, and accessibility issues, and some might introduce
systemic biases into the data. Far more extensive research would be needed
to determine the feasibility of these or other potential approaches. Once
Congress knows the tradeoffs of these various alternatives, it will be
better positioned to provide the Bureau with the guidance it needs to go
beyond research and conduct field tests of specific approaches. The Bureau
can conduct the research, or it can contract it out. Indeed, the National
Research Council of the National Academy of Sciences has conducted a
number of studies on the decennial census, including a review of the 2000
Census and an examination of reengineering the 2010 Census.
o A complete and accurate address list: The cornerstone of a successful
census is a quality address list. For the stateside census, the Bureau
goes to great lengths to develop what is essentially an inventory of all
known living quarters in the United States, including sending census
workers to canvass every street in the nation to verify addresses. The
Bureau uses this information to deliver questionnaires, follow up with
nonrespondents, determine vacancies, and identify households the Bureau
may have missed or counted more than once. Because it would be impractical
to develop an accurate parallel address list for overseas Americans, these
operations would be impossible and the quality of the data would suffer as
a result.
o Ability to detect invalid returns: Ensuring the integrity of the census
data requires the Bureau to have a mechanism to screen out invalid
responses. Stateside, the Bureau does this by associating an
identification number on the questionnaire to a specific address in the
Bureau's address list, as well as by field verification. However, the
Bureau's current approach to counting overseas Americans is unable to
determine whether or not a respondent does in fact reside abroad. So long
as a respondent provides certain pieces of information on the census
questionnaire, it will be eligible for further processing. The Bureau is
unable to confirm the point of origin for questionnaires completed on the
Internet, and postmarks on a paper questionnaire only tell the location
from which a form was mailed, not the place of residence of the
respondent. The Bureau has acknowledged that ensuring such validity might
be all but impossible for any reasonable level of effort and funding.
o Ability to follow up with non-respondents: Because participation in the
decennial census is mandatory, the Bureau sends enumerators to those
households that do not return their questionnaires. In cases where
household members cannot be contacted or refuse to answer all or part of a
census questionnaire, enumerators are to obtain data from neighbors, a
building manager, or other nonhousehold member presumed to know about its
residents. The Bureau also employs statistical techniques to impute data
when it lacks complete information on a household. Thus, by the end of
each decennial census, the Bureau has a fairly exhaustive count of
everyone in the nation. As noted above, because the Bureau lacks an
address list of overseas Americans, it is unable to follow up with
nonrespondents or impute information on missing households. As a result,
the Bureau will never be able to obtain a complete count of overseas
Americans.
o Cost model for estimating needed resources: The Bureau uses a cost
model and other baseline data to help it estimate the resources it needs
to conduct the stateside census. Key assumptions such as response levels
and workload are developed based on the Bureau's experience in counting
people decade after decade. However, the Bureau has only a handful of data
points with which to gauge the resources necessary for an overseas census,
and the tests it plans on conducting will only be of limited value in
modeling the costs of conducting a worldwide enumeration in 2010.
The lack of baseline data could cause the Bureau to over-or underestimate
the staffing, budget and other requirements of an overseas count. For
example, this was evident during the 2004 overseas test when the Bureau
estimated it would need around 100,000 Spanishlanguage questionnaires for
the Mexican test site. As only 35 Spanishlanguage questionnaires were
returned, it is now clear that the Bureau could have gotten by with
printing far fewer questionnaires for Mexico. However, the dilemma for the
Bureau is that its experience in the 2004 overseas test cannot be used to
project the number of Spanish-language questionnaires it would need for
Mexico or other Spanish-speaking countries in 2010. Similar problems would
apply to efforts to enumerate other countries.6
o Targeted and aggressive marketing campaign: The key to raising public
awareness of the census is an intensive outreach and promotion campaign.
As noted previously, the Bureau's marketing efforts for the 2000 Census
were far-reaching, and consisted of more than 250 ads in 17 languages that
were part of an effort to reach every household, including those in
historically undercounted populations. Replicating this level of effort on
a global scale would be both difficult and expensive, and the Bureau has
no plans to do so.
o Field infrastructure to execute census and deal with problems: The
Bureau had a vast network of 12 regional offices and 511 local census
offices to implement various operations for the 2000 Census. This
decentralized structure enabled the Bureau to carry out a number of
activities to help ensure a more complete and accurate count, as well as
deal with problems when they arose. Moreover, local census offices are an
important source of intelligence on the various enumeration
6 The Bureau plans to destroy all unused questionnaires for the 2004 test.
obstacles the Bureau faces on the ground. For example, during the 2000
Census, the Bureau called on them to identify hard-to-count population
groups and other challenges, and to develop action plans to address them.
The absence of a field infrastructure for an overseas census means that
the Bureau would have to rely heavily on contractors to conduct the
enumeration, and manage the entire enterprise from its headquarters in
Suitland, Maryland.
o Ability to measure coverage and accuracy: Since 1980, the Bureau has
measured the quality of the decennial census using statistical methods to
estimate the magnitude of any errors. The Bureau reports these estimates
by specific ethnic, racial, and other groups. For methodological reasons,
similar estimates cannot be generated for an overseas census. As a result,
the quality of the overseas count, and thus whether the numbers should be
used for specific purposes, could not be accurately determined.
Conclusions The 2004 test of the feasibility of an overseas enumeration
was an extremely valuable exercise in that it highlighted the numerous
obstacles to a cost-effective count of Americans abroad as an integral
part of the decennial census. Although more comprehensive results will not
be available until the Bureau completes its evaluation of the test early
next year, a key lesson learned is already clear: The current approach to
counting this population group-a voluntary survey that largely relies on
marketing to ensure a complete count-would be costly and yield poor
results. The tools and resources the Bureau has on hand to enumerate
overseas Americans are insufficient for overcoming the inherent obstacles
to a complete count, and it is unlikely that any refinements to this basic
design would produce substantially better results, and certainly not on a
par suitable for purposes of congressional apportionment.
What's more, the Bureau already faces the difficult task of carrying out a
cost-effective stateside enumeration in 2010. Securing a successful count
of Americans in Vienna, Virginia, is challenging enough; a complete count
of Americans in Vienna, Austria-and in scores of other countries around
the globe-would only add to the difficulties facing the Bureau as it looks
toward the next national head count. As a result, we believe that any
further tests or planning activities related to counting Americans
overseas as part of the decennial census would be an imprudent use of the
Bureau's limited resources.
That said, to the extent that Congress desires better data on the number
and characteristics of Americans abroad for various policy-making and
other nonapportionment purposes that require less precision, such
information does not necessarily need to be collected as part of the
decennial census, and could, in fact, be acquired through a separate
survey or other means. To help inform congressional decision making on
this issue, including decisions on whether Americans should be counted and
how the data should be used, it will be important for Congress to have the
results of the Bureau's evaluation of the 2004 overseas census test.
Equally important would be information on the cost, quality of data, and
logistical feasibility of counting Americans abroad using alternatives to
the decennial census. Once Congress knows the tradeoffs of these various
alternatives, it would be better positioned to provide the Bureau with the
direction it needs so that the Bureau could then develop and test an
approach that meets congressional requirements at reasonable resource
levels.
Matters for Given the obstacles to a cost-effective count of overseas
Americans as part
of the decennial census and, more specifically, obtaining data that is
ofCongressional sufficient quality to be used for congressional
apportionment, Congress Consideration may wish to consider eliminating
funding for any additional research,
planning, and development activities related to counting this population
as part of the decennial headcount, including funding for tests planned in
2006 and 2008. However, funding for the evaluation of the 2004 test should
continue as planned to help inform congressional decision making.
Should Congress still desire better data on the number of overseas
Americans, in lieu of the method tested in 2004, Congress might wish to
consider authorizing and funding research on the feasibility of counting
Americans abroad using alternatives to the decennial census.
Recommendations for Executive Action
To facilitate congressional decision making, we recommend that the
Secretary of Commerce ensure that the Bureau completes its evaluation of
the 2004 overseas census test as planned. Further, to the extent that
additional research is authorized and funded, the Bureau, in consultation
with Congress, should explore the feasibility of counting overseas
Americans using alternatives to the decennial census. Potential options
include
o conducting a separate survey,
o examining how the design and archiving of various government agency
administrative records might need to be refined to facilitate a more
accurate count of overseas Americans, and
o exchanging data with other countries' statistical agencies and
censuses, subject to applicable confidentiality and other provisions.
Consideration should also be given to whether the Bureau should conduct
this research on its own or whether it should be contracted out to the
National Academy of Sciences.
Agency Comments and The Secretary of Commerce forwarded written comments
from the U.S. Census Bureau on a draft of this report on August 5, 2004,
which are
Our Evaluation reprinted in the appendix. The Bureau agreed with our
conclusions and recommendations. Furthermore, the Bureau noted, "should
Congress request and fund" further research on counting overseas
Americans, it would be equipped to do that research itself.
As agreed with your offices, unless you release its contents earlier, we
plan no further distribution of this report until 30 days from its issue
date. At that time we will send copies to other interested congressional
committees, the Secretary of Commerce, and the Director of the U.S. Census
Bureau. Copies will be made available to others upon request. This report
will also be available at no charge on GAO's Web site at
http://www.gao.gov.
If you or your staff have any questions concerning this report, please
contact me on (202) 512-6806 or by e-mail at [email protected] or Robert
Goldenkoff, Assistant Director, at (202) 512-2757 or [email protected].
Key contributors to this report were Ellen Grady, Lisa Pearson, and
Timothy Wexler.
Patricia A. Dalton Director Strategic Issues
Appendix I
Comments from the Department of Commerce
Appendix I
Comments from the Department of
Commerce
GAO's Mission The Government Accountability Office, the audit, evaluation
and investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people. GAO
examines the use of public funds; evaluates federal programs and policies;
and provides analyses, recommendations, and other assistance to help
Congress make informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.
Obtaining Copies of The fastest and easiest way to obtain copies of GAO
documents at no cost
is through GAO's Web site (www.gao.gov). Each weekday, GAO postsGAO
Reports and newly released reports, testimony, and correspondence on its
Web site. To Testimony have GAO e-mail you a list of newly posted products
every afternoon, go to
www.gao.gov and select "Subscribe to Updates."
Order by Mail or Phone The first copy of each printed report is free.
Additional copies are $2 each. A check or money order should be made out
to the Superintendent of Documents. GAO also accepts VISA and Mastercard.
Orders for 100 or more copies mailed to a single address are discounted 25
percent. Orders should be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
D.C. 20548
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061
To Report Fraud, Contact:
Waste, and Abuse in Web site: www.gao.gov/fraudnet/fraudnet.htm
E-mail: [email protected] Programs Automated answering system: (800)
424-5454 or (202) 512-7470
Congressional Gloria Jarmon, Managing Director, [email protected] (202)
512-4400 U.S. Government Accountability Office, 441 G Street NW, Room 7125
Relations Washington, D.C. 20548
Public Affairs Jeff Nelligan, Managing Director, [email protected] (202)
512-4800 U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, D.C. 20548
Presorted Standard
Postage & Fees Paid
GAO
Permit No. GI00
United States
Government Accountability Office
Washington, D.C. 20548-0001
Official Business
Penalty for Private Use $300
Address Service Requested
*** End of document. ***