    2010 Census: Counting Americans Overseas as Part of the Decennial
    Census Would Not Be Cost-Effective (19-AUG-04, GAO-04-898).

    The U.S. Census Bureau (Bureau) has typically counted overseas
    members of the military, federal civilian employees, and their
    dependents. However, it usually excluded private citizens
    residing abroad. In July 2004, the Bureau completed a test of the
    practicality of counting all overseas Americans. GAO was asked to
    assess (1) whether the Bureau implemented the test consistent
    with its design, and (2) the lessons learned from the test
    results.
    -------------------------Indexing Terms-------------------------
    REPORTNUM:   GAO-04-898
        ACCNO:   A11698
      TITLE:     2010 Census: Counting Americans Overseas as Part of the
    Decennial Census Would Not Be Cost-Effective
         DATE:   08/19/2004
      SUBJECT:   Americans abroad
    	     Census
    	     Cost effectiveness analysis
    	     Data collection
    	     Public relations
    	     Statistical methods
    	     Surveys
    	     Testing
    	     Comparative analysis
    	     2010 Decennial Census
    	     France
    	     Kuwait
    	     Mexico
    	     2000 Decennial Census

    ******************************************************************
    ** This file contains an ASCII representation of the text of a  **
    ** GAO Product.                                                 **
    **                                                              **
    ** No attempt has been made to display graphic images, although **
    ** figure captions are reproduced.  Tables are included, but    **
    ** may not resemble those in the printed version.               **
    **                                                              **
    ** Please see the PDF (Portable Document Format) file, when     **
    ** available, for a complete electronic file of the printed     **
    ** document's contents.                                         **
    **                                                              **
    ******************************************************************
    GAO-04-898

                     United States Government Accountability Office

           GAO 	Report to the Subcommittee on Technology, Information Policy,
      Intergovernmental Relations and the Census, Committee on Government Reform,
                                House of Representatives

    August 2004

                                      2010 CENSUS

    Counting Americans Overseas as Part of the Decennial Census Would Not Be
    Cost-Effective

                                           a

    GAO-04-898

    Highlights of GAO-04-898, a report to the Subcommittee on Technology,
    Information Policy, Intergovernmental Relations and the Census, Committee
    on Government Reform, House of Representatives

    The U.S. Census Bureau (Bureau) has typically counted overseas members of
    the military, federal civilian employees, and their dependents. However,
    it usually excluded private citizens residing abroad. In July 2004, the
    Bureau completed a test of the practicality of counting all overseas
    Americans. GAO was asked to assess (1) whether the Bureau implemented the
    test consistent with its design, and (2) the lessons learned from the test
    results.

    Congress may wish to consider eliminating funding for additional research
    related to counting Americans abroad as part of the decennial census,
    including funding for tests planned in 2006 and 2008. However, funding for
    the evaluation of the 2004 test should continue as planned, particularly
    to inform congressional decision making on this issue. Should Congress
    desire better data on overseas Americans for certain policymaking and
    other nonapportionment purposes, Congress may wish to consider funding
    research on the feasibility of counting this group using alternatives to
    the decennial census. To facilitate this, we recommend that the Bureau, in
    consultation with Congress, research options such as a separate survey,
    administrative records, and data exchanges with other countries'
    statistical agencies. The Bureau agreed with our conclusions and
    recommendations.

    www.gao.gov/cgi-bin/getrpt?GAO-04-898.

    To view the full product, including the scope and methodology, click on
    the link above. For more information, contact Patricia A. Dalton at (202)
    512-6806 or daltonp@gao.gov.

    August 2004

    2010 CENSUS

    Counting Americans Overseas as Part of the Decennial Census Would Not Be
    Cost-Effective

    The Bureau generally implemented the overseas census test on schedule and
    consistent with its research design. Still, participation was poor, with
    just 5,390 questionnaires returned from the three test sites-France,
    Kuwait, and Mexico. Moreover, because of the low response levels,
    obtaining those questionnaires proved to be quite expensive-around $1,450
    per response, which is far costlier on a unit basis than the 2000 Census.
    Although the two are not directly comparable because the 2000 Census
    included operations not used in the overseas test, the 2000 Census cost
    around $56 per household. Further, boosting the response rate globally
    might not be practical. On the domestic front, during the 2000 Census, the
    Bureau spent $374 million on a months-long publicity campaign that
    consisted of television and other advertising that helped yield a
    72-percent return rate. Replicating this level of effort on a worldwide
    basis would be difficult, and still would not produce a complete count.
    Ensuring a smooth overseas count could also stretch the Bureau's
    resources. For example, at each test site the Bureau encountered various
    challenges that needed to be resolved such as French privacy laws.
    Moreover, managing a complex operation from thousands of miles away also
    proved difficult.

    Enumerating Americans in Mexico and France

    The approach used to count the overseas population in the 2004 test-a
    voluntary survey that largely relies on marketing to secure a complete
    count, lacks the basic building blocks of a successful census. The Bureau
    has done some initial research on alternatives, but all require more
    extensive review. Given that the Bureau already faces the difficult task
    of securing a successful stateside count in 2010, having to simultaneously
    count Americans abroad would only add to the challenges facing the Bureau.

    Contents

        Letter                                                                  1
                                             Results in Brief                   2
                                                Background                      4
                                          Scope and Methodology                 6
                             2004 Overseas Test Was Generally Implemented as    7
                                                 Designed
                             Overseas Census Test Results Were Disappointing    9
                                                and Costly
                                               Conclusions                     20
                                 Matters for Congressional Consideration       21
                                   Recommendations for Executive Action        21
                                    Agency Comments and Our Evaluation         22
    Appendix
                Appendix I:      Comments from the Department of Commerce
                             Table 1: Key Components of Overseas Enumeration
        Tables                                     Were
                                     Generally Implemented as Planned           8
                            Table 2: Comparison of Responses Received for 2004
                                                 Overseas
                                               Census Test                      9
    Figures                  Figure 1: Key Decisions Facing Congress on
                                       Enumerating Americans Abroad             6
                                Figure 2: Census Bureau Ads Placed in the
                                           International Herald
                                                 Tribune                       10
                               Figure 3: Census Materials Were Prominently
                                           Displayed in Various
                                      Locations in France and Mexico           14
                              Figure 4: Key Building Blocks of a Successful    16
                                                  Census

    This is a work of the U.S. government and is not subject to copyright
    protection in the United States. It may be reproduced and distributed in
    its entirety without further permission from GAO. However, because this
    work may contain copyrighted images or other material, permission from the
    copyright holder may be necessary if you wish to reproduce this material
    separately.

    A

    United States Government Accountability Office Washington, D.C. 20548

    August 19, 2004

    The Honorable Adam H. Putnam

    Chairman

    The Honorable Wm. Lacy Clay

    Ranking Minority Member

    Subcommittee on Technology, Information Policy, Intergovernmental
    Relations and the Census

    Committee on Government Reform

    House of Representatives

    Although more than four million American citizens are believed to reside
    abroad, the precise number of overseas Americans is unknown. The U.S.
    Census Bureau (Bureau), the federal agency tasked with counting the
    nation's population every 10 years, has generally included in the census
    overseas members of the military, federal civilian employees, and their
    dependents (a group known collectively as "federally affiliated"
    individuals), but has typically excluded private citizens such as
    retirees, students, and business people.1

    Under the Constitution and federal statutes, the Bureau has discretion
    over whether to count Americans abroad. However, in recent years, the
    Bureau's policy of excluding private citizens from the decennial census
    has been called into question. For example, advocates of an overseas
    census claim that better demographic data on this population group would
    be useful for a variety of policy-making and business purposes, and would
    better represent their unique interests in Congress. Moreover, in January
    2001, Utah sued the Bureau's parent agency, the U.S. Department of
    Commerce, claiming that it lost a congressional seat because the 2000
    Census excluded the state's 11,000 Mormon missionaries and other private
    citizens living abroad.2 According to the Congressional Research Service,
    Utah would have gained a congressional seat had an additional 855 people
    been

    1 Only the 1970, 1990, and 2000 Censuses used counts of federally
    affiliated personnel for purposes of apportioning Congress.

    2 Utah v. Evans, 143 F. Supp. 2d 1290 (D. Utah 2001), aff'd, Utah v.
    Evans, 534 U.S. 1038 (2001).

    included in the state's apportionment totals.3 Utah's suit, however, was
    unsuccessful.

    As we noted in our May 2004 report on this issue, counting Americans
    abroad as an integral part of the 2010 Census would be a monumental task
    that would introduce new resource demands, risks, and uncertainties to an
    endeavor that was already facing a variety of difficulties.4 Specific
    challenges include policy questions such as who should be counted and how
    should the data be used, as well as logistical difficulties such as
    ensuring a complete count and verifying U.S. citizenship.

    To assess the practicality of counting overseas Americans, the Bureau held
    a test enumeration from February through July 2, 2004, in France, Kuwait,
    and Mexico. As agreed with your offices, we assessed (1) whether the
    Bureau implemented the test consistent with its design, and (2) the
    initial lessons learned from the test results and their implications for
    future overseas enumerations.

    To address these objectives, we reviewed applicable planning and other
    documents, and interviewed knowledgeable Bureau officials and
    representatives of private organizations who helped the Bureau promote the
    census at the three test sites. Further, to review how the Bureau was
    implementing the census at the test locations, we made on-site inspections
    in Paris, France, and Guadalajara, Mexico. We conducted our work from
    March through July 2004, in accordance with generally accepted government
    auditing standards.

    Results in Brief	The Bureau generally implemented the 2004 overseas census
    test consistent with its research design. Key elements of the Bureau's
    plan, such as developing a questionnaire specifically for overseas
    Americans and launching a marketing campaign designed to publicize the
    test, were generally carried out as planned. Moreover, the test was
    conducted on schedule.

    3 Congressional Research Service, House Apportionment: Could Census
    Corrections Shift a House Seat?, RS21638 (Washington, D.C.: Oct. 8, 2003).

    4 GAO, 2010 Census: Overseas Enumeration Test Raises Need for Clear Policy
    Direction, GAO-04-470 (Washington, D.C.: May 21, 2004).

    However, the response levels fell far short of what the Bureau planned for
    relative to the number of questionnaires it printed. For example, although
    the Bureau printed about 520,000 census forms for the three test sites-
    France, Kuwait, and Mexico-the actual number of paper responses it
    received only totaled 1,783, as of the end of the test in early July 2004.
    Another 3,607 responses were received via the Internet. Further, because
    of the low response levels, the data were expensive to obtain on a unit
    cost basis-around $1,450 per return. In contrast, the unit cost of the
    2000 Census was about $56 per household. Although the 2000 Census costs
    are not directly comparable to the 2004 overseas test because the 2000
    test included operations not used in the overseas test, the 2000 Census
    was the most expensive census in our nation's history.

    Further, substantially boosting overseas response levels might be
    infeasible. For example, during the 2000 Census, the Bureau spent $374
    million on a months-long publicity campaign that consisted of television,
    radio, and other forms of advertising that helped secure a 72-percent
    return rate. Replicating this level of effort on a worldwide basis would
    be impractical at best, and would not produce a complete count. Indeed,
    even after the Bureau's aggressive publicity effort in 2000, it still
    needed to follow-up with about 42 million households that did not complete
    their census forms.

    Ensuring a smooth overseas count could also stretch the Bureau's
    resources, and thus detract from domestic efforts. For example, at each
    test site the Bureau encountered various difficulties that needed to be
    worked out. The difficulties included addressing French privacy laws and
    delivery problems in Kuwait. Moreover, managing a complex operation from
    thousands of miles away was also hard. This was particularly evident in
    the logistical challenges the Bureau had in overseeing the performance of
    the private firm hired to publicize the census at the three test sites.

    The Bureau's longstanding experience in counting the nation's population
    has made it clear that a cost-effective census is assembled from key
    building blocks that include mandatory participation, a complete and
    accurate address list, and the ability to follow-up with nonrespondents.
    The approach the Bureau used to count the overseas population-a voluntary
    survey that relies heavily on marketing to secure a complete count-largely
    for reasons of practicality, lacks these building blocks, and it is
    unlikely that any refinements to this basic design would produce
    substantially better results. What's more, the Bureau already faces the
    difficult task of securing a successful stateside count in 2010. Having to

    count Americans abroad would only add to the challenges facing the Bureau.

    Given the obstacles to a cost-effective count of overseas Americans as
    part of the decennial census, Congress may wish to consider eliminating
    funding for the research, planning, and development activities related to
    counting this population group using the Bureau's current approach,
    including funding for tests planned in 2006 and 2008. However, funding for
    the evaluation of the 2004 test should continue as planned because it
    could provide useful data to Congress. Moreover, should Congress determine
    that a count of overseas Americans might be useful for certain
    policymaking and other nonapportionment purposes, Congress may wish to
    consider authorizing and funding research on the feasibility of counting
    Americans abroad using alternatives to the decennial census.

    To facilitate congressional decision making, we recommend that the Bureau,
    in consultation with Congress, research potential alternatives to the
    decennial census such as conducting a separate survey, examining how
    administrative records could be refined to produce a more accurate count
    of overseas Americans, and exchanging data with other countries'
    statistical agencies and censuses, subject to applicable confidentiality
    and other provisions. Once Congress knows the tradeoffs of these various
    options, it would be better positioned to provide the Bureau with the
    direction it needs so that the Bureau could then develop and test an
    approach that meets congressional requirements at reasonable resource
    levels.

    The Secretary of Commerce forwarded written comments from the U.S. Census
    Bureau on a draft of this report. The comments are reprinted in the
    appendix.

    The Bureau agreed with our conclusions and recommendations. Furthermore,
    the Bureau noted, "should Congress request and fund" further research on
    counting overseas Americans, it would be equipped to do that research
    itself.

    Background	According to the Bureau, no accurate estimate exists of the
    total number of Americans living abroad. The Constitution and federal law
    give the Bureau discretion to decide whether to count American citizens
    living abroad. In prior censuses, the Bureau has generally included
    "federally affiliated" groups-members of the military and federal
    employees and their

    dependents-but has excluded private citizens residing abroad from all but
    the 1960 and 1970 Censuses. The 2000 Census, using administrative records,
    found 576,367 federally affiliated Americans residing overseas, including
    226,363 military personnel, 30,576 civilian employees, and 319,428
    dependents of both groups.

    In response to congressional direction and the concerns of various private
    organizations, the Census Bureau launched a research and evaluation
    program to assess the practicality of counting both private and federally
    affiliated U.S. citizens residing abroad. The key part of this effort, the
    enumeration, took place from February 2004 through July 2, 2004, in
    France, Kuwait, and Mexico. To promote the overseas census test the Bureau
    relied on third parties-American organizations and businesses in the three
    countries-to communicate to their members and/or customers that an
    overseas enumeration of Americans was taking place and to make available
    to U.S. citizens either the paper questionnaire or Web site address.

    Currently, the Bureau is processing and analyzing the overseas
    questionnaire data and plans to complete an evaluation of the test results
    in early 2005. The Bureau estimates that it will have spent approximately
    $7.8 million in fiscal years 2003 through 2005 to plan, conduct, and
    evaluate the 2004 test. The Bureau has requested additional funds for
    fiscal year 2005 to plan for further testing scheduled for 2006. The
    Bureau also plans to include overseas testing in the 2008 dress rehearsal
    if it were to receive the necessary funding.5

    In May 2004 we reported on the design of the overseas enumeration test and
    concluded that because of various methodological limitations, the test
    results will only partially answer the Bureau's key research objectives
    concerning feasibility (as measured by such indicators as participation
    and number of valid returns), data quality, and cost. Further, we noted
    that the Bureau should not decide on its own whether or not to enumerate
    Americans overseas, and in fact would need congressional guidance on how
    to proceed. As shown in figure 1, the key decisions facing Congress in
    this regard include, in addition to the threshold question of whether
    American residing overseas should be counted, how the data should be

    5 At this point, the Bureau does not have cost data beyond fiscal year
    2005. Therefore, it is unknown what the costs will be for implementing and
    evaluating the 2006 overseas test or the 2008 overseas dress rehearsal.

    used and whether to enumerate this population group as part of the
    decennial census.

        Figure 1: Key Decisions Facing Congress on Enumerating Americans Abroad

                                 Source: GAO analysis.

    We also recommended that if further testing were to occur, that the Bureau
    resolve the shortcomings of the design of the 2004 test and better address
    the objectives of an overseas enumeration.

    Scope and	As agreed with your offices, our objectives for this report were
    to assess (1) whether the Bureau implemented the test consistent with its
    design,

    Methodology	and (2) the initial lessons learned from the test results and
    their implications for future overseas enumerations. To assess the first
    objective, we interviewed Bureau officials and compared the Bureau's test
    plans with what was actually done at the three test sites. We visited
    Paris, France, and Guadalajara, Mexico, to obtain the views of 12 private,
    civic, and other

    organizations on the implementation of the overseas census test, and/or to
    confirm at 36 organizations the availability of census material. In
    addition, to a more limited extent, we interviewed officials from third
    party organizations in Kuwait via the telephone or e-mail. We judgmentally
    selected these organizations because they had agreed to display census
    promotional materials and, in some cases, had also agreed to do one or
    more of the following activities: make available paper copies of the
    census questionnaire, publish information in a newsletter, post a link to
    a Web site, send outreach e-mail to members, and/or create speaking
    opportunities to discuss the census. The results of these visits are not
    necessarily representative of the larger universe of third-party
    organizations.

    To assess the implications of the test results on future overseas
    enumerations and the 2010 census, we obtained from Bureau officials
    preliminary results of the overseas census by test site and response mode
    as well as cost data. We also interviewed officials from the Bureau and
    third-party organizations to determine what lessons were learned from the
    test and the implications on future overseas enumeration efforts.

    2004 Overseas Test The Bureau's design for the 2004 overseas enumeration
    test was generally

    implemented as planned and completed on schedule. The Bureau's designWas
    Generally had four key components: the mode of response, the questionnaire
    Implemented as designed specifically for Americans living overseas, three
    test sites, and an Designed outreach and promotion program designed to
    communicate and educate

    Americans abroad that a test census was being conducted. Table 1

    describes each of these components in greater detail.

    Table 1: Key Components of Overseas Enumeration Were Generally Implemented
    as Planned

    Generally Key overseas test implemented component What the Bureau planned
    as planned? GAO observations

    Census form response Census questionnaire will be available Yes Paper
    version of the census form was available at

    mode in paper format or via the Internet.�various locations at the
    test sites. Questionnaire was available for completion on the Internet.
    Paper version was also available in Spanish in Mexico.

    Census test  2000 decennial short  Yes Asked respondents to provide their
        questions           form                           passport
                   questionnaire will be      numbers and social security numbers
                   modified for the           to verify
                   overseas enumeration.                 citizenship.
                                                    Asked respondents about their
                                                               employment status-
                                                 military, federal, or other.
                                                Asked for information on everyone
                                                                 in the household
                                               even if the person was not a U.S.
                                                           citizen.

    Test sites       Conduct test in three Yes   Overseas test was implemented
                               geographically                 in three countries:
                   diverse areas with large         France, Kuwait, and Mexico.
                           American
                       populations, and where
                               administrative
                  records may be available to
                                  help verify
                           results.

    Outreach/       Contract with a public Yes Public relations firm hired to
                            relations firm to                develop a
    Promotion     develop a communications         communication strategy.
                                  strategy to
                          inform and motivate       Strategy relied on public and
                           respondents living            private organizations in
                 in the selected countries to         each of the test sites.
                                   answer the
                           census.                        Organizations displayed
                                                      promotional materials about
                                                      the test, communicated test
                                                           census to members, and
                                                  distributed the paper census
                                                  forms to American
                                                            residents.

    Source: GAO.

    However, while the test was generally implemented as designed, our earlier
    report pointed out several methodological limitations with the design,
    such as not being able to calculate response rates because the universe of
    Americans is unknown or not being able to measure the quality of data
    because of the impracticality of developing an address list. As we discuss
    later in this report, it is these methodological limitations that impede
    the Bureau's ability to implement a successful overseas enumeration.

      Overseas Census Test Results Were Disappointing and Costly

    Although the 2004 overseas enumeration test ended in early July 2004 and
    the Bureau has just begun evaluating the results, the response levels were
    poor, and very expensive to obtain on a per unit basis. The response level
    to the overseas enumeration suggests that the current approach to counting
    overseas Americans-a voluntary survey that relies heavily on marketing to
    get people to participate-by itself cannot secure a successful head count.
    Further, obtaining the additional resources needed to produce
    substantially better results may not be feasible, and still not yield data
    that are comparable in quality to the stateside enumeration.

        Response Levels Were Disappointing and Costly to Obtain

    The 5,390 responses the Bureau received for this test were far below what
    the Bureau planned for when printing up materials and census forms. While
    the Bureau ordered 520,000 paper forms for three test sites, only 1,783
    census forms were completed and returned. Of these, 35 were Spanish
    language forms that were made available in Mexico. The remaining 3,607
    responses were completed via the Internet. Table 2 below shows the number
    of census questionnaires that the Bureau printed for each country and the
    number of responses they actually received in both the paper format and
    via the Internet.

    Table 2: Comparison of Responses Received for 2004 Overseas Census Test

    Number of responses by mode

                                       Test sites

    Number of forms printed for each test site Paper Internet

    Total number of responses received

                    Mexico        430,000a     869b            1,130        1,999
                    France          75,000     886             2,219        3,105
                    Kuwait          15,000           28     258
                     Total         520,000    1,783            3,607        5,390

    Source: U.S. Census Bureau.

    aThis includes 100,000 forms printed in Spanish. bThis includes 35 Spanish
    forms returned.

    Because of the low response levels, in early April 2004, the Bureau
    reversed its decision to not use paid advertising and in May 2004
    initiated a paid advertising campaign in France and Mexico. This included
    print and

    Internet ads in France and print and radio ads in Mexico. See figure 2 for
    examples of the ads used in the paid advertising campaign.

    Figure 2: Census Bureau Ads Placed in the International Herald Tribune

    Source: GAO.

    A Bureau official told us the ad campaign for the 2004 overseas test cost
    about $206,000. This official said there were surplus funds available in
    the project budget to use for this purpose due to lower than expected
    processing and postage costs for the overseas test. While the Bureau saw

    some increase in the number of responses after the paid advertising
    campaign began, this official said the increase was slight.

        Return on Investment for Overseas Enumeration is Low

    Not only were response levels low, they were extremely expensive to obtain
    on a unit basis-roughly $1,450 for each returned questionnaire, based on
    the $7.8 million the Bureau spent preparing for, implementing, and
    evaluating the 2004 overseas test. In contrast, the unit cost of the 2000
    Census was about $56 per household. Although the two surveys are not
    directly comparable because the 2000 Census costs covered operations not
    used in the overseas test, the 2000 Census was still the most expensive
    census in our nation's history.

        Securing a Higher Return Rate Would Be an Enormous Challenge

    The main reason for the high unit cost is the low return rate. However,
    significantly boosting participation levels may not be feasible. The
    Bureau's experience in the 2000 Census highlights the level of effort that
    was needed to raise public awareness about the census and get people to
    complete their forms. For the 2000 decennial, the Bureau spent $374
    million on a comprehensive marketing, communications, and partnership
    effort. The campaign consisted of a five-part strategy conducted in three
    waves beginning in the fall of 1999 and continuing past Census Day (April
    1, 2000). The effort helped secure a 72-percent return rate. Specific
    elements included television, radio, and other mass media advertising;
    promotions and special events; and a census-in-schools program. Thus, over
    a period of several months, the American public was on the receiving end
    of a steady drumbeat of advertising aimed at publicizing the census and
    motivating them to respond.

    The Bureau also filled 594 full-time partnership specialist positions.
    These individuals were responsible for mobilizing support for the census
    on a grassroots basis by working with governmental entities, private
    companies, and religious and social service groups, and other
    organizations.

    Replicating this level of effort on a worldwide basis would be
    impractical, and still would not produce a complete count. Indeed, even
    after the Bureau's aggressive marketing effort in 2000, it still had to
    follow-up with about 42 million households that did not return their
    census forms. Moreover, because there are no reliable figures on the
    number of Americans residing overseas, the Bureau would not have a good
    measure of the number of people that did not participate, or the overall
    quality of the data.

        Ensuring a Smooth Overseas Enumeration Could Stretch the Bureau's Resources

    Country-specific Issues Created Implementation Problems

    The Bureau's experience in conducting the 2004 overseas test underscored
    the difficulties of administering a complex operation from thousands of
    miles away. Not surprisingly, as with any operation this complex, various
    challenges and unforeseen problems arose. While the Bureau was able to
    resolve them, its ability to do so should there be a full overseas
    enumeration as part of the 2010 Census, would be highly questionable as
    far more resources would be required. This was particularly evident in at
    least two areas: grappling with country-specific issues and overseeing the
    contractor responsible for raising public awareness of the census at the
    three test sites.

    The Bureau encountered a variety of implementation problems at each of the
    test sites. In some cases the problems were known in advance, in others,
    glitches developed at the last minute. Although such difficulties are to
    be expected given the magnitude of the Bureau's task, a key lesson learned
    from the test is that there would be no economy of scale in ramping up to
    a full enumeration of Americans abroad. In fact, just the opposite would
    be true. Because of the inevitability of country-specific problems, rather
    than conducting a single overseas count based on a standard set of rules
    and procedures (as is the case with the stateside census), the Bureau
    might end up administering what amounts to dozens of separate censuses-one
    for each of the countries it enumerates-each with its own set of
    procedures adapted to each country's unique requirements. The time and
    resources required to do this would likely be overwhelming and detract
    from the Bureau's stateside efforts.

    For example, during the overseas test, the Bureau found that French
    privacy laws restrict the collection of personal data such as race and
    ethnic information. However, these data are collected as part of the
    decennial census because they are key to implementing a number of civil
    rights laws such as the Voting Rights Act.

    Addressing France's privacy laws took a considerable amount of negotiation
    between the two countries, and was ultimately resolved after a formal
    agreement was developed. The Bureau issued and posted on its Web site an
    advisory that informed Americans living in France that it was not
    mandatory to respond to the questionnaire, the only recipient of the
    collected data is the Census Bureau, the data will be kept for one year,
    and the respondent has a right to access and correct the data collected.
    The Bureau was able to collect race and ethnic data-generally a prohibited
    practice without the respondents' permission-only after it received

    special approval from a French government agency. Initially, however, it
    looked as if the Bureau might have to redesign the census form if it
    wanted to use it in France.

    In Kuwait, delivery of the census materials was delayed by several weeks
    at the beginning of the test because they were accidentally addressed to
    the wrong contractor. Ultimately, the U.S. Embassy stepped-in to accept
    the boxes so that the enumeration could proceed. In Mexico, there was some
    initial confusion on the part of Mexican postal workers as to whether they
    could accept the postage-paid envelopes that the Bureau had provided to
    return the paper questionnaires for processing in the United States.

    Because of the small number of countries involved in the test, the Bureau
    was able to address the various problems it encountered. Still, the
    Bureau's experience indicates that it will be exceedingly difficult to
    identify and resolve in advance all the various laws, rules, societal
    factors, and a host of other potential glitches that could affect a full
    overseas enumeration.

    On-site Supervision of As noted previously, the Bureau hired a public
    relations firm to develop a

    Contractor Was Problematic	communications strategy to inform and motivate
    respondents living in the test countries to complete the census. The
    firm's responsibilities included identifying private companies, religious
    institutions, service organizations, and other entities that have contact
    with Americans abroad and could thus help publicize the census test.
    Specific activities the organizations could perform included displaying
    promotional materials and paper versions of the census questionnaire,
    publishing information in a newsletter, and posting information on their
    Web sites. Although the public relations firm appeared to go to great
    lengths to enlist the participation of these various entities-soliciting
    the support of hundreds of organizations in the three countries-the test
    revealed the difficulties of adequately overseeing a contractor operating
    in multiple sites overseas.

    For example, the public relations firm's tracking system indicated that
    around 440 entities had agreed to perform one or more types of promotional
    activities. However, our on-site inspections of several of these
    organizations in Paris, France, and Guadalajara, Mexico, that had agreed
    to display the census materials and/or distribute the questionnaires,
    uncovered several glitches. Of the 36 organizations we visited that were
    supposed to be displaying promotional literature, we found the information
    was only available at 15. In those cases, as shown in figure 3, the
    materials were generally displayed in prominent locations, typically on a
    table with posters on a nearby wall.

    Figure 3: Census Materials Were Prominently Displayed in Various Locations
    in France and Mexico

    Source: GAO.

    Five of these 15 organizations were also distributing the census
    questionnaire, but the forms were not readily accessible.

    However, at 21 sites we visited, we found various discrepancies between
    what the public relations firm indicated had occurred, and what actually
    took place. For example, while the firm's tracking system indicated that
    questionnaires would be available at a restaurant and an English-language
    bookstore in Guadalajara, none were available. In fact, the owner of the
    bookstore told us that no one from the Census Bureau or the public

    relations firm had contacted her about displaying materials for the
    overseas test.

    At the University of Guadalajara, although the tracking system indicated
    that an official had been contacted about, and agreed to help support the
    census test, that official told us no one had contacted him. As a result,
    when boxes of census materials were delivered to his school without any
    explanatory information, he did not know what to do with them, and had to
    telephone the U.S. Consulate in Guadalajara to figure out what they were
    for.

    Likewise, in Paris, we went to several locations where the tracking system
    indicated that census information would be available. None was. In fact,
    at some of these sites, not only was there no information about the
    census, but there was no indication that the organization we were looking
    for was at the address we had from the database.

    The results of the overseas test point to the difficulties of overseeing
    the contractor's performance. As census materials were made available at
    scores of locations across the three test countries, it would have been
    impractical for the Bureau to inspect each site. The difficulty of
    supervising contractors-and any field operation for that matter-would only
    be magnified in a global enumeration.

    The Design of the Overseas The Bureau's experience in counting the
    nation's population for the 2000 Census Lacks the Building and earlier
    censuses sheds light on some of the specific operations and Blocks of a
    Successful other elements that together form the building blocks of a
    successful head Census count (see fig. 4).

    Figure 4: Key Building Blocks of a Successful Census

    Source: GAO.

    While performing these activities does not necessarily guarantee a
    costeffective headcount, not performing them makes a quality count far
    less promising and puts the entire enterprise at risk. The current
    approach to counting overseas Americans lacks these building blocks, as
    most are infeasible to perform on an overseas population. Each is
    discussed in greater detail below.

    o 	Mandatory participation: Under federal law, all persons residing in the
    United States regardless of citizenship status are required to respond to
    the decennial census. By contrast, the overseas enumeration test was
    conducted as a voluntary survey where participation was optional. The
    Bureau has found that response rates to mandatory surveys are higher than
    the response rates to voluntary surveys. This in turn yields more complete
    data and helps hold down costs.

    o 	Early agreement on design: Both Congress and the Bureau need to agree
    on the fundamental design of an overseas census. Concurrence on the design
    helps ensure adequate planning, testing and funding levels. Conversely,
    the lack of an agreed-upon design raises the risk that basic design
    elements might change in the years ahead, while the opportunities to test
    those changes and integrate them with other operations will diminish.
    Under the Bureau's current plans, after the 2006 test, the Bureau would
    have just one more opportunity to test its prototype for an overseas
    enumeration-a dress rehearsal in 2008. Any design changes after 2008 would
    not be tested in a real-world environment.

    The design of the census is driven in large part by the purposes for which
    the data will be used. Currently, no decisions have been made on whether
    the overseas data will be used for purposes of congressional
    apportionment, redistricting, allocating federal funds, or other
    applications. Some applications, such as apportionment, would require
    precise population counts and a very rigorous design that parallels the
    stateside count. Other applications, however, could get by with less
    precision and thus, a less stringent approach.

    As we noted previously, Congress will need to decide whether or not to
    count overseas Americans, and how the results should be used. The basis
    for these determinations needs to be sound research on the cost, quality
    of data, and logistical feasibility of the range of options for counting
    this population group. Possibilities include counting Americans via a
    separate survey, administrative records such as passport and voter
    registration forms, and/or records maintained by other countries such as
    published census records and work permits.

    The Bureau's initial research has shown that each of these options has
    coverage, accuracy, and accessibility issues, and some might introduce
    systemic biases into the data. Far more extensive research would be needed
    to determine the feasibility of these or other potential approaches. Once
    Congress knows the tradeoffs of these various alternatives, it will be
    better positioned to provide the Bureau with the guidance it needs to go
    beyond research and conduct field tests of specific approaches. The Bureau
    can conduct the research, or it can contract it out. Indeed, the National
    Research Council of the National Academy of Sciences has conducted a
    number of studies on the decennial census, including a review of the 2000
    Census and an examination of reengineering the 2010 Census.

    o 	A complete and accurate address list: The cornerstone of a successful
    census is a quality address list. For the stateside census, the Bureau
    goes to great lengths to develop what is essentially an inventory of all
    known living quarters in the United States, including sending census
    workers to canvass every street in the nation to verify addresses. The
    Bureau uses this information to deliver questionnaires, follow up with
    nonrespondents, determine vacancies, and identify households the Bureau
    may have missed or counted more than once. Because it would be impractical
    to develop an accurate parallel address list for overseas Americans, these
    operations would be impossible and the quality of the data would suffer as
    a result.

    o 	Ability to detect invalid returns: Ensuring the integrity of the census
    data requires the Bureau to have a mechanism to screen out invalid
    responses. Stateside, the Bureau does this by associating an
    identification number on the questionnaire to a specific address in the
    Bureau's address list, as well as by field verification. However, the
    Bureau's current approach to counting overseas Americans is unable to
    determine whether or not a respondent does in fact reside abroad. So long
    as a respondent provides certain pieces of information on the census
    questionnaire, it will be eligible for further processing. The Bureau is
    unable to confirm the point of origin for questionnaires completed on the
    Internet, and postmarks on a paper questionnaire only tell the location
    from which a form was mailed, not the place of residence of the
    respondent. The Bureau has acknowledged that ensuring such validity might
    be all but impossible for any reasonable level of effort and funding.

    o 	Ability to follow up with non-respondents: Because participation in the
    decennial census is mandatory, the Bureau sends enumerators to those
    households that do not return their questionnaires. In cases where
    household members cannot be contacted or refuse to answer all or part of a
    census questionnaire, enumerators are to obtain data from neighbors, a
    building manager, or other nonhousehold member presumed to know about its
    residents. The Bureau also employs statistical techniques to impute data
    when it lacks complete information on a household. Thus, by the end of
    each decennial census, the Bureau has a fairly exhaustive count of
    everyone in the nation. As noted above, because the Bureau lacks an
    address list of overseas Americans, it is unable to follow up with
    nonrespondents or impute information on missing households. As a result,
    the Bureau will never be able to obtain a complete count of overseas
    Americans.

    o 	Cost model for estimating needed resources: The Bureau uses a cost
    model and other baseline data to help it estimate the resources it needs
    to conduct the stateside census. Key assumptions such as response levels
    and workload are developed based on the Bureau's experience in counting
    people decade after decade. However, the Bureau has only a handful of data
    points with which to gauge the resources necessary for an overseas census,
    and the tests it plans on conducting will only be of limited value in
    modeling the costs of conducting a worldwide enumeration in 2010.

    The lack of baseline data could cause the Bureau to over-or underestimate
    the staffing, budget and other requirements of an overseas count. For
    example, this was evident during the 2004 overseas test when the Bureau
    estimated it would need around 100,000 Spanishlanguage questionnaires for
    the Mexican test site. As only 35 Spanishlanguage questionnaires were
    returned, it is now clear that the Bureau could have gotten by with
    printing far fewer questionnaires for Mexico. However, the dilemma for the
    Bureau is that its experience in the 2004 overseas test cannot be used to
    project the number of Spanish-language questionnaires it would need for
    Mexico or other Spanish-speaking countries in 2010. Similar problems would
    apply to efforts to enumerate other countries.6

    o 	Targeted and aggressive marketing campaign: The key to raising public
    awareness of the census is an intensive outreach and promotion campaign.
    As noted previously, the Bureau's marketing efforts for the 2000 Census
    were far-reaching, and consisted of more than 250 ads in 17 languages that
    were part of an effort to reach every household, including those in
    historically undercounted populations. Replicating this level of effort on
    a global scale would be both difficult and expensive, and the Bureau has
    no plans to do so.

    o 	Field infrastructure to execute census and deal with problems: The
    Bureau had a vast network of 12 regional offices and 511 local census
    offices to implement various operations for the 2000 Census. This
    decentralized structure enabled the Bureau to carry out a number of
    activities to help ensure a more complete and accurate count, as well as
    deal with problems when they arose. Moreover, local census offices are an
    important source of intelligence on the various enumeration

    6 The Bureau plans to destroy all unused questionnaires for the 2004 test.

    obstacles the Bureau faces on the ground. For example, during the 2000
    Census, the Bureau called on them to identify hard-to-count population
    groups and other challenges, and to develop action plans to address them.
    The absence of a field infrastructure for an overseas census means that
    the Bureau would have to rely heavily on contractors to conduct the
    enumeration, and manage the entire enterprise from its headquarters in
    Suitland, Maryland.

    o 	Ability to measure coverage and accuracy: Since 1980, the Bureau has
    measured the quality of the decennial census using statistical methods to
    estimate the magnitude of any errors. The Bureau reports these estimates
    by specific ethnic, racial, and other groups. For methodological reasons,
    similar estimates cannot be generated for an overseas census. As a result,
    the quality of the overseas count, and thus whether the numbers should be
    used for specific purposes, could not be accurately determined.

    Conclusions	The 2004 test of the feasibility of an overseas enumeration
    was an extremely valuable exercise in that it highlighted the numerous
    obstacles to a cost-effective count of Americans abroad as an integral
    part of the decennial census. Although more comprehensive results will not
    be available until the Bureau completes its evaluation of the test early
    next year, a key lesson learned is already clear: The current approach to
    counting this population group-a voluntary survey that largely relies on
    marketing to ensure a complete count-would be costly and yield poor
    results. The tools and resources the Bureau has on hand to enumerate
    overseas Americans are insufficient for overcoming the inherent obstacles
    to a complete count, and it is unlikely that any refinements to this basic
    design would produce substantially better results, and certainly not on a
    par suitable for purposes of congressional apportionment.

    What's more, the Bureau already faces the difficult task of carrying out a
    cost-effective stateside enumeration in 2010. Securing a successful count
    of Americans in Vienna, Virginia, is challenging enough; a complete count
    of Americans in Vienna, Austria-and in scores of other countries around
    the globe-would only add to the difficulties facing the Bureau as it looks
    toward the next national head count. As a result, we believe that any
    further tests or planning activities related to counting Americans
    overseas as part of the decennial census would be an imprudent use of the
    Bureau's limited resources.

    That said, to the extent that Congress desires better data on the number
    and characteristics of Americans abroad for various policy-making and
    other nonapportionment purposes that require less precision, such
    information does not necessarily need to be collected as part of the
    decennial census, and could, in fact, be acquired through a separate
    survey or other means. To help inform congressional decision making on
    this issue, including decisions on whether Americans should be counted and
    how the data should be used, it will be important for Congress to have the
    results of the Bureau's evaluation of the 2004 overseas census test.
    Equally important would be information on the cost, quality of data, and
    logistical feasibility of counting Americans abroad using alternatives to
    the decennial census. Once Congress knows the tradeoffs of these various
    alternatives, it would be better positioned to provide the Bureau with the
    direction it needs so that the Bureau could then develop and test an
    approach that meets congressional requirements at reasonable resource
    levels.

    Matters for Given the obstacles to a cost-effective count of overseas
    Americans as part

    of the decennial census and, more specifically, obtaining data that is
    ofCongressional sufficient quality to be used for congressional
    apportionment, Congress Consideration may wish to consider eliminating
    funding for any additional research,

    planning, and development activities related to counting this population
    as part of the decennial headcount, including funding for tests planned in
    2006 and 2008. However, funding for the evaluation of the 2004 test should
    continue as planned to help inform congressional decision making.

    Should Congress still desire better data on the number of overseas
    Americans, in lieu of the method tested in 2004, Congress might wish to
    consider authorizing and funding research on the feasibility of counting
    Americans abroad using alternatives to the decennial census.

      Recommendations for Executive Action

    To facilitate congressional decision making, we recommend that the
    Secretary of Commerce ensure that the Bureau completes its evaluation of
    the 2004 overseas census test as planned. Further, to the extent that
    additional research is authorized and funded, the Bureau, in consultation
    with Congress, should explore the feasibility of counting overseas
    Americans using alternatives to the decennial census. Potential options
    include

    o  conducting a separate survey,

    o 	examining how the design and archiving of various government agency
    administrative records might need to be refined to facilitate a more
    accurate count of overseas Americans, and

    o 	exchanging data with other countries' statistical agencies and
    censuses, subject to applicable confidentiality and other provisions.

    Consideration should also be given to whether the Bureau should conduct
    this research on its own or whether it should be contracted out to the
    National Academy of Sciences.

    Agency Comments and 	The Secretary of Commerce forwarded written comments
    from the U.S. Census Bureau on a draft of this report on August 5, 2004,
    which are

    Our Evaluation	reprinted in the appendix. The Bureau agreed with our
    conclusions and recommendations. Furthermore, the Bureau noted, "should
    Congress request and fund" further research on counting overseas
    Americans, it would be equipped to do that research itself.

    As agreed with your offices, unless you release its contents earlier, we
    plan no further distribution of this report until 30 days from its issue
    date. At that time we will send copies to other interested congressional
    committees, the Secretary of Commerce, and the Director of the U.S. Census
    Bureau. Copies will be made available to others upon request. This report
    will also be available at no charge on GAO's Web site at
    http://www.gao.gov.

    If you or your staff have any questions concerning this report, please
    contact me on (202) 512-6806 or by e-mail at daltonp@gao.gov or Robert
    Goldenkoff, Assistant Director, at (202) 512-2757 or goldenkoffr@gao.gov.
    Key contributors to this report were Ellen Grady, Lisa Pearson, and
    Timothy Wexler.

    Patricia A. Dalton Director Strategic Issues

    Appendix I

    Comments from the Department of Commerce

    Appendix I
    Comments from the Department of
    Commerce

    GAO's Mission	The Government Accountability Office, the audit, evaluation
    and investigative arm of Congress, exists to support Congress in meeting
    its constitutional responsibilities and to help improve the performance
    and accountability of the federal government for the American people. GAO
    examines the use of public funds; evaluates federal programs and policies;
    and provides analyses, recommendations, and other assistance to help
    Congress make informed oversight, policy, and funding decisions. GAO's
    commitment to good government is reflected in its core values of
    accountability, integrity, and reliability.

    Obtaining Copies of The fastest and easiest way to obtain copies of GAO
    documents at no cost

    is through GAO's Web site (www.gao.gov). Each weekday, GAO postsGAO
    Reports and newly released reports, testimony, and correspondence on its
    Web site. To Testimony have GAO e-mail you a list of newly posted products
    every afternoon, go to

    www.gao.gov and select "Subscribe to Updates."

    Order by Mail or Phone	The first copy of each printed report is free.
    Additional copies are $2 each. A check or money order should be made out
    to the Superintendent of Documents. GAO also accepts VISA and Mastercard.
    Orders for 100 or more copies mailed to a single address are discounted 25
    percent. Orders should be sent to:

    U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
    D.C. 20548

    To order by Phone:	Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
    512-6061

      To Report Fraud, Contact:
      Waste, and Abuse in Web site: www.gao.gov/fraudnet/fraudnet.htm

    E-mail: fraudnet@gao.govFederal Programs Automated answering system: (800)
    424-5454 or (202) 512-7470

    Congressional	Gloria Jarmon, Managing Director, JarmonG@gao.gov (202)
    512-4400 U.S. Government Accountability Office, 441 G Street NW, Room 7125

    Relations Washington, D.C. 20548

    Public Affairs	Jeff Nelligan, Managing Director, NelliganJ@gao.gov (202)
    512-4800 U.S. Government Accountability Office, 441 G Street NW, Room 7149
    Washington, D.C. 20548

                                   Presorted Standard
                                  Postage & Fees Paid
                                          GAO
                                    Permit No. GI00

    United States
    Government Accountability Office
    Washington, D.C. 20548-0001

    Official Business
    Penalty for Private Use $300

    Address Service Requested
    *** End of document. ***