2010 Census: Bureau Needs to Specify How It Will Assess Coverage 
Follow-up Techniques and When It Will Produce Coverage		 
Measurement Results (15-APR-08, GAO-08-414).			 
                                                                 
An accurate decennial census relies on finding and counting	 
people--only once--in the right place and collecting complete and
correct information. GAO reviewed the Department of Commerce's	 
U.S. Census Bureau's (Bureau) plans to (1) improve the census's  
accuracy--or coverage--with its coverage follow-up operation, (2)
assess the accuracy of the census through its census coverage	 
measurement program, and (3) evaluate these efforts after 2010.  
GAO reviewed documentation and interviewed Bureau officials about
the Bureau's coverage follow-up and measurement plans. GAO	 
compared plans for measuring coverage in 2000 and 2010 and	 
interviewed 15 experts about changes to the plans.		 
-------------------------Indexing Terms------------------------- 
REPORTNUM:   GAO-08-414 					        
    ACCNO:   A81676						        
  TITLE:     2010 Census: Bureau Needs to Specify How It Will Assess  
Coverage Follow-up Techniques and When It Will Produce Coverage  
Measurement Results						 
     DATE:   04/15/2008 
  SUBJECT:   Census						 
	     Data collection					 
	     Data integrity					 
	     Design specifications				 
	     Operational testing				 
	     Program evaluation 				 
	     Program management 				 
	     Records management 				 
	     Strategic planning 				 
	     Surveys						 
	     Systems analysis					 
	     Systems conversions				 
	     Systems design					 
	     Systems evaluation 				 
	     Systems integrity					 
	     Systems testing					 
	     2010 Decennial Census				 

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO Product.                                                 **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
******************************************************************
GAO-08-414

   

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

Report to the Chairman, Committee on Oversight and Government Reform, 
House of Representatives: 

United States Government Accountability Office: 

GAO: 

April 2008: 

2010 Census: 

Bureau Needs to Specify How It Will Assess Coverage Follow-up 
Techniques and When It Will Produce Coverage Measurement Results: 

Improving and Measuring Census Coverage: 

GAO-08-414: 

GAO Highlights: 

Highlights of GAO-08-414, a report to the Chairman, Committee on 
Oversight and Government Reform, House of Representatives. 

Why GAO Did This Study: 

An accurate decennial census relies on finding and counting peopleï¿½only 
onceï¿½in the right place and collecting complete and correct 
information. GAO reviewed the Department of Commerceï¿½s U.S. Census 
Bureauï¿½s (Bureau) plans to (1) improve the censusï¿½s accuracyï¿½or 
coverageï¿½with its coverage follow-up operation, (2) assess the accuracy 
of the census through its census coverage measurement program, and (3) 
evaluate these efforts after 2010. 

GAO reviewed documentation and interviewed Bureau officials about the 
Bureauï¿½s coverage follow-up and measurement plans. GAO compared plans 
for measuring coverage in 2000 and 2010 and interviewed 15 experts 
about changes to the plans. 

What GAO Found: 

The Bureau plans to expand its coverage follow-up operation to improve 
the accuracy of enumeration data collected in 2010. As in Census 2000, 
coverage follow-up is intended to resolve count discrepancies (when the 
number of household members reported on a completed questionnaire fails 
to match the number of persons for whom information is collected) and 
complete the enumeration of large households (more than six persons 
living in a housing unit). For 2010, the Bureau researched new 
techniques to increase the number of ways a person or household could 
be selected for coverage follow-up. These include (1) using coverage 
probes on the census form to identify cases with potential under- or 
overcount problems, (2) using administrative records to identify 
households that include persons who may not have been counted, and (3) 
using computer matching to determine potential duplicate persons. 
However, it is not clear on what basis the Bureau will select or 
prioritize cases from this potentially expanded universe. The Bureau 
plans to further assess the techniques during the 2008 Dress Rehearsal, 
leaving little time to consider and implement improvements for the 2010 
coverage follow-up operation. Because of the potential for the coverage 
follow-up workload to increase, it will be important for the Bureau to 
be clear on how it will decide which techniques to use and how it will 
select coverage follow-up cases to ensure that the operation may best 
improve coverage. 

Although the Bureauï¿½s overall design for measurement is similar in many 
respects to the design for 2000, its measurement plan differs in 
important ways. The Bureau is planning to use a sample design similar 
to that used in 2000 and still intends to produce net coverage error 
estimates. However, the Bureau does not plan to use measurement results 
to adjust the 2010 count of the nationï¿½s population and is planning to 
make four significant changes: (1) conduct the measurement interview 4 
months later than it did in 2000, (2) use a different methodï¿½logistic 
regressionï¿½to estimate net error, (3) use estimates of the components 
of coverage error in an effort to improve future decennials, and (4) 
expand computer matching and clerical matching to match persons in its 
postenumeration survey to the entire census. Still, the Bureau has not 
specified when it will produce estimates of net coverage error and the 
components of coverage error for the 2010 Census. The Bureau also has 
not clearly described how it will associate the components of coverage 
error to census operations in order to improve future decennials. 
Because of the Bureauï¿½s changes to its coverage measurement program, it 
is important for the Bureau to solidify its plans and share them with 
stakeholders to ensure that there is an agreement on what the program 
will produce and when the Bureau will produce its results. 

The Bureau is in the early stages of planning how it will evaluate 
coverage follow-up and measurement after the 2010 Census. As part of 
this effort, the Bureau is considering recommendations of the National 
Academy of Sciences for the Bureauï¿½s program of evaluations and 
experiments for Census 2010. The Bureau has not yet developed a 
timeline for completing its evaluation plans. 

What GAO Recommends: 

GAO recommends that the Secretary of Commerce direct the Bureau to 
specify (1) criteria it will use to assess techniques for identifying 
coverage follow-up cases; (2) when it will provide coverage measurement 
estimates and how it will relate components of coverage error to 
specific census operations to improve future decennials; and (3) key 
decision points and plans for evaluating aspects of the 2010 coverage 
follow-up and measurement efforts. In commenting on a draft of this 
report, Commerce had no substantive disagreements with our 
recommendations and will develop action plans for each one; and 
provided technical comments that we incorporated as appropriate. 

To view the full product, including the scope and methodology, click on 
[hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-08-414]. For more 
information, contact Mathew J. Scirï¿½ at (202) 512-6806 or 
[email protected]. 

[End of section] 

Contents: 

Letter: 

Results in Brief: 

Background: 

Bureau Plans to Expand Coverage Follow-up to Improve Census 2010 but 
Has Not Finalized Plans for Identifying Follow-up Cases: 

Bureau Has Made Changes to Coverage Measurement for 2010 Intended to 
Benefit the 2020 Census, but Key Aspects of Program Plan Remain 
Unclear: 

Bureau Plans for Evaluating the 2010 Coverage Follow-up Operation and 
Census Coverage Measurement Program Are in Early Stages: 

Conclusions: 

Recommendations for Executive Action: 

Agency Comments and Our Evaluation: 

Appendix I: Objectives, Scope, and Methodology: 

Appendix II: Comments from the Department of Commerce: 

Appendix III: GAO Contact and Staff Acknowledgments: 

Figures: 

Figure 1: Decennial Census Population Net Undercount Rates from the 
Bureau's Demographic Analysis in Percentages, 1940-2000: 

Figure 2: Operation Flowchart for the Bureau's 2008 Dress Rehearsal 
Coverage Follow-up Operation: 

Figure 3: Undercount and Overcount Probes on Census Questionnaire for 
the 2008 Dress Rehearsal: 

Figure 4: Timeline for Coverage Follow-up Interviews and Census 
Coverage Measurement Interviews Using Actual Dates for 2000 and 
Estimated Dates for 2010: 

United States Government Accountability Office: 

Washington, DC 20548: 

April 15, 2008: 

The Honorable Henry A. Waxman: 
Chairman: 
Committee on Oversight and Government Reform: 
House of Representatives: 

Dear Mr. Chairman: 

An accurate decennial census relies on finding and counting people-- 
only once--in the right place and collecting complete and correct 
information. The Department of Commerce's U.S. Census Bureau (Bureau) 
faces the daunting challenge of cost effectively counting a population 
that is growing steadily larger, more diverse, and according to the 
Bureau, increasingly difficult to find and reluctant to participate in 
the census. These challenges are made even more difficult by the 
Bureau's April 3, 2008 decision to redesign certain aspects of the 
decennial census. Two basic errors can affect the count of the 
population: omission of persons who should have been counted and 
erroneous enumerations of persons who should not have been counted, 
such as persons counted more than once or persons counted in the wrong 
geographic place. To ensure accuracy, the Bureau, among other 
operations, will conduct a coverage follow-up operation in 2010 to 
determine if additional persons might have been missed (undercounted) 
from already enumerated households, or if persons might have been 
counted in error (overcounted). Further, apart from the enumeration 
process, the Bureau will conduct a census coverage measurement program 
to assess the accuracy of the 2010 Census. 

The census is a critical national effort mandated by the Constitution. 
Census data are used to apportion seats in the Congress, redraw 
congressional districts, allocate billions of dollars in federal 
assistance to state and local governments, and for numerous other 
public and private sector purposes. The Bureau estimates that it has 
generally undercounted the total population in prior censuses. For the 
first time, in 2000, although estimates showed that it undercounted 
certain population groups, the Bureau estimates that it somewhat 
overcounted the total population. For 2010, the Bureau plans to improve 
the accuracy of its population count by, among other things, expanding 
its coverage follow-up operation. Further, the Bureau and others 
questioned the accuracy of the data obtained from the Bureau's coverage 
measurement program in 2000. As a result, for 2010, the Bureau plans to 
revise its methodology for determining coverage error--the extent to 
which persons are under-or overcounted in the enumeration process. As 
agreed, we determined (1) how the Bureau plans to improve coverage 
through its coverage follow-up operation; (2) how the Bureau's census 
coverage measurement plans for assessing the accuracy of the 2010 
Census compare with efforts in 2000 and what the potential impact will 
be of those changes, including the Bureau's plans for using its 
measurement program results to adjust population counts; and (3) how 
the Bureau plans to evaluate the coverage follow-up and measurement 
efforts after the 2010 Census. 

To meet these objectives, we reviewed information about the Bureau's 
coverage follow-up and coverage measurement plans and interviewed 
Bureau officials. We reviewed relevant evaluations from Census 2000 and 
studies conducted by the Bureau and others, including the National 
Academy of Sciences. Further, we interviewed 15 experts about the 
Bureau's coverage measurement methodology and plans. We identified 
these experts based on published reports as well as recommendations 
from the Bureau and the National Academy of Sciences. (For a list of 
experts we interviewed, see app. I.) We conducted this performance 
audit from January 2007 to March 2008 in accordance with generally 
accepted government auditing standards. Those standards require that we 
plan and perform the audit to obtain sufficient, appropriate evidence 
to provide a reasonable basis for our findings and conclusions based on 
our audit objectives. We believe that the evidence obtained provides a 
reasonable basis for our findings and conclusions based on our audit 
objectives. 

Results in Brief: 

The Bureau plans to improve coverage by expanding the techniques it 
uses to identify and select cases for follow-up. However, it is not 
clear on what basis the Bureau will decide which techniques to use to 
identify potential cases, and which of these cases it will select or 
prioritize for coverage follow-up. As in Census 2000, coverage follow- 
up is intended to resolve count discrepancies (when the number of 
household members reported on a completed questionnaire fails to match 
the number of persons for whom information is collected) and complete 
the enumeration of large households (more than six persons living in a 
housing unit). The Bureau plans to include all such count discrepancy 
cases and all large households in its coverage follow-up universe in 
2010. To improve coverage for 2010, the Bureau researched new ways to 
identify a more comprehensive set of cases for follow-up. Specifically, 
to increase the number of ways a person or household could be selected 
for follow-up, the Bureau tested three new coverage follow-up 
techniques: (1) using coverage probes on the census form mailed to 
households and during the nonresponse follow-up interviews to identify 
instances where persons may be under-or overcounted; (2) using 
administrative records to identify households that include persons who 
may not have been counted; and (3) using computer matching to determine 
potential duplicate persons. The Bureau is uncertain how many cases for 
follow-up are likely to be found based on these new techniques. 
Nonetheless, the number of follow-up cases could be very large and very 
costly to resolve. Based on its experience with the 2008 Dress 
Rehearsal, the Bureau expects to decide what techniques it will use for 
identifying and selecting cases during the 2010 Census coverage follow- 
up operation and how many cases it can afford to select for follow-up. 
Bureau officials explained that they intend to consider cost and the 
percentage of cases resulting in a change relative to the cost--known 
as yield--but the Bureau has not fully specified how it will make 
decisions about expanding the coverage follow-up universe or how it 
will select or prioritize cases from those identified by the new 
techniques. Using results from the 2006 Census Test, the Bureau has 
already decided that coverage follow-up will only be conducted by 
telephone rather than personal visits because the yield was higher for 
telephone follow-up. This decision, however, does not necessarily 
reflect trade-offs in the limitations of each technique used to select 
cases for follow-up. For example, coverage may not be improved for 
groups for which the Bureau does not have telephone numbers--the very 
wealthy or the very poor. Given that the Bureau expects to use the 
dress rehearsal to learn more about the expanded selection techniques, 
and that the dress rehearsal has been delayed by a month, the Bureau 
will have little time to complete its final plans for coverage follow- 
up for the 2010 Census. Because of the potential for the coverage 
follow-up workload to expand, it will be important for the Bureau to be 
clear on how it will decide which techniques to use and cases to select 
to ensure that coverage follow-up may best improve coverage. 

Although the Bureau's plan for coverage measurement in 2010 is similar 
in many ways to its measurement design for 2000 and previous coverage 
evaluation programs, its plan differs in important ways, including its 
overall objectives, timing, statistical techniques used, and use of 
computer matching. Nonetheless, much of what the Bureau will do is yet 
to be decided, including most notably, when the Bureau will produce 
estimates of net coverage error and components of coverage error, and 
how it will meet one of its objectives for coverage measurement in 
2010--improving future census operations. One important aspect of the 
Bureau's plans for 2010--specification of the logistic regression 
model--cannot be known until after the Bureau collects the 2010 
coverage measurement data. Similarities between 2000 and 2010 include 
the Bureau's sample design for coverage measurement[Footnote 1] and 
that the Bureau once again intends to use a dual systems approach to 
produce net coverage error estimates--or the difference between the 
coverage measurement program's estimate of the total population and the 
decennial count of the total population.[Footnote 2] However, several 
of the Bureau's objectives for the 2010 coverage measurement program 
differ. For 2010, the Bureau places greater focus on improving future 
decennials. In contrast, for 2000, the Bureau designed coverage 
measurement to consider adjusting the count of the nation's population. 
The Bureau does not plan to use coverage measurement results to adjust 
the 2010 count of the nation's population because it believes that with 
a sufficiently correct count, adjustment would introduce as much or 
more error than it was designed to correct. The Bureau's measurement 
program in 2010 differs in other significant ways from that in 2000, 
for example, (1) starting the coverage measurement interview about 4 
months later, to permit expansion of the Bureau's efforts to conduct 
the coverage follow-up operation;[Footnote 3] (2) estimating net 
coverage error using logistic regression, which is a standard 
statistical modeling technique; (3) estimating the components of net 
coverage error to better understand the details of erroneous 
enumerations and omissions; and (4) expanding computer matching and 
clerical matching to match persons in the coverage measurement 
program's postenumeration survey to the entire census to resolve or 
identify duplicates.[Footnote 4] Bureau officials said that the 
coverage measurement interview will begin later because they expect to 
expand the coverage follow-up operation, and that to maintain 
independence and minimize possible contamination between enumeration 
and measurement, the Bureau did not want to overlap coverage follow-up 
and coverage measurement. Experts we interviewed were generally 
concerned about the contamination issue but also viewed the time delay 
as a very significant challenge because respondent recall[Footnote 5] 
is vital to conducting the measurement program. The Bureau has 
simulated the use of its new method of estimating net error--logistic 
regression--by using data from 2000 and found it to work. The Bureau 
will be unable to fully specify the model it will ultimately use until 
after the data collection phase of its coverage measurement program. 
One aspect of its plans that the Bureau should be able to specify now 
is when it will produce net coverage error estimates and components of 
coverage error estimates for 2010. Further, though the Bureau has 
preliminary plans--using data table shells--to associate components of 
coverage error to census operations, the Bureau has not clearly 
explained how it plans to meet its objective of improving future 
decennials. For the most part, experts we interviewed, after they 
reviewed our summary of the Bureau's coverage measurement plans, stated 
that the Bureau had improved its research model and had taken 
significant and constructive steps in attempting to address key issues 
for coverage error, but several experts noted that more detailed 
planning is needed. Because of the Bureau's changes to its coverage 
measurement program, it is important for the Bureau to make its plans 
clear and share them with stakeholders to ensure that there is an 
agreement on what the program will produce and when the Bureau will 
produce its measurement results. 

The Bureau's plans for evaluating its coverage follow-up operation and 
census coverage measurement program for the 2010 Census are in early 
stages of development. In terms of evaluation, the Bureau suggested 
research on various aspects of coverage follow-up and measurement and 
contracted with the National Academy of Sciences to recommend proposed 
research for the 2010 Census. The academy's final report is due by 
September 2009, but an interim report was issued December 2007. The 
interim report contained recommendations for the Bureau's program of 
evaluations and experiments for the 2010 Census. Bureau officials have 
no timeline yet for making decisions about the evaluations of the 
Bureau's coverage follow-up operation or census coverage measurement 
program following the 2010 Census, and they noted that to determine 
which recommendations to implement, the Bureau will consider costs and 
staffing needs. 

We are recommending that the Secretary of Commerce direct the Bureau to 
provide Congress and other census stakeholders with more specific plans 
on conducting coverage follow-up and census coverage measurement and 
evaluating the results of those programs for the 2010 Census. To 
improve program operations, we recommend that the Bureau specify the 
criteria it will use to assess the techniques for identifying cases for 
coverage follow-up in the 2008 Dress Rehearsal and the timeline it 
plans to follow that will ensure that the results of the assessment may 
be used in planning for 2010. To enhance transparency and oversight for 
improving coverage measurement for the 2010 Census, we recommend that 
the Bureau describe when it will provide estimates of net coverage 
error and components of coverage error and how it plans to relate 
components of coverage error to census operations in order to improve 
future decennials. Finally, we recommend that the Bureau provide key 
decision points and plans for evaluating aspects of the 2010 Census 
coverage follow-up operation and census coverage measurement program. 

The Secretary of Commerce provided written comments on a draft of this 
report. The comments are reprinted in their entirety in appendix II and 
we have incorporated in the report, as appropriate, the Department of 
Commerce's technical comments. Commerce had no substantive 
disagreements with our recommendations and stated that it will develop 
formal action plans for each recommendation. While Commerce believes a 
timeline for producing estimates of net coverage error and components 
of coverage error is certainly a reasonable expectation, Commerce noted 
that it is unclear as to how it can at this time provide any specifics 
on how coverage results might improve future censuses before the 
coverage data are obtained and analyzed. We recognize that the Bureau 
has developed table shells that associate components of coverage error 
to census operations, but these table shells and plans for how the 
Bureau will conduct its analyses have not been shared with the Congress 
and other stakeholders. In addition, it is not clear from current 
Bureau plans how it will go about obtaining the data required to 
complete the tables. 

Background: 

Historically, the census has been affected by undercounts--those who 
were missed in the decennial count. Further, certain groups may be 
undercounted more than others--such as babies, minorities, and renters 
who move often. Seeking to obtain an accurate count has been a concern 
since the first census in 1790. Concern about undercounting the 
population continued through the decades. In the 1940s, demographers 
began to obtain a more thorough understanding of the scope and nature 
of the undercount. For example, the selective service registration of 
October 1940 showed 2.8 percent more men than the census count. 
According to the Bureau, operations and programs designed to improve 
coverage have resulted in the total undercount declining in all but one 
decade since the 1940s. These measures of coverage are based on 
demographic analysis, which compares the census count to birth and 
death certificates and other administrative data (see fig. 1). 

Figure 1: Decennial Census Population Net Undercount Rates from the 
Bureau's Demographic Analysis in Percentages, 1940-2000: 

This figure is a versicle bar graph showing the decennial census 
population net undercount rates from the bureau's demographic analysis 
in percentages, 1940-2000. The X axis is the year, and the Y axis is 
the percentage. 

Year: 1940; 
Percentage: 5.4%. 

Year: 1950; 
Percentage: 4.1%. 

Year: 1960; 
Percentage: 3.1%. 

Year: 1970; 
Percentage: 2.7%. 

Year: 1980; 
Percentage: 1.2%. 

Year: 1990; 
Percentage: 1.6%. 

Year: 2000; 
Percentage: 0.1%. 

[See PDF for image] 

Source: U.S. Census Bureau. 

[End of figure] 

In contrast to the long-standing use of demographic analysis to 
estimate census undercount, modern coverage measurement began with the 
1980 Census, when the Bureau compared decennial figures to the results 
of an independent sample survey of the population. In using statistical 
methods such as these, the Bureau began to generate detailed measures 
of the differences among undercounts of particular ethnic, racial, and 
other groups. In 1990, the Bureau relied on its Post-Enumeration Survey 
to verify the data it collected through the 1990 Census. For this 
effort, the Bureau interviewed a sample of households several months 
after the 1990 Census, and compared the results to census 
questionnaires to determine if each sampled person was correctly 
counted, missed, or double counted in the census. Using the definition 
of those missed minus those double counted, the Bureau estimated 
whether there was a net undercount or net overcount. For the 1990 
Census, the Bureau estimated a net undercount of about 4 million 
people. During Census 2000, for the first time, the Bureau's coverage 
measurement program measured a net population overcount. To estimate 
the accuracy of the 2000 Census, the Bureau conducted its Accuracy and 
Coverage Evaluation, expecting it could be used to adjust the Census 
2000 results for all nonapportionment purposes if it improved the 
census data. The original March 2001 estimates of the Accuracy and 
Coverage Evaluation, which was an independent sample survey designed to 
estimate the number of people who were under-and overcounted in the 
census, a problem that the Bureau refers to as coverage error, became 
available in time to correct census files. However, in a revised March 
2003 evaluation, the Bureau was unable to conclude that the adjusted 
data were more accurate and determined that the 2000 Census tabulations 
would not be adjusted for any purpose. The decision not to adjust was 
consistent with decisions made during the 1990 decennial, when the 
Department of Commerce decided, because of other problems, not to use 
the postenumeration survey to adjust the decennial count. 

During this decade, the Bureau has conducted a testing and evaluation 
program to assess certain activities that it believes can improve the 
accuracy of the census. In preparing for the 2010 Census, the Bureau 
has conducted two field tests--in 2004, the Bureau tested in the Queens 
Borough of New York City and southwest Georgia and, in 2006, tested at 
the Cheyenne River Reservation in South Dakota and parts of Austin, 
Texas, in Travis County. In 2003 and 2005, the Bureau also conducted 
national surveys to test various aspects of changes planned for 2010. 
We reported earlier about the importance of these earlier assessments 
leading to a design that would be sufficiently mature so that the dress 
rehearsal for the 2010 Census demonstrates the feasibility of the 
various operations and technologies planned for the decennial under 
conditions that are as close as possible to the actual census.[Footnote 
6] The Bureau is currently conducting its dress rehearsal, one of the 
final planning activities for the decennial. Census Day for the 2008 
Dress Rehearsal activities in sites located in North Carolina and 
California will be May 1, 2008, a delay of 1 month from the Bureau's 
initial plan. 

The first 2010 Census operation, building an accurate address list, 
overlaps with dress rehearsal activities and has also already 
begun.[Footnote 7] For the 2010 Census, the majority of the housing 
units in the country will receive paper census questionnaires delivered 
either by mail or by census field workers before April 1, 2010. Those 
households that do not return their questionnaires will be contacted by 
census field workers during the nonresponse follow-up operation to 
determine the number of people living in the housing unit on Census Day 
and other information. The Bureau also uses other techniques for 
counting persons. For example, people in group quarters (such as 
college dormitories, homeless shelters, and nursing homes) will be 
enumerated in separate operations at the facilities by enumerators who 
will then leave forms listing the names of the people living or staying 
there and then return to pick up the completed forms and, if necessary, 
conduct follow-up interviews. 

For the 2010 Census, the Bureau has tested and planned revisions to the 
census form with the goal of improving the census' coverage. The 
revisions are intended to improve methods for ensuring that persons are 
counted in the correct place and only once.[Footnote 8] These changes 
include revising residency rules, simplifying the instructions provided 
to respondents on residency, and including questions--called coverage 
probes--about persons who may be missing or incorrectly included in the 
response. For Census 2000, the Bureau relied on 31 residence rules to 
instruct respondents on who to include on the mail-back form. The 
Bureau revised these 31 rules to 1 rule with 13 broad living situations 
for 2010. For 2010, the Bureau is also planning on using two coverage 
probes, one asking about potentially missed people (undercounted) and 
one asking about potentially duplicated people (overcounted), on the 
census form. Although the Bureau used coverage probes in the 
nonresponse follow-up operation for 2000, it did not include probes in 
the 2000 Census mailed questionnaires as is now planned for 2010. The 
Bureau intends to use answers to these coverage probes and other 
techniques to determine if an enumeration is likely to be incorrect or 
incomplete and, therefore, if cases should be selected for follow-up. 
The Bureau uses the results of the coverage follow-up operation to 
correct the results of the initial enumeration. Coverage follow-up 
interviews with respondents determine if changes should be made to 
their household rosters as reported on their initial census returns. 
Specifically, these results lead to a correction of census information 
and counts, as coverage follow-up results are combined with the initial 
enumeration and are used as the official enumeration of the household. 

Bureau Plans to Expand Coverage Follow-up to Improve Census 2010 but 
Has Not Finalized Plans for Identifying Follow-up Cases: 

For Census 2010, the Bureau is planning to improve coverage by 
expanding its coverage follow-up operation to obtain more accurate 
enumeration data. To prepare for this, the Bureau has conducted tests 
of an expanded follow-up operation. As in Census 2000, coverage follow- 
up is intended to resolve count discrepancies and complete the 
enumeration of large households. For 2010, the Bureau also plans to use 
additional techniques to increase the number of ways a person or 
household could be selected for follow-up. However, the Bureau has not 
finalized plans for how to identify and select cases from the expanded 
follow-up universe. The expansion of selection criteria for coverage 
follow-up has the potential to dramatically increase the workload for 
this operation and, as a result, increase costs. While the Bureau plans 
to budget for coverage follow-up in order to achieve the best results 
or highest yield of changes to household rosters, it is unclear on what 
basis the Bureau will assess the additional techniques for identifying 
potential cases or the basis for selecting cases for follow-up using 
the expanded techniques. The Bureau plans to assess its experiences 
with coverage follow-up, including important interfaces within the 
Bureau and between the Bureau and its contractor, during the 2008 Dress 
Rehearsal in making further decisions on follow-up for 2010. For 
example, three separate groups within the Bureau have responsibility 
for selecting follow-up cases, but one of those groups eliminates 
overlap in cases before sending them to the contractor for telephone 
follow-up. Recent decisions to delay the dress rehearsal, including 
follow-up activities, increase the risk that the results of additional 
testing of these techniques in 2008 will not be sufficiently timely or 
complete to improve operations for 2010, and the Bureau has not stated 
when it will complete its assessment of coverage follow-up for the 2008 
Dress Rehearsal. 

Bureau Plans to Expand Techniques for Identifying Coverage Follow-up 
Cases: 

The Bureau is planning to improve coverage by expanding its coverage 
follow-up operation. To meet this objective, the Bureau expects to rely 
on previously used techniques as well as techniques it is now 
developing to identify cases for follow-up. As in Census 2000, coverage 
follow-up for 2010 will include: 

* resolving count discrepancies, when the number of household members 
reported on a completed questionnaire fails to match the number of 
persons for whom information is collected, and: 

* completing the enumeration of large households, in which there are 
more than six persons living in a housing unit. 

For 2010, the Bureau also plans to use additional techniques for 
selecting a person or household for follow-up. The techniques planned 
include: 

* using questions--called coverage probes--on the census form itself 
and in the nonresponse follow-up interviews to identify where persons 
may be omitted or counted more than once, 

* using administrative records to identify persons who may be omitted, 
and: 

* using computer matching from the universe of census returns to 
identify persons who may have been counted more than once. 

Although the Bureau has not finalized coverage follow-up plans for the 
2010 Census, in June 2007, the Bureau developed a work flow and 
narrative for the 2008 Dress Rehearsal coverage follow-up operation. 
This work flow shows the process for collecting enumeration data, 
selecting coverage follow-up cases, and conducting follow-up (see fig. 
2). Once the Bureau selects a case for follow-up, it plans to work 
through its Decennial Response Integration System (DRIS) contractor, 
which will conduct coverage follow-up interviews by telephone. The 
telephone interviews determine if changes should be made to the 
household rosters as reported on the initial census returns. The 
questions in the follow-up interview are designed to identify if 
persons were omitted or counted in error because they should have been 
counted at a different address or not at all--for example, because a 
person had been born after or died before Census Day. 

Figure 2: Operation Flowchart for the Bureau's 2008 Dress Rehearsal 
Coverage Follow-up Operation: 

This figure is a an operation flowchart showing the bureau's 2008 dress 
rehearsal coverage follow-up operation. 

[See PDF for image] 

Source: GAO analysis of U.S. Census Bureau data. 

[End of figure] 

As shown in figure 2, enumeration data will be captured by DRIS. Also, 
in selecting cases for follow-up, the Bureau will identify any overlap 
in cases represented by large households, count discrepancies, coverage 
probes, administrative records review, and computer matching. While the 
Bureau has tested new techniques for coverage follow-up, it has not yet 
tested these steps with the new processing interfaces within the 
Bureau--Decennial Systems and Processing Office, Decennial Statistical 
Studies Division, and Data Integration Division--and between the Bureau 
and DRIS. During the 2008 Dress Rehearsal, from May 27 to August 9, 
2008, the Bureau will not only be assessing various aspects of the 
expanded follow-up, but it will also be testing for the first time the 
DRIS integration of data as well as the Bureau's process for 
identifying overlap cases. After the Bureau selects cases for follow- 
up, the DRIS contractor assigns an identification number to each case 
and oversees telephone interviews. Each case may receive up to seven 
follow-up telephone calls, and the DRIS contract includes such aspects 
as on-demand automated call distribution, call routing, reporting, and 
quality assurance monitoring. 

Resolving Count Discrepancies: 

For 2010, as in Census 2000, all count discrepancy cases identified on 
mailed paper census forms will be part of the coverage follow-up 
operation. The Bureau will check completed questionnaires for 
discrepancies between the number of persons reported as members of a 
household and the number of persons for whom census information was 
provided on the census forms. For example, questionnaires that fail 
edit checks might include cases where the respondent may indicate that 
five people were living in the housing unit on Census Day, but there is 
information for only two people. 

Enumerating Large Households: 

As in 2000, the Bureau is also aiming to improve coverage of people in 
housing units by completing the enumeration of large households, 
meaning those that indicate on their census forms that there are more 
than six persons in the housing unit and for which there was not enough 
room on the mailed questionnaire form for self-response. The 
questionnaire has space for only six household members to be fully 
listed. The nonresponse follow-up interview is not limited in the 
number of people that can be reported and all household members can be 
enumerated during nonresponse follow-up; therefore, no large household 
cases will need to be enumerated during the coverage follow-up 
operation as a result of the nonresponse follow-up interview. 

Using Coverage Probes to Identify Potential Under-or Overcounting: 

The Bureau plans to expand its universe for coverage follow-up in 2010 
by including cases resulting from the use of coverage probes. For the 
dress rehearsal, the Bureau plans to contact anyone who affirmatively 
answers the undercount question (which is at the household level) and 
the overcount question (which is at the person level) (see fig. 3). If 
either or both coverage probes prove unproductive in the dress 
rehearsal, Bureau officials indicated that they might be eliminated 
from the follow-up operation. The coverage probes are included on the 
mail-back questionnaire and among the questions asked during the 
nonresponse follow-up. During nonresponse follow-up, for example, if a 
person responds in the affirmative to the coverage undercount probe, 
the interviewer collects the name and demographic information for the 
added person. All names of individuals added to the household 
roster[Footnote 9] through the undercount probe during nonresponse 
follow-up will be sent to the coverage follow-up operation for 
additional follow-up. 

Figure 3: Undercount and Overcount Probes on Census Questionnaire for 
the 2008 Dress Rehearsal: 

This figure is a copy of undercount and overcount probes on census 
questionnaire for the 2008 dress rehearsal. 

[See PDF for image] 

Source: GAO presentation of U.S. Census Bureau information. 

[End of figure] 

As shown in figure 3, for the 2008 Dress Rehearsal, the undercount 
probe asks if there are any additional people staying at the residence 
on Census Day that were not included earlier--children, such as newborn 
babies or foster children; relatives, such as adult children, cousins, 
or in-laws; nonrelatives, such as roommates or live-in baby sitters; or 
people staying at the residence temporarily. The overcount probe for 
the 2008 Dress Rehearsal asks if persons sometimes lived or stayed 
elsewhere--such as while attending college, while in the military, 
while staying at a seasonal or second residence, during a child custody 
arrangement, while in jail or prison, while in a nursing home, or for 
another reason. The overcount probe is asked of the first six household 
members on the mail-back questionnaire and of all persons in the 
nonresponse follow-up interview. 

Evaluations of the coverage follow-up operation for the Bureau's 2004, 
2005, and 2006 tests examined the performance of both the undercount 
and overcount probes. In the Bureau's 2004 test, the overcount probe 
was found to be more effective in finding households with erroneous 
enumerations than the undercount probe was in finding persons omitted 
from the count. In this test several problems were identified with the 
undercount probe, most notably a low rate of added residents to the 
household roster. Results from the test showed that of the 3,919 
respondents at the Georgia test site who answered the undercount probe 
affirmatively, only 53 persons were added to the household roster. The 
Bureau noted that the low rate of added persons was "perplexing." Other 
problems included difficulties with the interviewer reading the probe 
as worded and indications that use of the probe may have actually 
increased the number of count discrepancy cases. 

In 2005, the Bureau tested two different versions of both the 
undercount and overcount probes, and changes to both probes were 
adopted for the 2006 Census Test. During the 2006 test, the Bureau 
found that the overcount probe "performed as expected" and generated a 
higher rate of deleted persons from the census test roster for both the 
mail-back questionnaire and nonresponse follow-up operation than other 
new techniques tested to identify coverage follow-up cases. Although 
the Bureau designed the undercount probe to add persons, the undercount 
probe resulted in more deleted persons than added persons.[Footnote 10] 
The Bureau noted that over half of the deletions stemmed from cases in 
which the respondent either answered "yes" to both coverage probes or 
the respondent answered "yes" to the undercount probe and had a count 
discrepancy. Indeed, in a separate analysis that examined the rate of 
overlap among coverage follow-up cases--meaning households were flagged 
for coverage follow-up through more than one technique--the Bureau 
found that the highest significant rate of overlap was among cases 
where the respondent answered "yes" to both coverage probes. 

Additionally, there were some operational difficulties using the 
undercount probe during the nonresponse follow-up operation in the 2006 
Census Test.[Footnote 11] For example, the Bureau found that although 
persons answered the undercount probe affirmatively (1,234 households) 
during nonresponse follow-up, very few persons (8) were added to the 
coverage follow-up roster.[Footnote 12] The Bureau referred to this 
number as "surprisingly low," because if a household replies "yes" to 
the undercount probe, the Bureau expected to see a person added to the 
roster for coverage follow-up. In its evaluation of coverage follow-up 
during the 2006 Census Test, the Bureau speculated that there could 
have been operational error during nonresponse follow-up that led to 
this discrepancy. Furthermore, of those 8 persons who were added and 
appeared in coverage follow-up, only 3 could be confirmed as residents 
during follow-up, leading the Bureau to conclude that the undercount 
question added persons incorrectly more than half of the time after 
nonresponse follow-up. Although the Bureau has made some slight 
modifications to the response options for the overcount probe it plans 
to use in the dress rehearsal, it plans to use the same undercount 
probe for the dress rehearsal as it did for the 2006 test. 

Using Administrative Records to Identify Potentially Omitted Persons: 

For the dress rehearsal, the Bureau plans one wave of matching with 
administrative records to identify duplicate or suspected incorrect 
data. For administrative matching, the Bureau will use the Enhanced 
Statistical Administrative Records System, which is a Bureau-wide 
resource that collects certain data from the Internal Revenue Service, 
the Social Security Administration,[Footnote 13] the Selective Service, 
the Department of Housing and Urban Development, as well as other 
federal agencies. The Bureau will match census returns against records 
from these sources using various algorithms to associate persons with 
housing units to identify households with potential 
undercounting.[Footnote 14] The Bureau plans to match names on census 
questionnaires with names in the administrative database--if a 
household is found to have more persons in the administrative records 
than on the census form, it would be a candidate for coverage follow- 
up. The Bureau first tested selecting cases for follow-up through 
comparison with administrative records in 2005. For the 2006 Census 
Test, all of the cases with matches from the administrative records 
search had personal visits; however, Bureau officials noted that 
personal visits were conducted because staff had sufficient time, given 
the smaller geographic area of a test site. Personal visits will not be 
part of the national 2010 coverage follow-up operation. The 2008 Dress 
Rehearsal marks the first time that cases selected through 
administrative record matching will be tested in the coverage follow-up 
telephone operation. 

Using Computer Matching to Identify Persons Who May Have Been 
Duplicated: 

For the dress rehearsal, the Bureau will conduct several waves of 
computer matching for its unduplication process. In 2000, the Bureau 
had difficulty with duplicate enumerations and did not have a planned 
operation for dealing with the problem; officials explained that in 
2000 there was an "ad hoc" plan that they developed "on the fly" for 
identifying duplicates. However, Bureau officials noted that one of the 
lessons learned from 2000 was that the Bureau needed to expand the 
search for duplicates by computer matching. Thus, to improve the 2010 
Census, along with the overcount probe specifically designed to try and 
find duplicates, the Bureau will also match census returns against the 
universe of all other census returns to identify duplicates nationwide; 
for the dress rehearsal, the matching will be conducted within the test 
site, rather than on a national scale. While the Bureau has been 
evaluating and testing this procedure, the limited likelihood of 
duplication of individuals that will occur within census test sites of 
limited geographic size, as compared to the actual national decennial, 
limits the Bureau's ability to forecast how the nationwide matching and 
unduplication system will affect workload for coverage follow- 
up.[Footnote 15] 

Although the Bureau has not finalized its coverage follow-up 
techniques, it plans to apply a computer-based record linkage system to 
the data collection files during the 2010 Census to match files against 
themselves to find potential duplicates based on housing unit and 
person information. Those households would then be selected for follow- 
up. For example, erroneous enumerations are created when duplicated 
people are counted correctly in one housing unit and incorrectly in 
another housing unit. Duplicated housing units, which occur when 
multiple addresses of the same housing unit are identified during 
census operations, are removed. Further, in contrast to Census 2000, 
the addresses that could be included in the census in both the housing 
unit and group quarter enumerations can no longer cause duplication 
because, according to the Department of Commerce, the Bureau now has 
one list for both housing units and group quarters. Consequently, 
according to Bureau officials, the Bureau faces the challenge of 
identifying the types of duplicates that can be most easily identified 
and addressed, which will result in the most cost-effective yield. 

The Bureau's 2006 test of computer matching found that potential 
duplicates generally were not deleted. Specifically, in its evaluation 
of computer matching during the 2006 Census Test, the Bureau found that 
93 percent of respondents marked as possible duplicates of persons 
counted outside the census block were neither deleted nor linked to a 
deleted match. The Bureau considered this result disappointing. It 
noted that the coverage follow-up interviews did not resolve the cases 
very well because follow-up interviews were not completed for 24 
percent of the cases and, in 61 percent of the cases, respondents did 
not confirm during the follow-up interview that they lived or stayed 
somewhere else. The interviewer cannot provide information about the 
suspected duplicate during the follow-up interview because of concerns 
related to confidentiality; the interview is limited in its potential 
for success in that the respondent has to volunteer information about 
the suspected duplicate case. Consequently, the Bureau is not only 
challenged in identifying suspected duplicate cases, but in actually 
resolving the cases as well. 

Bureau officials explain that for the unduplication process, the Bureau 
will assign a match score and identify links among suspected 
duplicates. The Bureau plans to develop a match score threshold that 
will vary by level of geographic distance between matches. The Bureau 
will then assess the quality of the match based on the match score and 
conduct further research on the cases exceeding this threshold. For 
example, once the Bureau identifies potential duplicates, it determines 
whether the potential duplication is a potential housing-level problem 
(duplicated persons identified within a census block) or person-level 
problem (more long distance-identified duplicates nationally). If it is 
a housing-level problem, then the Bureau would consider using personal 
visits by field staff from another of its operations, the field 
verification operation, to verify housing unit additions. Further, 
according to the Department of Commerce, field verification may be 
undertaken to determine if housing units should be deleted from the 
list. If it is a person-level problem, the Bureau would consider a 
coverage follow-up operation telephone interview to determine if the 
person is counted twice. 

Bureau Will Rely on Assessment of Dress Rehearsal to Complete Coverage 
Follow-up Plans for the 2010 Census: 

The Bureau is developing a number of techniques for selecting cases for 
follow-up and plans to expand the coverage follow-up operation overall. 
However, it has not determined which techniques it will use, what 
thresholds it will use in selecting cases for follow-up, or the 
resources it expects to devote to the coverage follow-up operation. The 
most pressing question remaining is how the Bureau will prioritize its 
expanded universe of follow-up cases identified through coverage 
probes, administrative record matching, and the unduplication process. 
Currently, households that display count discrepancies, are large, 
reply in the affirmative to coverage probes, are flagged through 
comparisons with administrative records, or are flagged as potential 
duplicates are all eligible for follow-up. Although there may be a 
large number of eligible cases for coverage follow-up, the Bureau is 
unsure how many cases it can financially afford for follow-up, and 
discussions are still under way within the Bureau about what units will 
be sent to the coverage follow-up operation. The Bureau is reviewing 
data from tests--such as the add and delete rates within each type of 
case--to aid in determining what types of techniques will be most 
productive in terms of identifying overcounted and missed persons. 

The Bureau plans to rely on additional data from its assessment of the 
coverage follow-up operation in the 2008 Dress Rehearsal in making 
decisions about which techniques, including coverage probes, 
administrative record matching, and the unduplication process, it plans 
to use for identifying and selecting cases for coverage follow-up in 
2010. Dress rehearsal plans for coverage follow-up should mirror the 
plans intended for the decennial because the 2008 Dress Rehearsal will 
be the Bureau's last opportunity to assess the various procedures and 
systems for 2010 under as close to census-like conditions as possible. 
However, the expanded coverage follow-up operation conducted during the 
dress rehearsal will not necessarily be what the Bureau implements in 
2010 because the Bureau plans to use the dress rehearsal to assess key 
aspects of a likely coverage follow-up operation, including (1) the 
potential coverage follow-up workload and how it may be distributed, 
(2) the extent to which the coverage follow-up operation improves 
census coverage through such factors as the percentage of added 
residents and deleted residents, and (3) problems administering the 
follow-up interview and managing work. The assessment will also address 
problems with the operational work flow and interfaces and provide 
recommendations to fix or improve these problems. While the Bureau's 
expectation is that it will have the necessary data from the dress 
rehearsal to inform the final plans for the 2010 coverage follow-up 
operation, Bureau officials have noted that the delayed dress rehearsal 
(a delay in the Census Day from April 1 to May 1, 2008) may make it 
more difficult to incorporate the findings from the dress rehearsal 
into the Bureau's plans for the 2010 Census. 

Bureau officials have stated in several forums that resource 
constraints are a factor in the final decision on the scope of follow- 
up in 2010. The expansion of selection criteria has the potential to 
dramatically increase the workload for coverage follow-up in 2010 and, 
as a result, increase costs. Bureau officials have cautioned that it is 
not necessarily cost-effective for the Bureau to follow up on the 
entire universe eligible to be included in the coverage follow-up 
operation. Thus, the Bureau will be using results--such as the 
percentage of cases resulting in a change relative to the cost or 
yield--from the dress rehearsal to decide which techniques to use and 
how many cases it can afford to include for follow-up. 

Evaluations of the impact of the criteria the Bureau will use in 
judging what techniques to use for identifying cases for follow-up or 
the thresholds for selecting specific cases for follow-up will be 
critical to determining the final plan for the coverage follow-up 
operation. The Bureau has made one judgment about how it will proceed 
for 2010 as a result of its 2006 test. Specifically, in its evaluation 
of the 2006 Census Test, the Bureau calculated that the cost was $155 
per roster change for coverage follow-up cases done by telephone versus 
$766 per roster change for cases done by personal visits. Consequently, 
the Bureau concluded that it should conduct follow-up interviews by 
telephone. This decision, based partly on the percentage of cases with 
a change and partly on cost, represents two criteria. 

Effective program management, including assessment of risks,[Footnote 
16] requires that agencies identify information that is critical to 
achieving important program objectives.[Footnote 17] In this case, the 
Bureau should clearly state the basis for deciding which techniques to 
use or the number of cases it will include for coverage follow-up. 
Because the Bureau has not specified these criteria, it is not clear 
what criteria--other than roster change and cost--the Bureau will 
consider. For example, the Bureau noted in its decision to conduct 
coverage follow-up by telephone that not adding a personal visit 
component to the operation would limit its ability to reduce coverage 
errors for households it could not reach by telephone. Bureau research 
indicated that households for which it did not have telephone numbers 
tended to be very wealthy or very poor. Thus, coverage may not be 
improved for these demographic groups through telephone follow-ups 
alone. 

Bureau Has Made Changes to Coverage Measurement for 2010 Intended to 
Benefit the 2020 Census, but Key Aspects of Program Plan Remain 
Unclear: 

The Bureau's overall design of its census coverage measurement program 
for the 2010 Census is similar in many respects to its measurement 
design for 2000 but differs in some important ways. The objectives, 
timing, and some techniques to be used for 2010 differ from those used 
in 2000; however, the Bureau has not made clear when it will produce 
estimates of net coverage error and components of coverage error, nor 
does it make clear how it will meet one of its objectives for coverage 
measurement in 2010--improving future census operations. In 2000, the 
Bureau focused on estimating net coverage error with the possibility of 
producing adjusted numbers at the census block level. In contrast, for 
2010, while still intending to produce net coverage error estimates at 
higher levels of geography--the national and state levels--the Bureau 
has refocused its efforts in order to use its measurement program to 
help improve enumeration methodologies for the future. Importantly, the 
Bureau does not plan for the measurement program to prepare adjusted 
numbers for the 2010 Census. In addition to starting coverage 
measurement interviews 4 months later than it did in 2000, the Bureau 
has also made several other important changes to its measurement 
program. For example, to better estimate net coverage error, the Bureau 
plans to use logistic regression for the first time. The Bureau also 
plans to assess the association of components of coverage error to 
various census operations. Finally, to better measure coverage, the 
Bureau plans to better identify duplicates by expanding its computer 
and clerical matching. Expert statisticians and methodologists we 
interviewed generally agreed that the Bureau's changes for its 2010 
measurement program are an improvement from the 2000 coverage 
measurement program; however, several experts noted that more detailed 
planning is needed. 

For the 2010 coverage measurement program, the Bureau expects to 
include multiple operations. First, the Bureau selects its sample of 
census block clusters to be included in the coverage measurement 
program. Using the sample, the Bureau collects information on the 
housing units in the census blocks to develop an address list 
independent from that used in the census enumeration. The Bureau then 
uses this address list to draw a sample of housing units to include in 
the person interview operation. During this operation, Bureau field 
staff interview--in person--residents of these housing units and 
collect demographic, Interview Day and Census Day residency information 
of all persons living at the housing unit. According to the Department 
of Commerce, additional information will be collected in 2010 to 
identify alternate addresses where persons on the coverage sample 
roster may have been counted on Census Day and this information will be 
used in person-based computer and clerical matching operations. Next, 
the Bureau compares data collected during the person interviews with 
all census enumerations. Where the Bureau is unable to make a match, 
the Bureau follows up by sending interviewers back to the housing unit 
to reinterview the person or persons in question. During the final 
operation, the Bureau produces estimates of coverage error for persons 
and housing units. For 2010, the Bureau plans to provide estimates of 
net coverage error at the national level by race/ethnicity, age/sex, 
and tenure--the status of an occupied housing unit as either owner 
occupied or renter occupied. The Bureau will also produce estimates of 
net coverage error for states. 

2010 Sample Design for Census Coverage Measurement Is Similar to That 
for 2000: 

The Bureau's 2010 coverage measurement program sample design is similar 
to that used for measurement in 2000. As in 2000, the Bureau plans to 
use a 300,000 national housing unit sample. The methods are basically 
consistent with the 2000 coverage measurement program and are expected 
to yield similar levels of reliability, although slightly different 
methods are being used to determine state sampling allocation. The 
objective for 2010 is to have the sample allocated proportionally 
across states and the District of Columbia, with a minimum of 1,800 
housing units in each state. Moreover, the Bureau is using a 
differential within state sample allocation in order to allow for 
oversampling in areas as needed. Notably, according to Bureau 
officials, it is possible that that the sampling numbers could still 
change depending on budget or operational constraints. Most experts we 
spoke with thought that the Bureau's sample design was appropriate for 
its objectives. 

Census Coverage Measurement Interviews Will Begin Later, and Estimates 
Will Take Longer to Complete in 2010: 

In 2000, the timeline for coverage measurement overlapped with the 
Bureau's coverage follow-up efforts, though it was conducted 
independently from the census enumeration; however, measurement and 
coverage follow-up are planned to be conducted sequentially for the 
2010 Census. Specifically, in 2010, the census coverage measurement 
interview will begin after the coverage follow-up operation is 
completed (see fig. 4). According to the Bureau's 2010 Census 
Operations and Systems Plan, dated August 31, 2007, the Bureau expects 
to conduct the person interviews for coverage measurement from August 
20 through October 16, 2010. In 2000, the Bureau conducted the coverage 
measurement person interviews from April 24 through September 11, 2000. 

Figure 4: Timeline for Coverage Follow-up Interviews and Census 
Coverage Measurement Interviews Using Actual Dates for 2000 and 
Estimated Dates for 2010: 

This figure is a timeline for coverage follow-up interviews and census 
coverage measurement interviews using actual dates for 2000 and 
estimated dates for 2010. 

[See PDF for image] 

Source: GAO analysis of U.S. Census Bureau data. 

[End of figure] 

According to the Bureau, the expansion of the coverage follow-up 
operation contributed to delaying the measurement program in order to 
maintain data independence and reduce the likelihood of data 
contamination resulting from the overlap of follow-up and measurement 
field operations.[Footnote 18] In past coverage measurement programs, 
the person interview phase overlapped with coverage follow-up conducted 
as part of the census enumeration. In 2010, the Bureau plans to delay 
the start of the person interview phase for coverage measurement to 
minimize the risk of data contamination. While many experts agreed that 
delaying the coverage measurement interviews will reduce the risk of 
data contamination, some experts also highlighted that the delay may 
also negatively affect the data quality because it could contribute to 
recall bias. Recall bias, as it pertains to coverage measurement, means 
that a person incorrectly recalls where he or she or other household 
members resided on Census Day and thus biases the coverage measurement 
data. Two experts also noted that delaying coverage measurement 
interviews could also affect quality of data on movers. Generally, 
experts said that decreasing the amount of time between the coverage 
follow-up interview and the coverage measurement interview would 
benefit the coverage measurement data. However, a few experts who were 
also familiar with the issue stated that they were unsure of the 
magnitude of contamination and, therefore, did not know if the Bureau 
should decrease the time. Several of these experts argued that the 
Bureau could make a better case by using results from its contamination 
studies to determine if the delay is warranted. In 2006, the Bureau 
conducted a contamination study using the results of the 2006 Census 
Test to determine if the coverage measurement person interviews caused 
a difference in responses to the coverage follow-up 
interviews.[Footnote 19] Although the Bureau stated that the number of 
sample cases involved in the study was small and that the result should 
be viewed with caution, it concluded that there might be some small 
impact on coverage follow-up results if the measurement interview is 
conducted at the same time as the coverage follow-up interview. 

For 2010, the Bureau also expects to complete its coverage error 
estimates later than was the case for the Bureau's initial 2000 
estimates. In 2000, the Bureau produced initial estimates in March 
2001; however, after evaluating the 2001 estimates, the Bureau released 
revised estimates in March 2003. According to Bureau officials, the 
Bureau has preliminary plans to complete coverage measurement 
estimation[Footnote 20] for the 2010 Census sometime after October 
2011[Footnote 21]--6 months after the Bureau provides population counts 
for redistricting. Bureau officials have stated that the decision to 
delay measurement estimation will allow for more time to adequately 
evaluate the 2010 measurement data and estimates, noting that the 
Bureau did not have sufficient time for analysis in 2000. 

Objectives for the 2010 Census Coverage Measurement Differ from Those 
for 2000, but How the Bureau Will Meet New Objectives Is Unclear: 

The Bureau's 2010 coverage measurement objectives have changed from 
those of Census 2000. In 2000, the Bureau focused on estimating net 
coverage error with the possibility of adjusting the census count at 
the census block level of geography. Though the Bureau still intends to 
produce net error estimates for 2010--at the state level and higher--it 
has refocused the coverage measurement program on improving future 
enumeration methodologies; consequently, it does not plan to use the 
results of its coverage measurement program to improve or adjust the 
count of the nation's population in 2010. For 2010, the Bureau's goals 
for measuring coverage accuracy include the following aspects: 

* The Bureau plans to use a dual systems approach to estimate net 
coverage error by using logistic regression instead of the 
poststratification technique used in 2000. The Bureau believes its move 
to logistic regression will also allow more flexibility because it can 
incorporate a greater number and type of factors or predictors in the 
model. 

* The Bureau plans to improve future census operations by associating 
estimates of components of coverage error, including erroneous 
enumerations and omissions, to census operations. 

* To better resolve and identify duplicate enumerations, a major source 
of error in previous decennials and coverage measurement programs, the 
Bureau plans to expand computer and clerical matching to match persons 
in its postenumeration survey to the entire census from the more 
limited matching done in 2000. 

The Bureau plans to conduct a census coverage measurement program for 
the 2010 Census, but Bureau officials said that they did not know when 
the Bureau would produce detailed documentation describing its plans. 
The Bureau has developed preliminary plans for associating components 
of coverage error to census operations, but these plans also are not 
finalized. 

Bureau Does Not Plan to Adjust the 2010 Census in Time for 
Redistricting: 

According to the Bureau, it has no plans to use the results of its 
coverage measurement program to improve or adjust the 2010 Census 
count. In response to a May 12, 2003 letter from Representative William 
Lacy Clay asking about decennial plans, the Bureau explained that with 
a sufficiently correct count, adjustment would introduce as much or 
more error than it was designed to correct. If the Bureau were asked by 
the Congress to provide adjusted numbers, Bureau officials commented 
that it would be unable to quickly or easily do so because the current 
coverage measurement program's timeline produces estimates after 
October 2011, and the Bureau has not designed the software to produce 
estimates at lower geographic levels. Consequently, according to the 
Bureau, it cannot prepare adjusted numbers in time to meet the legal 
deadline for producing count data for redistricting, which is April 1, 
2011. Furthermore, the Bureau is not planning to provide net error 
estimates below the state level; such estimates are necessary for 
redistricting. 

Some of the interviewed experts agreed that producing adjusted numbers 
would be challenging; however, several also said that the Bureau could 
still provide some adjusted numbers if asked to do so. Regarding the 
challenges, two experts stated that adjusted numbers could be produced 
at a higher geographic level, the national level, but not accurately at 
lower geographic levels, the state and local levels, without 
appropriate changes to the current design and methodology. For example, 
one expert explained that the Bureau would need to use 
poststratification to produce estimates at the state and local levels. 
Additionally, several experts noted that the 2010 coverage measurement 
timeline poses a challenge for producing adjusted numbers by the April 
1, 2011, redistricting deadline. 

Bureau Plans to Use a Different Statistical Method for Estimating Net 
Error but Has Not Fully Specified Its Plans: 

The Bureau plans to use logistic regression to improve its estimation 
of net coverage error. Of the experts familiar with logistic 
regression, the majority thinks that the methodology is an improvement; 
however, a few experts question how the Bureau will explain a more 
complex method to its stakeholders. Previously, for the 2000 coverage 
measurement, the Bureau used a poststratification[Footnote 22] method. 
However, Bureau officials said that poststratification was limited 
because of the small sample sizes of many of the strata. They explained 
that logistic regression permits greater flexibility in dealing with 
small sample sizes because it allows for a greater number and type of 
factors or predictors in the model.[Footnote 23] By definition, 
poststratification is limited because it only allows for a small number 
of factors with defined categories to be included. In addition to 
greater flexibility, logistic regression can potentially reduce errors 
in estimation. According to Bureau officials, to verify estimates from 
logistic regression, the Bureau is also considering using some 
poststratification. Logistic regression provides greater flexibility in 
dealing with small samples, but it may be more difficult 
methodologically to use logistic regression to produce state-level 
estimates by race/ethnicity, age/sex, and tenure or to estimate lower 
than the state level. Some experts argued for continuing to produce 
estimates using poststratification to enable direct comparisons of 
error estimates to prior decennials and to verify that logistic 
regression is working. 

To enhance transparency and oversight for improving coverage 
measurement for the 2010 Census, the Bureau should provide the Congress 
with timely and accurate information to allow monitoring of agency 
activities--specifically, timely and accurate information about its 
plans for coverage measurement.[Footnote 24] The Bureau plans to refine 
its estimation methods for 2010 by using logistic regression to 
estimate net coverage error, but it does not have an operational or 
working plan that outlines what resources--such as the amount of time 
that it will take to run the data set(s)--will be used to complete this 
operation. Further, despite conducting some exploratory logistic 
regression work using the 2006 test data, as well as simulating its 
estimation model for logistic regression using decennial data from 2000 
and finding workable models, the Bureau has not documented what 
approaches or processes it intends to apply in the further development 
of the regression model or models it plans to use. In addition, the 
Bureau does not have a written plan detailing how it will explain to 
stakeholders and Bureau data users the rationale for moving from 
poststratification to logistic regression and what effect this change 
may have on data interpretation--particularly as it relates to 
comparisons with past census data. Experts we interviewed also noted 
some challenges, including communicating the new method to users of 
census data and stakeholders, having an adequate number of staff to 
complete the analysis, and designing the best models--such as choosing 
the variables to include. 

Bureau's New Focus on Estimating Components of Coverage Error Is 
Intended to Benefit Future Decennials, but Plans Are Incomplete: 

The Bureau plans to estimate the components of coverage error for 2010 
in order to associate the errors to census operations to improve future 
decennials; in 2000, the coverage measurement program focused on net 
coverage error, not components of coverage error. Although the Bureau 
will once again estimate net coverage error, Bureau documentation shows 
that estimating the components of coverage error is now a higher 
priority for the Bureau. Coverage error comprises two types of errors: 
errors of omission and errors from a person or housing unit mistakenly 
enumerated (erroneous enumerations). The impact of omissions and 
erroneous enumerations are combined to determine the net error. 

During Census 2000, the Bureau estimated the net coverage error of the 
census. However, net error can disguise significant count problems--the 
Bureau could have a large overcount in some areas and large undercount 
elsewhere, and it might balance out to a small net error nationwide. In 
addition, the coverage measurement definitions of erroneous 
enumerations used in 2000 were not as precise as those that will be 
used in 2010. For example, the 2000 coverage measurement program did 
not require Bureau officials to ascertain where a person should have 
been counted. Instead, some enumerations were treated as omissions or 
erroneous enumerations in an area where they should have been counted. 
According to the Department of Commerce, because of the Bureau's new 
goal to measure the components of coverage error, the Bureau will use a 
definition for component error estimation that is not as strict as that 
used for net error. For 2010, erroneous enumerations for the coverage 
measurement program are defined to include the following: duplicate 
enumerations, persons born after or who die before Census Day, or 
persons who are not residents of a housing unit in the United States on 
Census Day. 

Though the Bureau recognizes the potential benefits of estimating the 
components of coverage error to improve future decennials, it has not 
fully developed its plans for doing so in 2010. Providing the Congress 
and other key stakeholders with information about how the Bureau plans 
to use the 2010 coverage measurement program to improve future 
decennials would enhance transparency and oversight of coverage 
measurement.[Footnote 25] For Census 2000, the Bureau did not associate 
the estimates of erroneous enumerations and omissions to specific 
census operations. For the 2010 Census, the Bureau has not finalized 
plans to evaluate the quality of its estimates of components of 
coverage error (i.e., estimates of variance) and has only recently 
determined the level of geography for which estimates will be made. 
Moreover, although the Bureau has developed table shells that associate 
components of coverage error to census operations, the Bureau has not 
clearly explained how it plans to implement operations and data 
collection strategies that will support the analyses and lead to the 
completion of the tables and convert those analyses into plans to 
improve the census operation for 2020. Importantly, Bureau officials 
stated that it will be difficult to link estimates of the components of 
coverage error to specific census operations. Instead, they 
characterize the process of associating the components of error to 
census operations as an exploratory effort. A Bureau official explained 
that associating errors of omission to operations will be most 
challenging because omitting persons from the census count could be 
caused by errors in multiple operations. For example, a person could 
have been omitted because of a bad address or an incorrect enumeration. 
It may not be possible to determine which operation contributed to the 
omission. In contrast, she said that it may be more feasible to 
associate errors of erroneous enumerations to individual census 
operations because there is more that they know about the enumerations. 

In associating errors to census operations, a Bureau official stated 
that the Bureau may use a modeling approach to identify those factors 
with the highest probability and greatest significance in predicting 
specific erroneous enumeration errors. Additionally, Bureau officials 
noted that how the Bureau associates components of coverage error to 
census operations cannot be determined without first determining the 
final models for estimating the components of coverage error for the 
2010 Census. These models cannot be specified without the actual 2010 
Census data and therefore will be specified after data collection. 
Finally, as recommended by the National Academy of Sciences, Bureau 
officials acknowledged that there is ongoing experimental research 
involving the use of a master trace sample database in associating the 
components of coverage error to census operations. This database is 
intended to help the Bureau trace response and operational data through 
various stages of the 2010 Census--such as address listing and data 
collection. The National Academy of Sciences notes that such a database 
should allow the Bureau to retain a full history of relevant census 
operations for a designated sample of households in census blocks. The 
Bureau should then be able to use the database to investigate the 
source of a variety of potential deficiencies that can arise in a 
complicated undertaking like the decennial census. More specifically, 
the database could be used to determine what percentage of census 
omissions are from partially enumerated households and what percentage 
of omissions are from the merged administrative records database. 
According to a Bureau official, the Bureau plans to begin developing 
the database in 2008. However, the official noted several limitations 
with such a database. For example, the database consists of a sample of 
the U.S. population, as opposed to the whole census population. Still, 
Bureau officials noted that improving the master trace sample database 
in 2010 is important, citing the example of using the database for 
complex cluster analyses that link to coverage measurement. 

While most experts we contacted agreed that estimating the components 
of coverage error will likely help to improve future decennials, some 
experts questioned the Bureau's ability to link the actual errors to 
specific census operations. For example, one expert said that it "will 
be practically impossible to project errors to operations." 
Additionally, some experts discussed other challenges that the Bureau 
may face in accurately estimating the components of coverage error, 
including distinguishing between different types of erroneous 
enumerations and omissions as well as resolving duplicates and 
accurately matching the data. Still, some experts did say that it may 
be possible for the Bureau to link certain erroneous enumerations to 
actual operations. Because of the Bureau's new objective to estimate 
components of coverage error and relate them to census operations, it 
is important for the Bureau to solidify its plans and share them with 
stakeholders to ensure that there is an understanding on what the 
program will produce in order to improve future decennials. 

Bureau Plans Expanded Matching for Resolving and Identifying Duplicate 
Enumerations: 

Similar to the matching planned to expand the coverage follow-up 
operation, the Bureau's plans for its 2010 coverage measurement program 
include expanding the matching process in order to better resolve and 
identify duplicate enumerations--a major source of error in the 2000 
Census and coverage measurement program. Some of the experts we 
contacted are confident that the plans will be an improvement. Shortly 
after the census enumeration is completed, a postenumeration survey 
will be conducted in a sample of block clusters. This re-enumeration 
sample--known as the P-sample--is used with the E-sample (the actual 
census enumeration in those sample areas) to derive a corrected 
population estimate.[Footnote 26] For 2010, the Bureau will also use 
computer matching to compare the P-sample with all census enumerations, 
with the objective of identifying and resolving duplicate enumerations. 
This is an expansion from 2000, when the P-sample was matched against 
enumerations from sample blocks and surrounding blocks only, although 
housing units are matched to census housing in the search area. As in 
2000, the Bureau will also use clerical matching to do additional 
matching, but will also do nationwide matching.[Footnote 27] 

Despite the Bureau's effort to improve its matching process and 
minimize the number of duplicates in 2010, the Bureau does not yet have 
complete plans for the 2008 Dress Rehearsal or for the 2010 Census to 
help accomplish this goal. Bureau officials noted that materials are 
being developed for certain matching procedures, such as clerical 
matching. Further, Bureau officials said that budgetary constraints may 
also affect the matching operations and changes may occur before or 
following the 2008 Dress Rehearsal. For example, the Bureau may not 
conduct person matching during the dress rehearsal. 

Though a few experts cited the challenges the Bureau will continue to 
face in resolving duplicate enumerations, several also noted that the 
expanded matching process should help to reduce the number of 
duplicates considerably. An expert explained that by expanding the 
geography covered in the Bureau's matching operations, it will be 
easier to identify duplicates as well as households that may have been 
overlooked in previous coverage measurement matching operations because 
the search area was limited to respective census blocks and a few 
nearby surrounding geographic areas. Still, he also said that the 
expanded matching will be an improvement only if implemented well. 

Bureau Plans for Evaluating the 2010 Coverage Follow-up Operation and 
Census Coverage Measurement Program Are in Early Stages: 

The Bureau is in the early stages of developing plans for evaluating 
its 2010 Census coverage follow-up operation and census coverage 
measurement program. To assist in this effort, the Bureau has 
contracted with the National Academy of Sciences to recommend proposed 
research for the 2010 Census. The academy's final report is due in 
September 2009.[Footnote 28] In its December 2007 interim report, the 
academy recommended that the Bureau conduct certain experiments and 
evaluations, such as assessing the use of administrative records and 
coverage probes for identifying cases for follow-up.[Footnote 29] 
Bureau officials have also suggested research on various aspects of 
coverage follow-up and measurement for 2010. Officials said that the 
Bureau will consider costs and staffing needs in deciding what it will 
evaluate. However, the Bureau has not yet developed a timetable for 
making decisions about evaluating the 2010 coverage follow-up operation 
or census coverage measurement program. 

For the most recent decennial censuses, the Bureau has carried out 
experiments and evaluations. A census experiment usually involves field 
data collection during the census in which alternatives to current 
census processes are used and assessed for a subset of the population. 
An evaluation is usually an analysis conducted on data collected as 
part of the decennial process to determine whether individual steps in 
the census operated as expected. The Bureau plans to work with internal 
and external stakeholders on defining and developing its program for 
experiments and evaluations for the 2010 Census, with design work and 
implementation starting in 2008 and continuing through 2011. The final 
set of activities would include analysis, documentation, and 
presentation of the research, and these activities would start in 2009 
and be completed by the end of the fiscal year 2013. 

According to the National Academy of Sciences' December 2007 interim 
report, its purpose was to reduce the possible subjects for census 
research from a list of over 50 research topics. The academy reviewed 
the Bureau's initial list of research topics to help reduce the 
possible subjects for census experimentation, and offered broad advice 
in its interim report on plans for evaluations of the 2010 Census. For 
this smaller number of research topics, the academy recommended several 
experiments and evaluations relating to coverage improvement and 
measurement. For example, the academy recommended that the Bureau 
assess the use of administrative records in the event that the Bureau 
opts to review administrative records to identify potential omissions. 
It also recommended that the Bureau include one or more alternate 
questionnaire experiments, such as an examination of the usefulness of 
the residence rules and concepts on the census questionnaire. Further, 
the academy recommended a possible experiment on comparing telephone 
interviews to personal interviews for the coverage follow-up operation 
because a telephone-only operation would follow up for only those 
households that provide a contact telephone number. However, at this 
point, the Bureau has not yet made definitive decisions about what it 
will evaluate and when it will make these decisions. 

In its interim report, the National Academy of Sciences suggested that 
the Bureau design a master trace sample database--a tool for 
investigating the source of potential deficiencies that can arise in 
such a complicated undertaking as the decennial census--that would help 
assess the benefits of the coverage follow-up interview. This database 
could link person, household, and housing unit characteristics; census 
processes; and the presence or absence of census component coverage 
error. The academy suggested that the use of this database would better 
identify the sources of high rates of census component coverage error, 
including erroneous enumerations, enumerations in the wrong place, 
duplications, and omissions. The academy also suggested that the Bureau 
use a reverse record check for coverage measurement. It noted that the 
reverse record check is an alternative method for estimating the 
completeness of census coverage of the population, which may have 
advantages over the methods of dual systems estimation and demographic 
analysis that have been used for this purpose to date. This procedure 
has not been used previously to evaluate census data, mainly because of 
the 10-year period between censuses, which complicates the need to 
trace people's addresses from one census to the next. However, the 
Bureau now uses the American Community Survey, a monthly survey of 
250,000 housing units to collect and provide long-form data on an 
annual basis. A reverse check would facilitate comparing decennial 
enumerations to information collected more frequently during the 
American Community Survey. 

The National Academy of Sciences believes that it will be very 
important for the Bureau to finalize specific designs of its 
experiments by summer 2008 to meet the planning needs of the census. 
Because the data needed to support census evaluations are typically 
output files from the census itself, the exact structure of individual 
evaluations is not yet as time-sensitive as the experiments. However, 
the academy notes that some early planning for evaluations is critical 
so that the necessary data extracts can be prepared and retained, 
especially because much of the data collection in 2010 will be carried 
out by contractors, and thus data retention requirements need to be 
arranged with contractors as early as possible. It also suggests that 
the Bureau increase its in-house expertise in experimental design for 
the census. Some experts we interviewed about the Bureau's plans for 
coverage measurement also noted that the Bureau should plan its 
evaluations with more urgency. An expert noted that in 1990 and 2000, 
when the Bureau faced challenges with its adjustment process, there 
were fewer staff members available to work on evaluations; he said that 
perhaps this would not be a problem for the 2010 Census because the 
Bureau did not plan to adjust the census count. Further, a few experts 
suggested that the Bureau should ask outside researchers, in addition 
to the National Academy of Sciences, for assistance and feedback in its 
evaluation planning and development. A few experts also suggested that 
the Bureau should consider developing more qualitative evaluations, 
such as studies to look deeper into why certain populations, such as 
the homeless, are omitted from the census at greater rates. 

Conclusions: 

With only 2 years remaining before Census Day, the Bureau faces 
formidable challenges in successfully implementing and completing the 
2010 Census. These challenges are even more formidable given the 
Bureau's recent decision to redesign certain aspects of the 2010 
Census, and make more critical the need for the Bureau to have in place 
specific and concrete plans for ensuring and measuring the accuracy of 
the decennial census. To accurately count everyone residing in the 
United States, the Bureau must overcome significant challenges due to 
the nation's increasing diversity, as well as, according to the Bureau, 
a population increasingly reluctant to participate in surveys. In 
response to these challenges, the Bureau plans to expand its coverage 
follow-up operation--a final quality assurance method intended to help 
accurately enumerate the population--by focusing on resolving count 
discrepancies, enumerating large households, and following up on 
households with potential enumeration problems identified through 
techniques the Bureau plans to further test during its dress rehearsal. 
As in prior decennials, the Bureau plans to measure the accuracy of the 
2010 decennial through a coverage measurement program. The Bureau 
intends to focus its coverage measurement efforts in 2010 on learning 
more about the components of census error in order to improve 
operations during the next decennial--2020--and future decennials. The 
Bureau does not plan to use its 2010 coverage measurement program to 
adjust the count of the nation's population. 

The Bureau has initiated efforts to improve the accuracy of the census 
count by expanding coverage follow-up and improving measurement; 
however, some areas bear enhanced monitoring, oversight, and management 
as the Bureau moves from its 2008 Dress Rehearsal to final plans for 
the 2010 Census. Providing internal and external stakeholders a full 
description of its plans for coverage follow-up and measurement will 
help the Bureau ensure that the plans for and expected outcomes of 
these important operations are understood not only by the Bureau but 
also by stakeholders. Importantly, many of the operational decisions 
for the 2010 Census depend on the upcoming 2008 Dress Rehearsal. For 
example, the Bureau has not finalized its plans for 2010 regarding the 
scope of its coverage follow-up operation and will base final decisions 
on how to select cases for follow-up in part on its experiences in the 
dress rehearsal. Coverage follow-up improvements have been tested on a 
site-level scale but have not yet been tested with new processing 
interfaces between the Bureau and its contractor for collecting and 
disseminating information. The Bureau plans to expand coverage follow- 
up but has not clearly stated the basis it will use for deciding which 
techniques to use or the number of cases it will include for follow-up. 
The Bureau will rely on the dress rehearsal to assess the effectiveness 
of certain coverage follow-up techniques, such as the undercount probe. 
Existing analyses suggest that this probe is of limited use. Overall, 
given the uncertainty surrounding key aspects of the Bureau's plans for 
coverage follow-up for 2010, as well as the delayed dress rehearsal, 
the content and magnitude of the 2010 coverage follow-up bear continued 
scrutiny. 

Operational plans for coverage measurement lack specificity, and the 
2010 Census will be the first time that the Bureau can implement its 
expanded computer matching for both coverage follow-up and measurement 
on a national level. Importantly, although an announced purpose for 
revising the census coverage measurement program was to provide 
improvements for the 2020 Census and the Bureau is working toward 
associating components of error to specific operations, it has neither 
clearly explained how it will do this nor the expected dates for when 
its plans for this analysis will be complete. All of these unknown 
aspects increase the level(s) of risk of not meeting these important 
goals. By having plans and timelines in place, the Bureau increases the 
likelihood of completing its coverage measurement analyses in time to 
inform the planning for the 2020 decennial. 

Because the Bureau plans significant changes to its coverage follow-up 
and measurement activities, it should be planning well in advance of 
the decennial how it will evaluate and measure their success. Timely 
and informed oversight, monitoring, and management can promote the 
success of these evaluative activities. The Bureau's program for 
evaluations and experiments has great potential to help improve the 
next decennial. The National Academy of Sciences has made interim 
recommendations on various ways that the Bureau could assess its 
decennial data collection activities. Among other things, the academy 
recommends using a master trace sample because such a sample would 
improve the Bureau's efforts to relate census errors to census 
operations. We believe that such a master trace sample has the 
potential to help the Bureau better understand sources of error. 
Nonetheless, designing such a sample is a daunting task. The Bureau has 
begun to evaluate how to implement the academy's recommendation. The 
academy's interim report also suggested that the Bureau soon finalize 
specific designs of the experiments to meet its planning needs so that 
necessary data retention requirements could be arranged with 
contractors as early as possible. 

Recommendations for Executive Action: 

We are recommending that the Secretary of Commerce direct the Bureau to 
provide the Congress and other census stakeholders with more specific 
plans on conducting coverage follow-up, implementing coverage 
measurement, and evaluating the results of those programs for the 2010 
Census. Specifically, we recommend that the Bureau provide: 

* the criteria it will use to assess the techniques for identifying 
cases for the coverage follow-up operation in the 2008 Dress Rehearsal 
and the timeline for completing this assessment to permit planning for 
2010--the timeline should specify dates for preliminary assessments, 
final assessments, and when operational changes might be made to the 
2010 Census; 

* its plans for conducting coverage measurement for the 2010 Census, 
which should include a description of when it will provide estimates of 
net coverage error and components of coverage error and how it plans to 
relate components of coverage error to census operations in order to 
improve future decennials; and: 

* key decision points and plans for evaluating aspects of the 2010 
Census coverage follow-up operation and census coverage measurement 
program. 

Agency Comments and Our Evaluation: 

We provided a draft of this report to the Secretary of Commerce for his 
review and comment. Commerce had no substantive disagreements with our 
recommendations and stated that it will develop formal action plans for 
each recommendation. While Commerce believes a timeline for producing 
estimates of net coverage error and components of coverage error is 
certainly a reasonable expectation, Commerce noted that it is unclear 
as to how it can at this time provide any specifics on how coverage 
results might improve future censuses before the coverage data are 
obtained and analyzed. We recognize that the Bureau has developed table 
shells that associate components of coverage error to census 
operations, but these table shells and plans for how the Bureau will 
conduct its analyses have not been shared with the Congress and other 
stakeholders. In addition, it is not clear from current Bureau plans 
how it will go about obtaining the data required to complete the 
tables. 

As agreed with your office, unless you publicly announce the contents 
of this report earlier, we plan no further distribution until 30 days 
after the report date. We will then send copies of this report to the 
Secretary of Commerce, the Department of Commerce's Inspector General, 
the Director of the U.S. Census Bureau, and interested congressional 
committees. We will make copies available to others upon request. This 
report will also be available at no charge on our Web site at 
[hyperlink, http://www.gao.gov]. 

If you or your staff have any questions concerning this report, please 
contact me at (202) 512-6806 or [email protected]. Contact points for our 
Offices of Congressional Relations and Public Affairs may be found on 
the last page of this report. GAO staff who made major contributions to 
this report are listed in appendix III. 

Sincerely yours, 

Mathew J. Scirï¿½: 

Director, Strategic Issues: 

[End of section] 

Appendix I: Objectives, Scope, and Methodology: 

To determine (1) how the U.S. Census Bureau (Bureau) plans to improve 
coverage through its coverage follow-up operation, (2) how the Bureau's 
census coverage measurement plans for assessing the accuracy of the 
2010 Census compare with efforts in 2000, and (3) how the Bureau plans 
to evaluate the coverage follow-up and measurement efforts after the 
2010 Census, we requested and obtained source documents from the 
Bureau's headquarters in Suitland, Maryland. We also interviewed Bureau 
officials in several of its divisions, including the Decennial 
Management Division and the Decennial Statistical Studies Division. 
Further, we reviewed evaluations regarding coverage follow-up and 
coverage measurement. 

In addition, for the second objective, related to the Bureau's census 
coverage measurement program, we interviewed 15 experts about the 
Bureau's coverage measurement methodology and plans. Prior to our 
interviews, we summarized the Bureau's plans for coverage measurement 
and met with Bureau officials to obtain confirmation of our summary. We 
then identified experts based on published reports as well as 
recommendations from the Bureau and the National Academy of Sciences 
and contacted them about reviewing our summary of the Bureau's 
measurement plans and interviewing them about the Bureau's plans, 
including the extent to which the Bureau's plans addressed concerns and 
limitations of previous coverage evaluation programs, the extent to 
which using logistic regression instead of poststratification would 
improve the estimation of net coverage error, the main challenges in 
estimating components of coverage error, and the implications of 
delaying the coverage measurement interviews. Fifteen experts responded 
affirmatively to our request for interviews and we subsequently 
interviewed the following persons: 

Dr. Barbara Bailar: 
Statistical Consultant: 
Washington, D.C.: 

Dr. Thomas Belin: 
Professor of Biostatistics, Psychiatry, and Biobehavioral Sciences:  
Department of Biostatistics: 
School of Public Health: 
University of California, Los Angeles: 
Los Angeles, California: 

Dr. Michael Brick: 
Senior Statistician: 
Vice President, Director of Survey Methods: 
Associate Director of the Statistical Staff at Westat: 
Rockville, Maryland: 

Dr. James Brown: 
Senior Lecturer in Official Statistics: 
School of the Social Sciences: 
University of Southampton Southampton, United Kingdom: 

Mr. Dave Dolson: 
Director: 
Social Survey Methods Division: 
Statistics Canada: 
Ottawa, Canada: 

Dr. Stephen Fienberg: 
Maurice Falk: 
University Professor of Statistics and Social Science: 
Department of Statistics: 
Carnegie Mellon University: 
Pittsburgh, Pennsylvania: 

Dr. Tim Johnson: 
Director: 
Survey Research Laboratory: 
University of Illinois at Chicago: 
Chicago, Illinois: 

Mr. Don Royce: 
Director General: 
Methodology Branch: 
Statistics Canada: 
Ottawa, Canada: 

Dr. Duane Steffey: 
Director: 
Statistical and Data Sciences: 
Exponet, Inc.: 
Menlo Park, California: 

Dr. John Thompson: 
Executive Vice President of Survey Operations: 
National Opinion Research Center at the University of Chicago: 
Chicago, Illinois: 

Dr. Robert Tortora: 
Chief Methodologist: 
Gallup Washington: 
Washington, D.C.: 

Dr. Roger Tourangeau: 
Director: 
Joint Program in Survey Methodology: 
University of Maryland: 
College Park, Maryland: 
and: 
Acting Director: 
Program in Survey Methodology: 
University of Michigan: 
Ann Arbor, Michigan: 

Dr. Kenneth Wachter: 
Professor of Demography and Statistics: 
University of California at Berkeley: 
Berkeley, California: 

Dr. Martin Wells: 
Director of Research: 
Charles A. Alexander: 
Professor of Statistical Sciences: 
Cornell University: 
Ithaca, New York: 

Dr. Kirk Wolter: 
Head of the Center for Excellence in Survey Research: 
Professor in the Department of Statistics: 
Senior Fellow at the National Opinion Research Center: 
University of Chicago Chicago, Illinois: 

We conducted this performance audit from January 2007 to March 2008 in 
accordance with generally accepted government auditing standards. Those 
standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on our audit objectives. We believe that 
the evidence obtained provides a reasonable basis for our findings and 
conclusions based on our audit objectives. 

[End of section] 

Appendix II: Comments from the Department of Commerce: 

Note: GAO comments supplementing those in the report text appear at the 
end of this appendix. 

The Secretary Of Commerce: 
Washington, D.C. 20230: 

April 3, 2008: 

Mr. Mathew J. Scire: 
Director: 
Strategic Issues: 
United States Government Accountability Office: 
Washington, DC 20548 

Dear Mr. Scire: 

The U.S. Department of Commerce appreciates the opportunity to comment 
on the United States Government Accountability Office's Draft Report 
Entitled Census 2010: Bureau Needs to Specify How It Will Assess 
Coverage Follow-up Techniques and When It Will Produce Coverage 
Measurement Results, (GAO-08-414). I enclose the Department's comments 
on this report. 

Sincerely, 

Signed by: 

Carlos M. Gutierrez: 

Enclosure: 

U.S. Department of Commerce Comments on the United States Government 
Accountability Office Draft Report Entitled Census 2010: Bureau Needs 
to Specify How It Will Assess Coverage Follow-up Techniques and When It 
Will Produce Coverage Measurement Results, (GAO-08-414) March 2008:  

The U.S. Department of Commerce appreciates the United States 
Government Accountability Office's (GAO) efforts to review our coverage 
improvement and coverage measurement processes planned for the 2010 
Census and this opportunity to review the draft report. 

We have no substantive disagreements with these recommendations. Once 
the report is issued in final form, we will develop formal action plans 
for each recommendation. While we believe a timeline for producing the 
estimates is certainly a reasonable expectation (Recommendation 2), we 
are unclear as to how we could at this time provide any specifics on 
how our coverage results might improve future censuses before the 
coverage data are obtained and analyzed by Census Bureau and other 
experts. 

Specific Comments on the Draft Report: 

Page - Highlights: Item 4 in the second paragraph reads (4) expand 
computer matching to match housing units and persons in its post 
enumeration survey to the entire census. 

(See comment 1.): 

Comment: The item should be revised to read (4) expand computer and 
clerical matching to match persons in its post enumeration survey to 
the entire census. The nationwide matching is not done for housing 
units. 

Page 4: The error above regarding expanding computer matching also 
appears on page 4 of your report and should be revised. 

(See comment 2.): 

Page 4: For clarification, there is important information that needs to 
be included about the coverage measurement interviewing starting four 
months later than in 2000. The telephone phase was dropped due to the 
increased complexity of the person interview instrument, and loss of 
time in the conversion from a telephone phase to personal-visit 
implementation. Part of the delay in starting the 2010 Person Interview 
operation is due to eliminating the telephone phase. (For Census 2000, 
the person field visit started on 6/1/00, while the telephone phase 
started on 4/24/00). Although the interviewing will start four months 
later compared to the Census 2000 phone phase, person interviewing ends 
only one and a half months later than in 2000 (9/1/2000 vs. 
10/16/2010). This should reduce the concerns related to recall bias. 

(See comment 3.): 

Page 4, footnote 2: The footnote reads, The dual systems approach . 
estimates of the remaining error in the count after omissions and 
erroneous enumerations offset one another. Comment: The second sentence 
may give a wrong impression that a net error estimate requires 
estimates of omissions and erroneous enumerations. To be more accurate, 
we usually say "reflects" rather than "estimates." We would suggest 
correcting the sentence to read, The net error reflects the error 
remaining after omissions and erroneous enumeration offset one another. 

(See comment 4.): 

Page 4, footnote 3: The footnote reads, "Before computer matching is 
completed, the Bureau conducts . to estimate errors in the census. such 
as nonexistent persons or duplicates." 

(See comment 5. Now footnote 4, p4..): 

Comment: We offer several clarifications to this footnote. A post 
enumeration survey consists of two samples: the P and E samples. P-
sample persons are matched not only to the E sample, but also to the 
entire census. E-sample persons are matched to the entire census to 
identify and resolve duplicates. E-sample housing units are matched to 
census enumerations in the block cluster and surrounding ring of 
blocks. 

Page 16: For clarification, as written, the third paragraph implies 
that when the nonresponse follow-up (NRFU) undercount probe tells us 
there may be an additional person who was not included on the NRFU 
roster, the coverage follow-up (CFU) interview will clarify the 
residency situation so that a person who should be included (but was 
not on the NRFU roster) will be added in CFU. The report describes this 
as "included in the CFU roster after NRFU," which is not accurate. We 
do not add these persons and collect these data during NRFU. If CFU 
roster clarifications indicate someone should be added, this happens 
during CFU. 

(See comment 6. Now footnotes 11, p. 16..): 

Page 18: For clarification of the first paragraph, the limitation with 
identifying duplicates in site tests is not so much the limited sample 
size of the test sites, but rather the limited likelihood of 
duplication of individuals that will occur within a site of limited 
geographic size. 

(See comment 7.): 

Page 18: We suggest two clarifications to the second paragraph: (1) 
duplicated housing units occur when multiple addresses or 
representations of an address for the same housing unit appear on the 
address list. The ones that are "identified during census operations" 
are removed from the list for subsequent operations and (2) the next 
sentence about units being included on both the housing unit list and 
group quarters list is no longer accurate, which represents a big 
change from Census 2000. We now have one list for both, with units 
identified as either housing units or group quarters. Therefore, in 
general, we can no longer have the situation described in the document 
that "addresses can be included in the census in both the housing unit 
and group quarters enumerations." 

(See comment 8. Now on p. 19..): 

Page 19: We suggest two clarifications to the second paragraph: (I) the 
description of the unduplication process is a bit reversed. During the 
automated matching process, a match score is assigned to all links 
based on the level of match. The Census Bureau assesses the quality of 
the match based on the match score and (2) the Field Verification 
operation verifies housing unit status of the units selected for follow-
up in the operation. Units that are invalid are marked as deletes or 
duplicates, and valid units are marked as verified. There are no 
additions during the Field Verification. In some cases, the Field 
Verification operation verifies the existence of units added in other 
operations, in particular, the Be Counted program. For the housing unit 
unduplication cases, the units are already on the list, and Field 
Verification is undertaken in order to determine if housing units 
should be deleted from the list. 

Page 20: For clarification of the second paragraph, the Census Bureau 
plans to use all our CFU test results to make final decisions about 
what to implement in 2010. not just the 2008 results. In particular, we 
have results from the 2005 National Content Test and 2006 Census Test. 
The 2008 implementation was intended to be an effort to collect 
additional data. 

(See comment 10.): 

Page 22: Your statement reads, "Finally, to better measure coverage, 
the Bureau plans to better identify duplicates by expanding its 
computer matching." 

(See comment 11.): 

Comment: While true, we also will expand clerical matching. The 
following revision is suggested: "Finally, to better measure coverage, 
the Bureau plans to better identify duplicates by expanding its 
computer and clerical matching." 

Page 23: The first sentence reads, "During this operation, Bureau field 
staff interviewï¿½ in personï¿½residents of these housing units and collect 
demographic and Census Day residency information of all persons living 
at the housing unit." 

Comment: The sentence should be revised to read as follows: "During 
this operation, Census Bureau field staff interviewï¿½in personï¿½residents 
of these housing units and collect demographics, Interview Day, and 
Census Day residency information of all persons living at the housing 
unit. Additional information will be collected in 2010 to identify 
alternate addresses where the coverage sample roster people may have 
been counted on Census Day. This information will be used in person-
based computer and clerical matching operations." 

(See comment 12.): 

Page 23: The fourth sentence in the first paragraph reads, 
"Additionally, the Bureau conducts computer matching to further resolve 
cases." 

(See comment 13.): 

Comment: This statement is inaccurate. A computer match is not done 
after we conduct the follow-up. 

Page 23: The last sentence in the first paragraph reads, "However, 
unlike for Census 2000, the Bureau will not produce state-level 
estimates by race/ethnicity, age/sex, and tenure or estimates lower 
than the state level." 

(See comment 14.): 

Comment: This statement is incorrect. The Census Bureau did not produce 
any state- level estimates broken down by any characteristics in Census 
2000. 

Page 26: The third bullet also reflects the same error as occurred on 
the Highlights page and page 4. Refer to the comment made on the 
Highlights page above. 

(See comment 15.): 

Page 28, first full paragraph: For clarification, the Census Bureau has 
not "redefined what constitutes an erroneous enumeration" because of 
any shortcomings. Instead, because of the new goal to measure the 
components of coverage error, it became important to use (for component 
error estimation) a definition that is not as strict as that used for 
net error. But the strict definition will continue to be used for net 
error in 2010. 

(See comment 16. Now p. 29..): 

Page 31, second paragraph: For clarification, technically speaking, the 
P-sample is not used to estimate the correct population, but is used to 
estimate the match rate, while the E-sample is used to estimate the 
correct enumeration rate. Both of these samples are needed to estimate 
the true population. Matching is performed to the census, not the E-
sample, although housing units are only matched to census housing in 
the search area. 

(See comment 17. Now footnote 26, p. 31): 

Page 31, footnote 25: Your footnote reads, "Bureau officials have noted 
that the clerical matching procedures being planned for 2010 are 
comparable to those used for coverage measurement in the 2000 Census." 

(See comment 18. Now footnote 27, p. 32..): 

Comment: The clerical matching procedures for 2010 are much more 
complex than in 2000. We have to account for the duplicate searching at 
alternate respondent-provided addresses collected in the person 
interviews and person follow-up interviews. So they are not strictly 
comparable. 

Pages 31-32, last sentence: Your statement reads, "Although the Bureau 
will not include the coverage measurement person follow-up interviews 
in the dress rehearsal, for 2010 the Bureau plans to use information 
collected from those interviews as well as initial person interviews in 
its computer-matching operation." 

(See comment 19.): 

Comment: This statement is inaccurate. Information collected from 
Person Follow-up is not used in computer matching, but only in clerical 
matching. 

Page 33: The last paragraph references 52 research topics covering all 
aspects of its decennial operations that the Bureau provided for the 
National Academy of Sciences to review. 

Comment: The paragraph is misleading and should be clarified as 
follows: The Census Bureau provided a list of 54 research questions 
sorted among 11 research topics to the National Academy of Sciences for 
their review. For the most part, the research questions were not 
identified as either evaluations or experiments. The Academy did not 
reduce the list of subjects for census experimentation to about six. 
Rather, the Academy's interim report presents its ideas for 2010 Census 
experiments, which in some cases are similar to the Census Bureau's 
proposed research.

(See comment 20.): 

The following are our comments on the Department of Commerce's letter 
dated April 3, 2008. 

GAO Comments: 

We clarified our recommendation on relating components of coverage 
error to census operations in order to improve future decennials. 
Specifically, we recognize that the Bureau has developed table shells 
that associate components of coverage error to census operations, but 
these table shells and plans for how the Bureau will conduct its 
analyses should be shared with the Congress and other stakeholders. In 
addition, it is not clear from current Bureau plans how it will go 
about obtaining the data required to complete the tables. 

1. We have revised the Highlights page to reflect the Department of 
Commerce's clarification that for the census coverage measurement 
program, the Bureau will expand its computer and clerical matching to 
match persons in its postenumeration survey. 

2. We have revised the report to reflect the Department of Commerce's 
clarification that the Bureau will expand its computer and clerical 
matching to match persons in its postenumeration survey. 

3. In footnote 3, we have revised the report to clarify that the 
although the coverage measurement interview will start 4 months later 
compared to the Census 2000 telephone phase, person interviewing for 
the 2010 Census ends 1-1/2 months later than in 2000. 

4. In footnote 2, we have revised the sentence to indicate that the net 
error reflects the error remaining after omissions and erroneous 
enumerations offset one another. 

5. In footnote 4, we have revised the definition of the P-sample and E- 
sample based on the Department of Commerce's clarification. 

6. On page 16, we have clarified information about using the undercount 
probe during nonresponse follow-up and coverage follow-up. 

7. On page 18, we have clarified the information about the limited 
likelihood of duplication of individuals that will occur within a site 
of limited geographic size. 

8. On page 19, we have incorporated the clarifying information provided 
by the Department of Commerce. 

9. On page 19, we have clarified information about the matching 
process. 

10. On page 20, we have revised the report to reflect that the 2008 
Dress Rehearsal is intended to provide additional data to make final 
decisions about the Bureau's implementation of coverage follow-up in 
2010. 

11. On page 22, we have revised the report to reflect that the Bureau 
will expand its computer and clerical matching to match persons in its 
postenumeration survey. 

12. On page 23, we have incorporated the Department of Commerce's 
revisions on information that is collected during the census coverage 
measurement program. 

13. On page 23, we have deleted the incorrect sentence. 

14. On page 23, we have deleted the incorrect sentence. 

15. On page 26, we have revised the sentence to reflect that the Bureau 
will expand its computer and clerical matching to match persons in its 
postenumeration survey. 

16. On page 29, we revised the sentence to clarify the reason for 
refining the Bureau's definition for erroneous enumeration. 

17. On page 31, we clarified the definition of the P-sample and E- 
sample as suggested. 

18. On page 31, for footnote 26, we acknowledged the change in clerical 
matching for the 2010 Census. 

19. On page 31, we deleted the incorrect sentence. 

20. On page 33, we provided attribution for the information to the 
National Academy of Sciences' December 2007 interim report. 

[End of section] 

Appendix III: GAO Contact and Staff Acknowledgments: 

GAO Contact: 

Mathew J. Scirï¿½ (202) 512-6806 or [email protected]: 

Acknowledgments: 

In addition to the contact name above, Ron Fecso, Chief Statistician; 
Ernie Hazera, Assistant Director; Jenna Aurand; Betty Clark; and 
Jennifer Edwards made key contributions to this report. Tom Beall, 
Andrea Levine, Donna Miller, Lisa Mirel, and Elizabeth Wood provided 
significant technical support. 

[End of section] 

Footnotes: 

[1] The Bureau is planning to use a 300,000 national housing unit 
sample. 

[2] The dual systems approach uses a postenumeration survey to produce 
an estimate of the true population, which is then used to produce an 
estimate of the net coverage error. The net error reflects the error 
remaining after omissions and erroneous enumerations offset one 
another. See GAO, 1990 Census Adjustment: Estimating Census Accuracy-- 
A Complex Task, GAO/GGD-91-42 (Washington, D.C.: Mar. 11, 1991). 

[3] Although the interviewing will start 4 months later compared to the 
Census 2000 telephone phase, person interviewing for the 2010 Census 
ends 1-1/2 months later than in 2000. 

[4] The Bureau conducts a postenumeration survey, independent of the 
census, in a sample of block clusters. This sample is known as the P- 
sample and estimates the number of persons missed by the census. A 
postenumeration survey consists of two samples: the P-sample and the E- 
sample. P-sample persons are matched not only to the E-sample, but also 
to the entire census. E-sample persons are matched to the entire census 
to identify and resolve duplicates. E-sample housing units are matched 
to census enumerations in the block cluster and surrounding ring of 
blocks. 

[5] The coverage measurement interview depends upon the respondents' 
ability to recall the composition of their households on Census Day. 
The longer the period of time between Census Day and the day that the 
measurement interview is conducted, the harder it is for respondents to 
recall membership of their households on Census Day. 

[6] For example, see GAO, 2010 Census: Cost and Design Issues Need to 
Be Addressed Soon, GAO-04-37 (Washington, D.C.: Jan. 15, 2004); 2010 
Census: Basic Design Has Potential, but Remaining Challenges Need 
Prompt Resolution, GAO-05-9 (Washington, D.C.: Jan. 12, 2005); and 2010 
Census: Census Bureau Should Refine Recruiting and Hiring Efforts and 
Enhance Training of Temporary Field Staff, GAO-07-361 (Washington, 
D.C.: Apr. 27, 2007). 

[7] Starting in January 2007, the Bureau notified state and local 
governments that it would seek their help in developing a complete 
address file through the Local Update of Census Addresses program. 
Address canvassing--a field operation to build a complete and accurate 
address list in which census field workers go door to door verifying 
and correcting addresses for all households and street features 
contained on decennial maps--will begin in April 2009. For more 
information, see GAO, 2010 Census: Census Bureau Has Improved the Local 
Update of Census Addresses Program, but Challenges Remain, GAO-07-736 
(Washington, D.C.: June 14, 2007). 

[8] See also National Academy of Sciences, Once, and Only Once, and in 
the Right Place: Residence Rules in the Decennial Census (Washington, 
D.C.: 2006). 

[9] A household that completes a coverage follow-up interview could 
either finish the interview with the same roster of household members 
listed at the beginning of the interview or make some change to the 
roster. A roster change occurs when a household adds at least one 
person to the original roster, deletes at least one person from the 
original roster, or both. 

[10] Of the 4,191 completed interviews, at least one person was added 
to 360 households while at least one person was deleted from 531 
households. 

[11] During the nonresponse follow-up operation, if persons reply 
affirmatively to the undercount probe, the census field workers collect 
their names and demographic information. Then, these persons appear on 
the coverage follow-up roster and the coverage follow-up interview 
confirms the Census Day residency for these persons. 

[12] While only 8 persons were included in the coverage follow-up 
roster after nonresponse follow-up, the coverage follow-up added an 
additional 69 persons from households that were initially enumerated 
through nonresponse follow-up. 

[13] Bureau officials said that its administrative records system uses 
a unique identifier in place of the Social Security number. Social 
Security numbers are only used for the purpose of validating the 
Bureau's administrative records system. 

[14] If the records do not match, the Bureau does not inform the other 
agencies because of issues related to census confidentiality. 

[15] In the 2004 Census Test, the Bureau was successful in identifying 
duplicates using person matching, but it discovered operational 
complications with trying to resolve person and housing situations 
within the same operation. Basically, the Bureau concluded that with 
such a complicated interview, it was too much to ask of the 
interviewers that they understand the distinction between person and 
housing duplication. A major component of the 2006 Census Test was to 
find a way to determine before follow-up whether person duplication was 
resulting from housing-level issues or person-level issues. 

[16] See GAO, Standards for Internal Control in the Federal Government, 
GAO/AIMD-00-21.3.1 (Washington, D.C.: Nov. 1999). 

[17] See GAO, Internal Control Management and Evaluation Tool, GAO-01-
1008G (Washington, D.C.: Aug. 2001). 

[18] Data contamination occurs when a person's inclusion or exclusion 
from one census operation affects the probability of his or her 
inclusion in a second independent census operation. 

[19] The experts were not aware of these results because the documents 
were for internal Bureau use only. 

[20] According to Bureau officials, the dress rehearsal timeline is the 
best current indicator of the timeline for the 2010 Census. However, 
Bureau officials have told us that coverage measurement estimation will 
not be done for the 2008 Dress Rehearsal. In addition, the Bureau will 
not include its coverage measurement housing unit field operation, 
housing matching operation, or person follow-up. The Bureau is also 
considering dropping person matching from the dress rehearsal. 

[21] The exact date is yet to be determined. 

[22] Poststratification entailed calculating dual systems estimates 
separately for different demographic and geographic groups. Dual 
systems estimation requires that a postenumeration survey be conducted. 
It produces an estimate of the true population, which is then used to 
produce an estimate of the net coverage error. In the 2000 coverage 
measurement program, over 400 strata or groups were examined. 

[23] For instance, logistic regression allows for the use of continuous 
variables like age, instead of only the categorical variables used in 
poststratification. Additionally, unlike poststratification, logistic 
regression is able to include or omit interactions between variables. 

[24] See GAO-01-1008G. 

[25] See GAO-01-1008G. 

[26] According to the Department of Commerce, the P-sample is used to 
estimate the match rate, while the E-sample is used to estimate the 
correct enumeration rate. Both of these samples are needed to derive a 
corrected population estimate. Matching is performed to the census, not 
the E-sample. 

[27] Bureau officials have noted that clerical matching procedures 
being planned for 2010 are largely comparable to those used for 
coverage measurement in the 2000 Census although, with additional 
information being collected in 2010, the Bureau will have to account 
for duplicate searching at alternate respondent-provided addresses 
collected in the person interviews and person follow-up interviews. 

[28] The National Academy of Sciences has convened a panel of experts 
to review the Bureau's program of research, evaluation, and 
experimentation for the 2008 Dress Rehearsal and the 2010 Census. The 
panel will consider priorities for evaluation in the 2010 Census. The 
panel will conduct its work over a 3-year period, from September 2006 
to September 2009. 

[29] National Academy of Sciences, Experimentation and Evaluation Plans 
for the 2010 Census: Interim Report (Washington, D.C.: Dec. 7, 2007). 

GAO's Mission: 

The Government Accountability Office, the audit, evaluation and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the performance 
and accountability of the federal government for the American people. 
GAO examines the use of public funds; evaluates federal programs and 
policies; and provides analyses, recommendations, and other assistance 
to help Congress make informed oversight, policy, and funding 
decisions. GAO's commitment to good government is reflected in its core 
values of accountability, integrity, and reliability.  

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each 
weekday, GAO posts newly released reports, testimony, and 
correspondence on its Web site. To have GAO e-mail you a list of newly 
posted products every afternoon, go to [hyperlink, http://www.gao.gov] 
and select "E-mail Updates."  

Order by Mail or Phone: 

The first copy of each printed report is free. Additional copies are $2 
each. A check or money order should be made out to the Superintendent 
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 
more copies mailed to a single address are discounted 25 percent. 
Orders should be sent to:  

U.S. Government Accountability Office: 
441 G Street NW, Room LM: 
Washington, D.C. 20548:  

To order by Phone: 
Voice: (202) 512-6000: 
TDD: (202) 512-2537: 
Fax: (202) 512-6061:  

To Report Fraud, Waste, and Abuse in Federal Programs:  

Contact:  

Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: 
E-mail: [email protected]: 
Automated answering system: (800) 424-5454 or (202) 512-7470:  

Congressional Relations:  

Ralph Dawn, Managing Director, [email protected]: 
(202) 512-4400: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7125: 
Washington, D.C. 20548:  

Public Affairs: 

Chuck Young, Managing Director, [email protected]: 
(202) 512-4800: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7149: 
Washington, D.C. 20548: 
*** End of document. ***