2000 Census: Refinements to Full Count Review Program Could Improve Future Data Quality (07/03/2002, GAO-02-562}

-------------------------Indexing Terms-------------------------
REPORTNUM:   GAO-02-562
    TITLE:   2000 Census: Refinements to Full Count Review Program Could Improve Future Data Quality
     DATE:   07/03/2002



This is the accessible text file for GAO report number GAO-02-562 
entitled '2000 Census: Refinements to Full Count Review Program Could 
Improve Future Data Quality' which was released on July 3, 2002. 

This text file was formatted by the U.S. General Accounting Office 
(GAO) to be accessible to users with visual impairments, as part of a 
longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to [email protected]. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

United States General Accounting Office: 
GAO: 

Report to Congressional Addressees: 

July 2002: 

2000 Census: 

Refinements to Full Count Review Program Could Improve Future Data 
Quality: 

Contents: 

Letter: 

Results in Brief: 

Scope and Methodology: 

Background: 

The Numbers and Kinds of Issues Identified During Full Count Review: 

The Bureau Resolved Few Issues Prior to Releasing Public Law Data: 

Lessons Learned That Can Inform Future Data Clearance Reviews: 

Conclusions: 

Recommendations for Executive Action: 

Agency Comments and Our Evaluation: 

Appendix I: Disposition of Data Issues Identified During Full Count 
Review by State: 

Appendix II: Comments From the Department of Commerce: 

Appendix III: GAO Products on the Results of the 2000 Census and 
Lessons Learned for a More Cost-Effective Census in 2010: 

Tables: 

Table 1: Types of Issues Flagged During Full Count Review: 

Table 2: Disposition of Data Issues Identified During Full Count 
Review: 

[End of section] 

United States General Accounting Office: 
Washington, DC 20548: 

July 3, 2002: 

The Honorable Henry A. Waxman: 
Ranking Minority Member: 
Committee on Government Reform: 
House of Representatives: 

The Honorable William Lacy Clay, Jr. 
The Honorable Carolyn B. Maloney: 
House of Representatives: 

Demographic Full Count Review was one of a number of quality assurance
programs the Bureau of the Census developed for the 2000 decennial
headcount to help ensure the completeness and accuracy of census data.
Analysts were to identify, investigate, and document suspected data
discrepancies or ï¿½issuesï¿½ in order to clear census data files and 
products for subsequent processing or public release. Bureau reviewers 
were to determine whether and how to correct the data by weighing 
quality improvements against time and budget constraints. 

According to bureau officials, because the bureau lacked sufficient 
staff to conduct Full Count Review on its own, it contracted out some 
of the analystsï¿½ work to members of the Federal-State Cooperative 
Program for Population Estimates (FSCPE), an organization composed of 
state demographers that, since its inception in 1973, has worked with 
the bureau to ensure accurate state and local population estimates. The
bureau believed that FSCPE membersï¿½ knowledge of the demographic
characteristics of their respective states could help the bureau review 
data files and products, including politically sensitive apportionment 
and redistricting data files known as public law data. [Footnote 1] 

This letter responds to your request to review the FSCPE membersï¿½ 
participation in the Full Count Review program. As agreed with your 
offices, we provide information on (1) the number and kinds of data 
issues identified by FSCPE members and bureau analysts, (2) how the 
bureau used the information developed during Full Count Review, and (3) 
lessons learned from the conduct of Full Count Review that can help the 
bureau plan for the 2010 Census. This report is the latest in a series 
of reviews that examine the results of key census-taking operations and 
highlight opportunities for reform (see app. III for a list of products 
issued to date). 

Results in Brief: 

FSCPE members documented 1,402 data issues, which was about 29 percent 
of the 4,809 issues identified by both FSCPE and bureau analysts during 
Full Count Review. Of the 4,809 issues, 1,599 (33 percent) dealt with 
ï¿½group quarters,ï¿½ where the location or population counts for prisons,
nursing homes, dormitories, and other group living facilities differed 
from what analysts expected. Of the 1,599 group quarters issues, FSCPE
members identified 567 (35 percent). Discrepancies relating to housing
unit counts, population data, and demographic characteristics accounted
for 1,150 issues (24 percent), 375 of which (33 percent) were 
identified by FSCPE members. The bureau was unable to classify 2,060 
issues (43 percent) because of insufficient documentation. 

According to bureau officials, 5 of the 4,809 issues identified during 
Full Count Review were corrected prior to the December 31, 2000, 
release of apportionment data, and the April 1, 2001, release of 
redistricting data. All five were group quarters issues where the 
bureau had the correct population count for each facility but placed 
them in the wrong locations. The ï¿½misplacedï¿½ group quarters included a 
military base, a federal medical center, and multiple facilities at two 
prisons and a college. The bureau was able to correct these issues 
because FSCPE members identified them early in the Full Count Review 
process and thoroughly documented them so that they did not require 
additional research or field verification. Because the bureau lacked 
the time and field staff from its regional offices to do any further 
investigative work, the 4,804 remaining issues went unresolved prior to 
the release of the redistricting data. As a result, uncertainties 
surround the accuracy of the census data for the affected localities. 

Overall, of the 4,809 issues identified during Full Count Review, 4,267 
(89 percent) were not subjected to further investigation by the bureau 
in large part because of insufficient documentation. The bureau plans 
to review 537 issues as part of a subsequent effort called the Count 
Question Resolution (CQR) program, which the bureau developed to 
respond to challenges to the census data brought by state, local, or 
tribal governments (see app. I for the disposition of Full Count Review 
data issues by state). 

The bureauï¿½s preliminary plans for the 2010 Census include a Full Count
Review program. Our examination identified several areas where 
improvements are warranted. Foremost among these is that it will be
important for the bureau to investigate and resolve a larger number of
issues prior to the release of the public law data. We found three 
factors that limited the bureauï¿½s ability to do so for the 2000 Census. 
First, bureau officials said that correcting individual issues was 
beyond the scope of the Full Count Review program. They noted that the 
program was developed in February 1999, just 14 months prior to Census 
Day 2000 and, as a result, the bureau was unable to test the program, 
or integrate it with other census operations and units that could have 
investigated the issues and made corrections. 

Second, the bureauï¿½s requirements for documenting data issues were not
clearly defined. As a result, there was considerable variation in the 
quality of the documentation analysts used to support issues, and in a 
number of cases, the bureau had difficulty determining the precise 
nature of an issue, or if in fact an issue even existed. 

A third, and related item that limited the bureauï¿½s ability to resolve 
a larger number of issues was the fact that the bureau had no mechanism 
for managing its workload. Unlike the CQR program where the bureau
requires specific documentation before committing resources to
investigate local issues, the Full Count Review program had no filter 
for screening submissions. Had the bureau first categorized issues 
based on the quality and precision of the documentation provided, the 
bureau could have prioritized its investigative workload and perhaps 
reconciled a larger number of data issues. 

Another area where there is room for improvement concerns the 
consistency and clarity of the Full Count Review programï¿½s objectives. 
For example, training materials noted that one purpose of the Full Count
Review program was to document issues and ï¿½fix what can be fixed.ï¿½
However, this conflicts with statements from bureau managers that
correcting issues was outside the scope of the Full Count Review 
program. The different messages may have raised FSCPE membersï¿½ 
expectations that the bureau was going to correct a larger number of 
data issues than it actually did. 

That the apportionment and redistricting data were released with around
4,800 unresolved data issues of unknown validity, magnitude, and impact
is cause for concern. The bureau did not fully exploit the Full Count
Review programï¿½s potential, and missed an opportunity to verify and 
possibly improve the quality of the public law data. To help develop the
Full Count Review program into a more effective tool for enhancing data
quality in the future, as well as to make better use of FSCPE membersï¿½
input, we recommend that the Secretary of Commerce explore ways of
reconciling a larger number of data issues prior to the release of 
public law data. Specific steps include: 

(1) planning the Full Count Review program early in the census cycle and
testing procedures under conditions as close to the actual census as
possible; 

(2) integrating the Full Count Review program with other census 
organizational units and operations to ensure the bureau has sufficient
time and field support to investigate issues; 

(3) developing clear guidelines on the minimum documentation needed for
the bureau to investigate individual data issues; 

(4) categorizing issues on the basis of the quality and precision of the
documentation, and investigating first those issues that are best
documented and thus more easily resolved, and; 

(5) exploring the feasibility of using staff from the bureauï¿½s regional
offices to help investigate data issues in the field prior to the 
release of public law data. 

The Secretary of Commerce should also ensure that the bureau clarifies
and consistently communicates to FSCPE members the objectives of the
Full Count Review program and how the bureau plans to use the
information derived from it. 

The Secretary of Commerce forwarded written comments from the Bureau
of the Census on a draft of this report (see app. II). The bureau 
concurred with all of our recommendations and had no comments on them. 
The bureau also provided minor technical corrections that we 
incorporated in our report as appropriate. 

Scope and Methodology: 

To obtain information on the number and kinds of issues identified by 
the FSCPE and bureau analysts and to determine how the bureau used the
information developed during the Full Count Review program, we analyzed 
the work papers submitted by FSCPE members and other participants in 
the Full Count Review program. We also analyzed data from the bureauï¿½s 
Count Review Information System, a database that the bureau used to 
track issues flagged during the review process. We did not 
independently verify the information it contained. 

To identify lessons learned for future improvements, we examined bureau
training manuals, statements of work, process models, and other 
documents that described the objectives, processes, and decision-making
criteria. We also reviewed the results of a survey the bureau conducted 
of FSCPE members that asked them to rate their experience with Full 
Count Review processes and tools, bureau staff, and the overall 
effectiveness of the Full Count Review program. In addition, we 
interviewed managers in the bureauï¿½s Population Division and other 
officials responsible for implementing the Full Count Review program, 
as well as three FSCPE members. 

We performed our audit in Washington, D.C., and the bureauï¿½s 
headquarters in Suitland, Maryland, between May 2001 and April 2002. Our
work was done in accordance with generally accepted government
auditing standards. 

On April 26, 2002, we requested comments on a draft of this report from
the Secretary of Commerce. The Secretary forwarded the bureauï¿½s written
comments on June 11, 2002 (see app. II). We address them in the ï¿½Agency
Comments and Evaluationï¿½ section of this report. 

Background: 

Accurate census results are critical because the data are used to
reapportion seats in the House of Representatives and for congressional
redistricting. Moreover, census data remain an important element in
allocating federal aid to state and local governments. With billions of
dollars at stake, the data are scrutinized intensely for accuracy. 

To help ensure the accuracy of census data, the bureau conducted a
number of quality assurance programs throughout the course of the
census. One such program was the Full Count Review program, which was
designed to rapidly examine, rectify if possible, and clear census data 
files and products for subsequent processing or public release. The 
bureau expected data analysts to identify data discrepancies, 
anomalies, and other data ï¿½issuesï¿½ by checking the data for its overall 
reasonableness, as well as for its consistency with historical and 
demographic data, and other census data products. The Full Count Review 
program ran from June 2000 through March 2001. 

According to bureau officials, because the bureau could not complete the
Full Count Review workload without a costly staff increase, some of the
analystsï¿½ work was contracted to members of the FSCPE, an organization
composed of state demographers that works with the bureau to ensure
accurate state and local population estimates. 

The bureau contracted with 53 FSCPE members who reviewed data for 39
states and Puerto Rico. Bureau employees reviewed data for the 11
remaining states and the District of Columbia without FSCPE 
representation in Full Count Review. Bureau and FSCPE analysts were to
ensure that (1) group quarters were correctly placed or ï¿½geocodedï¿½ on
census maps, and that their population counts and demographic
characteristics appeared reasonable and (2) population counts of other
areas were in line with population estimates. They were to describe each
issue flagged and provide supporting documentation derived from bureau
resources and/or resources of the respective state government.
Additionally, bureau officials stated that staff from the regional 
offices reviewed demographic data from the 50 states, Puerto Rico, and 
the District of Columbia. They focused on identifying inconsistent
demographic characteristics and did not necessarily concentrate on any
one particular state or locality. The bureau reimbursed state 
governments for wages and expenses FSCPE members incurred. 

A separate set of employees from the bureauï¿½s Population Division
assessed issues identified by Full Count Review analysts based on (1) 
the adequacy of the documentation supporting each issue, and (2) 
whether or not they believed the issue to be resolvable through follow-
up research by the bureau. Those issues deemed to have adequate 
documentation were classified as a ï¿½group quarters,ï¿½ ï¿½housing unit,ï¿½ or 
ï¿½householdï¿½ or ï¿½otherï¿½ issue. Bureau officials told us that the 
remaining issues could not be categorized because the nature of the 
issue could not be determined from the documentation. 

The Numbers and Kinds of Issues Identified During Full Count Review: 

Bureau data show that after reviewing census data for 39 states and 
Puerto Rico, FSCPE members identified a total of 1,402 issues, or about 
29 percent of the 4,809 issues collectively flagged during Full Count 
Review (see table 1). Since the bureau has yet to resolve most of these 
issues, it is not known whether they are necessarily errors. 

Table 1: Types of Issues Flagged During Full Count Review: 

Type of issue: Group quarters; 
Issues identified by FSCPE members: 567; 
Issues identified by bureau personnel: 1,032; 
Total: 1,599. 

Type of issue: Housing unit; 
Issues identified by FSCPE members: 203; 
Issues identified by bureau personnel: 276; 
Total: 479. 

Type of issue: Household; 
Issues identified by FSCPE members: 134; 
Issues identified by bureau personnel: 154; 
Total: 288. 

Type of issue: Other; 
Issues identified by FSCPE members: 38; 
Issues identified by bureau personnel: 345; 
Total: 383. 

Type of issue: No type assigned; 
Issues identified by FSCPE members: 460; 
Issues identified by bureau personnel: 1,600; 
Total: 2,060. 

Type of issue: Total; 
Issues identified by FSCPE members: 1,402; 
Issues identified by bureau personnel: 3,407; 
Total: 4,809. 

Note: 201 issues were placed in more than one category. 

Source: GAO analysis of U.S. Census Bureau data. 

[End of table] 

Table 1 also shows that group quarters issues were those most frequently
identified by the bureau, accounting for 1,599 of the 4,809 issues 
identified (33 percent). Group quarters issues relate to suspected 
discrepancies in the population counts and locations of prisons, 
dormitories, nursing homes, and similar group living arrangements. 
Analysts also identified 479 housing unit issues (10 percent of the 
total), and 288 household issues (6 percent of the total). With housing 
unit issues, the count of occupied housing units differed from what 
analysts expected while household issues had population data for 
occupied residences that differed from what analysts expected. There 
were also 383 issues (8 percent) that the bureau classified as ï¿½otherï¿½. 
They contained questions concerning the demographic characteristics of 
the data such as age, race, and gender. The bureau was unable to 
classify 2,060 issues (43 percent). Bureau officials told us that in 
these cases, analysts did not provide sufficient documentation for the 
bureau to determine the nature of the issue. 

According to bureau officials, bureau analysts identified a larger 
number of issues than FSCPE membersï¿½and a far larger number of issues 
for which the bureau could not assign a typeï¿½because bureau analysts 
used an automated process that compared data from the 2000 Census to
independent benchmarks such as the 1990 Census, and flagged any 
anomalies. This process alerted bureau officials that there were data
discrepancies, but did not indicate their nature. By comparison, FSCPE
members compared census data to administrative records and other data,
and were better able to document specific issues. 

Examples of the three issue categories and how they were found include: 

Group quarters issues: Analysts noticed that the group quarters 
population count in a particular census tract of a large midwestern 
city appeared to be too high, while a neighboring tract had a 
correspondingly low group quarters population count. By comparing state 
administrative records to information obtained from bureau resources, 
analysts determined that bureau data had placed college dormitories in 
the wrong tract. 

Housing unit issues: An urban area had a large amount of redevelopment
since the 1990 census. As part of this, several condominiums and 
apartment complexes were built which substantially increased the number
of housing units in a particular census tract. However, when the analyst
compared population data from the 1990 Census and 2000 Census, the
2000 Census did not appear to reflect this increase, and it was 
flagged. 

Household issues: Data from the 2000 Census appeared to accurately 
reflect the large amount of new house construction that had taken place
within a specific census tract. However, because the population count
differed from that indicated by other data sources, the analyst flagged 
it as an issue to avoid undercounting the population. 

The Bureau Resolved Few Issues Prior to Releasing Public Law Data: 

Bureau officials told us that they used the Full Count Review program to
identify systemic errors such as those that could be produced by 
software problems. None were found. The officials noted that the bureau 
generally did not use the Full Count Review program to resolve 
individual issues. According to bureau officials, the bureau corrected 
data for 5 of the 4,809 issues prior to the December 31, 2000, release 
of reapportionment data and the April 1, 2001, release of redistricting 
data. 

According to bureau officials, FSCPE members identified the five issues,
all of which involved group quarters that were placed in the wrong
locations, but the population counts were correct. They included (1) a
military base in Nevada, (2) 10 facilities at a college in Wisconsin, 
(3) 9 facilities at a prison in New York City, (4) 14 facilities at a 
Washington prison, and (5) a federal medical center in Massachusetts. 

Bureau officials said that the bureau was able to correct these issues 
for two reasons. First, FSCPE analysts found them early in the Full 
Count Review program, while the bureau was processing a key geographic 
data file and was thus able to incorporate the corrections before the 
data were finalized. Second, the FSCPE analysts had thoroughly 
documented the issues and recommended how the bureau should correct the 
errors. The five errors did not require additional research or field 
verification. 

Bureau officials told us that they lacked the time to research the 
remaining issues, as well as field staff to inspect purported 
discrepancies prior to the release of the public law data. As a result, 
the bureau missed an important opportunity to verify and possibly 
improve the quality of the data, and instead the apportionment and 
redistricting data were released with more than 4,800 unresolved 
issues. Until these issues are resolved, uncertainties will surround 
the accuracy of the census data for the affected localities. 

Some of the issues might be resolved under the CQR program, which the
bureau designed to respond to challenges to housing unit and group
quarters population counts received from state, local, or tribal
governments. [Footnote 2] However, as shown in table 2, of the 4,804 
issues remaining after Full Count Review, 1,994 (42 percent) were 
referred to CQR, and of these, 537 (11 percent) were accepted for 
further investigation. The remaining 1,457 issues referred to CQR did 
not meet the bureauï¿½s documentation requirements and consequently, the 
bureau took no further action on them (see app. 1 for the disposition 
of Full Count Review data issues by state). 

Table 2: Disposition of Data Issues Identified During Full Count 
Review: 

Action: Identified during Full Count Review; 
Number of issues: 4,809. 

Action: Corrected during Full Count Review; 
Number of issues: 5. 

Action: Identified during Full Count Review but not referred to CQR; 
Number of issues: 2,810. 

Action: Referred to CQR; 
Number of issues: 1,994. 

Action: Accepted by CQR; 
Number of issues: 537. 

Action: Rejected by CQR; 
Number of issues: 1,457. 

Source: GAO analysis of U.S. Census Bureau data. 

[End of table] 

Lessons Learned That Can Inform Future Data Clearance Reviews: 

The overall results of the Full Count Review program and FSCPE membersï¿½ 
participation appear to be mixed. On the one hand, the bureau reported 
that the Full Count Review program was successful in that it met a 
number of performance goals. For example, the bureau reported that the 
Full Count Review program was comprehensive in its review of geography
and content, and was completed in time to release the public law data on
schedule. 

Moreover, between January and February 2001, the bureau surveyed the
40 entities that participated in Full Count Review and the results 
suggest that most FSCPE members were satisfied with their Full Count 
Review experience. For example, respondents indicated that they were 
generally satisfied with such aspects of the program as its processes 
and technical tools, bureau staff, and the overall effectiveness of the 
review in terms of positioning states to use and understand census 
data. In addition, bureau officials believe the Full Count Review 
program benefited from FSCPE membersï¿½ local demographic knowledge. 

Nevertheless, our review of the Full Count Review program highlighted
several areas where there is room for future improvement. It will be
important for the bureau to address these shortcomings as its 
preliminary plans call for a similar operation as part of the 2010 
Census. According to bureau officials, the bureau plans to include a 
Full Count Review program in census tests it expects to conduct later 
in the decade. 

Foremost among the areas in need of improvement is resolving, to the
extent practical, a larger number of data issues prior to the release of
apportionment data by December 31 of the census year, and redistricting
data by April 1 of the following year. We found three factors that 
limited the bureauï¿½s ability to do so. 

First, according to bureau officials, resolving individual issues was 
outside the scope of the Full Count Review program. They explained that 
the program was poorly integrated with other census operations and 
units that could have investigated the issues and corrected the data if 
warranted. This was because the Full Count Review program, with FSCPE
participation, was not conceived until February 1999, which was
extremely late in the census cycle, coming just 14 months before Census
Day, April 1, 2000. 

The timing of the decision stemmed from the Supreme Courtï¿½s January
1999 ruling [Footnote 3] that prohibited the bureau from using 
statistical sampling for purposes of congressional apportionment (the 
bureau originally planned a ï¿½one-numberï¿½ census that would have 
integrated the results of a sample survey with the traditional census 
to provide one adjusted set of census numbers). Faced with the larger 
workload of reviewing two sets of dataï¿½adjusted and unadjustedï¿½the 
bureau decided to enlist the help of FSCPE members in order to meet the 
deadlines for releasing the public law data. Additionally, the bureauï¿½s 
decision came after the 1998 dress rehearsal for the 2000 Census, which 
meant that the bureau had no opportunity to test the Full Count Review 
program in an operational environment. 

Bureau officials explained that if more time or staff were available in 
the future, it would be possible to correct a larger number of 
individual issues prior to the release of the public law data. They 
noted that field staff would be needed to help verify issues, and the 
effort would require close coordination with several bureau units. 

A second factor that affected the bureauï¿½s ability to correct a larger
number of issues was that the bureauï¿½s requirements for documenting data
issues were not clearly defined. For example, the training materials we
examined did not provide any specific guidance on the type of evidence
analysts needed to support data issues. Instead, the training materials 
told analysts to supply as much supporting information as necessary. 
This could help explain the variation that we observed in the quality 
of the documentation analysts provided. Indeed, while some analysts 
provided only minimal data, others supported issues with state and local
administrative records, historical data, photographs, and maps. In some
cases, the bureau had difficulty determining the precise nature of an 
issue or if in fact an issue even existed. 

In contrast, the CQR program provides comprehensive guidelines on the
documentation required for making submissions. The guidance available
on the bureauï¿½s CQR web site notes that before the bureau will 
investigate concerns raised by government and tribal officials, such 
officials must first supply specific information. The guidance then 
details the information needed to support boundary corrections, 
geocoding and coverage corrections, and group quarters population 
corrections. 

A third, and related factor that affected the bureauï¿½s ability to 
resolve a larger number of issues stemmed from the fact that the bureau 
had no mechanism for managing the Full Count Review workload. Unlike the
CQR program, where the bureau required local governments to provide
specific documentation before it would commit resources to investigate
local data issues, the Full Count Review program had no filter for
screening submissions based on the quality of the documentation. Better
guidance on documenting issues for the Full Count Review program could
make the bureauï¿½s follow-up investigations more efficient. 

Another area where there is room for improvement concerns the 
consistency and clarity in which the bureau communicated the objectives
of the Full Count Review program and how the bureau planned to use 
analystsï¿½ input. For example, materials used to train FSCPE members 
noted that one purpose of Full Count Review was to document issues and
ï¿½fix what can be fixed.ï¿½ However, this appears to be inconsistent with
statements made by bureau officials, who noted that resolving individual
issues was beyond the scope of the Full Count Review program. Moreover,
according to one bureau official, it was not clear internally what was
meant by ï¿½fix what can be fixed.ï¿½ 

None of the bureauï¿½s documentation or training manuals that we reviewed
explicitly stated that the bureau would only check for systemic errors.
Because of the inconsistent message on the purpose of the Full Count 
Review program, the bureau may have set up the expectation that a larger
number of issues would be resolved during Full Count Review. For 
example, one FSCPE member told us that he expected FSCPE members would 
identify any geographic discrepancies that contrasted with preliminary 
census data, and the bureau would investigate and make the necessary 
changes. He noted that both he and his staff were very ï¿½dismayedï¿½ to 
find out that certain discrepancies involving group quarters were not 
resolved prior to the release of the public law data. Another FSCPE 
member told us that participants were strongly motivated by the 
expectation that everything would be done to correct the census data. 

Conclusions: 

The Full Count Review program was one of a series of quality assurance
efforts the bureau implemented throughout the census that helped ensure
the bureau released accurate data. Moreover, FSCPE membersï¿½ 
participation, and specifically their expertise and knowledge of local
geography, demographics, and housing arrangements, had the potential to
identify data issues that the bureau might have otherwise missed. 

However, the fact that the apportionment and redistricting data were
released with around 4,800 unresolved data issues of unknown validity,
magnitude, and impact, is cause for concern, and indicates that the 
bureau missed an opportunity to verify and possibly improve the quality 
of the public law data. Given the importance of accurate census data 
and the resources that bureau staff and FSCPE members invest in the 
Full Count Review program, it will be important for the bureau to 
explore how to make better use of the program for correcting potential 
errors in census data in the future. 

It will also be important for the bureau to clarify the purpose of the 
Full Count Review program and convey that purpose clearly and 
consistently to FSCPE members. Doing so could help ensure that the 
bureau meets FSCPE membersï¿½ expectations. 

Recommendations for Executive Action: 

To help ensure the accuracy and completeness of census data and take 
full advantage of the Full Count Review program and FSCPE membersï¿½
participation, we recommend that the Secretary of Commerce direct the
bureau to develop ways to resolve a larger number of data issues prior 
to the release of the public law data. Specifically, consideration 
should be given to: 

(1) planning the Full Count Review program early in the census cycle and
testing procedures under conditions as close to the actual census as
possible; 

(2) integrating the Full Count Review program with other census
organizational units and operations to ensure the bureau has sufficient
time and field support to investigate issues; 

(3) developing clear guidelines on the minimum documentation needed for
the bureau to investigate individual data issues; 

(4) categorizing issues on the basis of the quality and precision of the
documentation, and investigating first those issues that are best
documented and thus more easily resolved, and; 

(5) exploring the feasibility of using staff from the bureauï¿½s regional
offices to help investigate data issues in the field prior to the 
release of public law data. 

Moreover, to ensure no expectation gaps develop between the bureau and
FSCPE members, the Secretary of Commerce should also ensure that the 
bureau clarifies and consistently communicates to participants the 
objectives of the Full Count Review program and how the bureau plans to
use the information derived from it. 

Agency Comments and Our Evaluation: 

The Secretary of Commerce forwarded written comments from the Bureau
of the Census on a draft of this report (see app. II). The bureau 
concurred with all of our recommendations and had no comments on them. 
The bureau also provided minor technical corrections that we 
incorporated in our report as appropriate. 

As agreed with your office, unless you publicly announce its contents
earlier, we plan no further distribution of this report until 30 days 
from its issue date. At that time, we will send copies to other 
interested congressional committees, the Secretary of Commerce, and the 
Director of the Bureau of the Census. Copies will be made available to 
others upon request. In addition, the report will be available at no 
charge on the GAO Web site at [hyperlink, http://www.gao.gov]. Corinna 
Wengryn, Ty Mitchell, and Robert Goldenkoff made major contributions to 
this report. If you have any questions concerning this report, please 
contact me on (202) 512-6806. 

Signed by: 

Patricia A. Dalton: 
Director: 
Strategic Issues: 

[End of section] 

Appendix I: Disposition of Data Issues Identified During Full Count 
Review by State: 

State: Alabama; 
Issues identified by FSCPE members: 18; 
Issues identified by bureau personnel: 54; 
Total accepted for research by Count Question Resolution program: 3. 

State: Alaska; 
Issues identified by FSCPE members: 21; 
Issues identified by bureau personnel: 77; 
Total accepted for research by Count Question Resolution program: 4. 

State: Arizona; 
Issues identified by FSCPE members: 17; 
Issues identified by bureau personnel: 57; 
Total accepted for research by Count Question Resolution program: 5. 

State: Arkansas; 
Issues identified by FSCPE members: 65; 
Issues identified by bureau personnel: 72; 
Total accepted for research by Count Question Resolution program: 16. 

State: California; 
Issues identified by FSCPE members: 20; 
Issues identified by bureau personnel: 71; 
Total accepted for research by Count Question Resolution program: 12. 

State: Colorado; 
Issues identified by FSCPE members: 36; 
Issues identified by bureau personnel: 29; 
Total accepted for research by Count Question Resolution program: 5. 

State: Connecticut; 
Issues identified by FSCPE members: 0; 
Issues identified by bureau personnel: 70; 
Total accepted for research by Count Question Resolution program: 16. 

State: Delaware; 
Issues identified by FSCPE members: 0; 
Issues identified by bureau personnel: 67; 
Total accepted for research by Count Question Resolution program: 0. 

State: District of Columbia; 
Issues identified by FSCPE members: 18; 
Issues identified by bureau personnel: 61; 
Total accepted for research by Count Question Resolution program: 1. 

State: Florida; 
Issues identified by FSCPE members: 39; 
Issues identified by bureau personnel: 61; 
Total accepted for research by Count Question Resolution program: 14. 

State: Georgia; 
Issues identified by FSCPE members: 23; 
Issues identified by bureau personnel: 42; 
Total accepted for research by Count Question Resolution program: 6. 

State: Hawaii; 
Issues identified by FSCPE members: 0; 
Issues identified by bureau personnel: 127; 
Total accepted for research by Count Question Resolution program: 5. 

State: Idaho; 
Issues identified by FSCPE members: 0; 
Issues identified by bureau personnel: 62; 
Total accepted for research by Count Question Resolution program: 2. 

State: Illinois; 
Issues identified by FSCPE members: 112; 
Issues identified by bureau personnel: 52; 
Total accepted for research by Count Question Resolution program: 15. 

State: Indiana; 
Issues identified by FSCPE members: 21; 
Issues identified by bureau personnel: 55; 
Total accepted for research by Count Question Resolution program: 9. 

State: Iowa; 
Issues identified by FSCPE members: 71; 
Issues identified by bureau personnel: 38; 
Total accepted for research by Count Question Resolution program: 21. 

State: Kansas; 
Issues identified by FSCPE members: 19; 
Issues identified by bureau personnel: 46; 
Total accepted for research by Count Question Resolution program: 6. 

State: Kentucky; 
Issues identified by FSCPE members: 16; 
Issues identified by bureau personnel: 60; 
Total accepted for research by Count Question Resolution program: 11. 

State: Louisiana; 
Issues identified by FSCPE members: 25; 
Issues identified by bureau personnel: 66; 
Total accepted for research by Count Question Resolution program: 3. 

State: Maine; 
Issues identified by FSCPE members: 0; 
Issues identified by bureau personnel: 67; 
Total accepted for research by Count Question Resolution program: 1. 

State: Maryland; 
Issues identified by FSCPE members: 8; 
Issues identified by bureau personnel: 39; 
Total accepted for research by Count Question Resolution program: 0. 

State: Massachusetts; 
Issues identified by FSCPE members: 29; 
Issues identified by bureau personnel: 38; 
Total accepted for research by Count Question Resolution program: 14. 

State: Michigan; 
Issues identified by FSCPE members: 23; 
Issues identified by bureau personnel: 51; 
Total accepted for research by Count Question Resolution program: 32. 

State: Minnesota; 
Issues identified by FSCPE members: 69; 
Issues identified by bureau personnel: 44; 
Total accepted for research by Count Question Resolution program: 30. 

State: Mississippi; 
Issues identified by FSCPE members: 0; 
Issues identified by bureau personnel: 123; 
Total accepted for research by Count Question Resolution program: 1. 

State: Missouri; 
Issues identified by FSCPE members: 21; 
Issues identified by bureau personnel: 37; 
Total accepted for research by Count Question Resolution program: 7. 

State: Montana; 
Issues identified by FSCPE members: 27; 
Issues identified by bureau personnel: 69; 
Total accepted for research by Count Question Resolution program: 12. 

State: Nebraska; 
Issues identified by FSCPE members: 0; 
Issues identified by bureau personnel: 93; 
Total accepted for research by Count Question Resolution program: 20. 

State: Nevada; 
Issues identified by FSCPE members: 29; 
Issues identified by bureau personnel: 40; 
Total accepted for research by Count Question Resolution program: 5. 

State: New Hampshire; 
Issues identified by FSCPE members: 0; 
Issues identified by bureau personnel: 39; 
Total accepted for research by Count Question Resolution program: 0. 

State: New Jersey; 
Issues identified by FSCPE members: 58; 
Issues identified by bureau personnel: 52; 
Total accepted for research by Count Question Resolution program: 13. 

State: New York; 
Issues identified by FSCPE members: 33; 
Issues identified by bureau personnel: 100; 
Total accepted for research by Count Question Resolution program: 33. 

State: North Carolina; 
Issues identified by FSCPE members: 0; 
Issues identified by bureau personnel: 149; 
Total accepted for research by Count Question Resolution program: 31. 

State: North Dakota; 
Issues identified by FSCPE members: 19; 
Issues identified by bureau personnel: 47; 
Total accepted for research by Count Question Resolution program: 11. 

State: Ohio; 
Issues identified by FSCPE members: 0; 
Issues identified by bureau personnel: 174; 
Total accepted for research by Count Question Resolution program: 25. 

State: Oklahoma; 
Issues identified by FSCPE members: 14; 
Issues identified by bureau personnel: 41; 
Total accepted for research by Count Question Resolution program: 13. 

State: Oregon; 
Issues identified by FSCPE members: 7; 
Issues identified by bureau personnel: 57; 
Total accepted for research by Count Question Resolution program: 3. 

State: Pennsylvania; 
Issues identified by FSCPE members: 51; 
Issues identified by bureau personnel: 56; 
Total accepted for research by Count Question Resolution program: 12. 

State: Puerto Rico; 
Issues identified by FSCPE members: 20; 
Issues identified by bureau personnel: 57; 
Total accepted for research by Count Question Resolution program: 18. 

State: Rhode Island; 
Issues identified by FSCPE members: 25; 
Issues identified by bureau personnel: 26; 
Total accepted for research by Count Question Resolution program: 7. 

State: South Carolina; 
Issues identified by FSCPE members: 43; 
Issues identified by bureau personnel: 51; 
Total accepted for research by Count Question Resolution program: 11. 

State: South Dakota; 
Issues identified by FSCPE members: 33; 
Issues identified by bureau personnel: 50; 
Total accepted for research by Count Question Resolution program: 11. 

State: Tennessee; 
Issues identified by FSCPE members: 31; 
Issues identified by bureau personnel: 55; 
Total accepted for research by Count Question Resolution program: 7. 

State: Texas; 
Issues identified by FSCPE members: 86; 
Issues identified by bureau personnel: 93; 
Total accepted for research by Count Question Resolution program: 1. 

State: Utah; 
Issues identified by FSCPE members: 19; 
Issues identified by bureau personnel: 59; 
Total accepted for research by Count Question Resolution program: 8. 

State: Vermont; 
Issues identified by FSCPE members: 112; 
Issues identified by bureau personnel: 35; 
Total accepted for research by Count Question Resolution program: 12. 

State: Virginia; 
Issues identified by FSCPE members: 0; 
Issues identified by bureau personnel: 135; 
Total accepted for research by Count Question Resolution program: 8. 

State: Washington; 
Issues identified by FSCPE members: 59; 
Issues identified by bureau personnel: 52; 
Total accepted for research by Count Question Resolution program: 8. 

State: West Virginia; 
Issues identified by FSCPE members: 7; 
Issues identified by bureau personnel: 59; 
Total accepted for research by Count Question Resolution program: 4. 

State: Wisconsin; 
Issues identified by FSCPE members: 0; 
Issues identified by bureau personnel: 124; 
Total accepted for research by Count Question Resolution program: 11. 

State: Wyoming; 
Issues identified by FSCPE members: 26; 
Issues identified by bureau personnel: 39; 
Total accepted for research by Count Question Resolution program: 10. 

State: Total; 
Issues identified by FSCPE members: 1,402; 
Issues identified by bureau personnel: 3,407; 
Total accepted for research by Count Question Resolution program: 537. 

Note: The table includes 201 issues that were placed in more than one 
issue category (the bureau had five categories of issues depending on 
the nature of the discrepancy). 

Source: GAO analysis of U.S. Census Bureau data. 

[End of table] 

[End of section] 

Appendix II: Comments From the Department of Commerce: 

The Secretary Of Commerce: 
Washington, D.C. 20230: 

June 11, 2002: 

Ms. Patricia A. Dalton: 
Director, Strategic Issues: 
U.S. General Accounting Office: 
Washington, DC 20548: 

Dear Ms. Dalton: 

The Department of Commerce appreciates the opportunity to comment on 
the General Accounting Office draft report entitled 2000 Census: 
Refinements to Full Count Review Program Could Improve Future Data 
Quality. The Department's comments on this report are enclosed. 

Warm regards, 

Signed by: 

Donald L. Evans: 

Enclosure: 

Comments from the U.S. Department of Commerce: 
U.S. Census Bureau: 

U.S. General Accounting Office draft report entitled 2000 Census: 
Refinements to Full Count Review Program Could Improve Future Data 
Quality: 

Comments on the Text of the Report: 

1. Section: Page 5, last paragraph, continued on page 6 ï¿½ "The Full 
Count Review program ran from June through August, 2000." 

Comment: The period June through August 2000 was the training phase of 
the Full Count Review program. The program ran from June 2000 through 
March 2001. 

2. Section: Page 6, second full paragraph, first sentence ï¿½ "The bureau 
contracted with 53 SCPE members who reviewed data for 38 states, the 
District of Columbia, and Puerto Rico." 

Comment: The 53 FSCPE members represented 39 states and Puerto Rico. 
The District of Columbia did not participate in the Full Count Review 
program. 

3. Section: Page 6, second full paragraph, second sentence ï¿½ "Bureau 
employees reviewed data for the 12 remaining states without FSCPE 
representation in Full Count Review." 

Comment: The sentence should read "... remaining 11 states ...." 

4. Section: Page 7, first paragraph -- "Bureau data show that after 
reviewing census data for 38 states, the District of Columbia, and 
Puerto Rico, FSCPE members identified ...." 

Comment: See comment in response to Item #2 above. 

5. Section: Page 7, Table 1. 

Comment: In the GAO analysis in Table 1, 201 of the issues were placed 
in more than one category. For this reason, the Census Bureau cannot 
independently verify the accuracy of these data. 

6. Section: Page 9, last paragraph, last sentence ï¿½ "In all, of the 
4,809 issued [sic] identified during Full Count Review, 4,267 (89 
percent), were not subjected to any further review by the bureau ...." 

Comment: The 1,457 issues that were referred to the Count Question 
Resolution (CQR) program, but rejected, were "subjected to further 
review," in the context of the CQR program criteria. That they failed 
to meet the criteria for processing under the CQR, and no further 
action was taken with respect to these issues, does not mean that no 
additional review was undertaken after they were identified during the 
Full Count Review program. 

7. Section : Page 10, third paragraph, last sentence ï¿½ "According to 
bureau officials, if funding permits, the bureau will include a Full 
Count Review program as part of a 2004 test of census-taking 
activities." 

Comment: the Census Bureau is still developing its plan for the 2004 
Census Test and has not determined at this time whether it will include 
a Full Count Review program. However, we certainly plan to include such 
a program in census tests later in the decade. 

Responses to GAO Recommendations: 

Census Bureau Response: The Census Bureau concurs with the 
recommendations and has no specific comments on them at this time. 

[End of enclosure] 

[End of section] 

Appendix III: GAO Products on the Results of the 2000 Census and 
Lessons Learned for a More Cost-Effective Census in 2010: 

U.S. General Accounting Office. 2000 Census: Coverage Evaluation
Matching Implemented As Planned, but Census Bureau Should Evaluate
Lessons Learned. GAO-02-297. Washington, D.C.: March 14, 2002. 

U.S. General Accounting Office. 2000 Census: Best Practices and Lessons 
Learned for a More Cost-Effective Nonresponse Follow-Up. GAO-02-196. 
Washington, D.C.: February 11, 2002. 

U.S. General Accounting Office. 2000 Census: Coverage Evaluation 
Interviewing Overcame Challenges, but Further Research Needed. GAO-
02-26. Washington, D.C.: December 31, 2001. 

U.S. General Accounting Office. 2000 Census: Analysis of Fiscal Year 
2000 Budget and Internal Control Weaknesses at the U.S. Census Bureau. 
GAO-02-30. Washington, D.C.: December 28, 2001. 

U.S. General Accounting Office. 2000 Census: Significant Increase in 
Cost Per Housing Unit Compared to 1990 Census. GAO-02-31. Washington, 
D.C.: December 11, 2001. 

U.S. General Accounting Office. 2000 Census: Better Productivity Data 
Needed for Future Planning and Budgeting. GAO-02-4. Washington, D.C.:
October 4, 2001. 

U.S. General Accounting Office. 2000 Census: Review of Partnership 
Program Highlights Best Practices for Future Operations. GAO-01-579.
Washington, D.C.: August 20, 2001. 

U.S. General Accounting Office. Decennial Censuses: Historical Data on
Enumerator Productivity Are Limited. GAO-01-208R. Washington, D.C.:
January 5, 2001. 

U.S. General Accounting Office. 2000 Census: Information on Short- and 
Long-Form Response Rates. GAO/GGD-00-127R. Washington, D.C.: June 7, 
2000. 

[End of section] 

Footnotes: 

[1] The Census Act (13 USC sec. 141(a,b)) requires the Secretary of 
Commerce to deliver state population counts to the President within 9 
months after the census date (for the 2000 Census, this meant no later 
than December 31, 2000). State population counts are used to 
reapportion seats in the U.S. House of Representatives. The Census Act 
also requires the Secretary of Commerce to send census population 
tabulations to the states no later than 1 year after the April 1 
decennial census date (13 USC sec. 141 (c)). These numbers are used
for redistricting. 

[2] CQR began in late June 2001, and is scheduled to end in September 
2003. 

[3] Department of Commerce v. United States House of Representatives, 
525 U.S. 316 (1999). 

[End of section] 

GAOï¿½s Mission: 

The General Accounting Office, the investigative arm of Congress, 
exists to support Congress in meeting its constitutional 
responsibilities and to help improve the performance and accountability 
of the federal government for the American people. GAO examines the use 
of public funds; evaluates federal programs and policies; and provides 
analyses, recommendations, and other assistance to help Congress make 
informed oversight, policy, and funding decisions. GAOï¿½s commitment to 
good government is reflected in its core values of accountability, 
integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through the Internet. GAOï¿½s Web site [hyperlink, 
http://www.gao.gov] contains abstracts and fulltext files of current 
reports and testimony and an expanding archive of older products. The 
Web site features a search engine to help you locate documents using 
key words and phrases. You can print these documents in their entirety, 
including charts and other graphics. 

Each day, GAO issues a list of newly released reports, testimony, and 
correspondence. GAO posts this list, known as ï¿½Todayï¿½s Reports,ï¿½ on its 
Web site daily. The list contains links to the full-text document 
files. To have GAO e-mail this list to you every afternoon, go to 
[hyperlink, http://www.gao.gov] and select ï¿½Subscribe to daily E-mail 
alert for newly released productsï¿½ under the GAO Reports heading. 

Order by Mail or Phone: 

The first copy of each printed report is free. Additional copies are $2 
each. A check or money order should be made out to the Superintendent 
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 
more copies mailed to a single address are discounted 25 percent. 
Orders should be sent to: 

U.S. General Accounting Office: 
441 G Street NW, Room LM: 
Washington, D.C. 20548: 

To order by Phone: 
Voice: (202) 512-6000: 
TDD: (202) 512-2537: 
Fax: (202) 512-6061: 

To Report Fraud, Waste, and Abuse in Federal Programs Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: 
E-mail: [email protected]: 
Automated answering system: (800) 424-5454 or (202) 512-7470: 

Public Affairs: 

Jeff Nelligan, managing director, [email protected]: 
(202) 512-4800: 
U.S. General Accounting Office: 
441 G Street NW, Room 7149:
Washington, D.C. 20548: