2000 Census: Better Productivity Data Needed for Future Planning
and Budgeting (04-OCT-01, GAO-02-4).
Nonresponse follow-up was the most expensive and labor-intensive
of all Census 2000 operations. The Census Bureau spent $1.2
billion and used more than 500,000 enumerators to obtain census
information from 42 million nonresponding households in less than
10 weeks. Because of this colossal workload, even small
variations in productivity had significant cost implications.
Workload and enumerator productivity have historically been two
of the largest drivers of census costs, and the Bureau developed
its budget model for the 2000 Census using key assumptions about
these two variables. Nationally, enumerators completed their
nonresponse follow-up workload at a rate of 1.04 housing units
per hour--slightly exceeding the Bureau's expected rate of 1.03
housing units per hour. Productivity varied for the four primary
types of local census offices, ranging from 0.90 housing units
per hour in inner-city and urban areas to 1.10 cases per hour in
rural areas. In refining the data, the Bureau corrected what it
considered to be the most significant discrepancy--a
misclassification of some employees' time charges that overstated
the number of hours worked by nonresponse follow-up enumerators
and understated enumerator production rates.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-02-4
ACCNO: A02061
TITLE: 2000 Census: Better Productivity Data Needed for Future
Planning and Budgeting
DATE: 10/04/2001
SUBJECT: Census
Cost analysis
Data integrity
Productivity in government
1990 Decennial Census
2000 Decennial Census
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-02-4
A
Report to Congressional Requesters
October 2001 2000 CENSUS Better Productivity Data Needed for Future
Planning and Budgeting
GAO- 02- 4
October 4, 2001 The Honorable Henry A. Waxman Ranking Minority Member
Committee on Government Reform House of Representatives
The Honorable Carolyn B. Maloney House of Representatives
Nonresponse follow- up was the most expensive and labor- intensive of all
Census 2000 operations. According to the Bureau of the Census, it cost $1.
2 billion (about 29 percent of the $4. 1 billion spent on decennial
activities in fiscal year 2000) and required over 500,000 enumerators to
obtain census information from about 42 million nonresponding households
in under 10 weeks. Because of this colossal workload, even small
variations in productivity can have significant cost implications. For
example, if enumerators had needed as little as half a day more to
complete their workloads, it would have added over 2 million staff hours
and at least $16 million to the cost of the operation, assuming everyone
worked at the
Bureau*s minimum pay rate of about $8.25 per hour. 1 Not surprisingly,
workload and enumerator productivity have historically been two of the
largest drivers of census costs, and the Bureau developed its budget for
the 2000 Census using a model that contained key assumptions about these
two variables. In our January 2001 response to your request for
information on
enumerator productivity, we reported that productivity data for the 2000
Census was unavailable because the Bureau had not yet assessed its
reliability. 2 Since then, the Bureau completed its reliability assessment
and
made certain refinements. As agreed with your offices, this report follows
up on that earlier request and presents information on (1) enumerator
productivity rates by type of local census office and (2) the Bureau*s
methodology for refining the productivity data.
1 Enumerator wage rates ranged from about $8. 25 to $18. 50 depending on
location. 2 Decennial Censuses: Historical Data on Enumerator Productivity
Are Limited (GAO- 01- 208R, Jan. 5, 2001).
We obtained information on enumerator productivity rates by analyzing
Bureau productivity data and the underlying assumptions used to calculate
them. To obtain information on the Bureau*s methodology for refining the
data and factors that could improve the collection and analysis of
productivity data in the future, we interviewed officials from the
Bureau*s Decennial Management Division and reviewed, but did not audit,
relevant Bureau documents. On August 20, 2001, we requested comments on a
draft of this report from the Secretary of Commerce. In his September 18,
2001, written response, the Secretary of Commerce informed us that
Department of Commerce and Bureau of the Census officials generally agreed
with the
report and had no specific comments on its content or recommendations. We
performed our work from April through July 2001, in Washington, D. C., in
accordance with generally accepted government auditing standards.
Results in Brief Nationally, enumerators completed their nonresponse
follow- up workload at a rate of 1.04 housing units per hour* slightly
exceeding the Bureau*s
expected rate of 1.03 housing units per hour. Productivity varied for the
four primary types of local census offices, ranging from 0.90 housing
units per hour in inner- city and urban areas, to 1. 10 cases per hour in
rural areas. In refining the data thus far, the Bureau corrected for what
it believes was the most significant discrepancy: a misclassification of
certain employees* time charges that overstated the number of hours worked
by nonresponse
follow- up enumerators and understated enumerator production rates. The
Bureau has not yet made any adjustments for a second problem identified in
the data-- employees who worked on both earlier census operations and
nonresponse follow- up and who charged their time to codes for those
earlier activities rather than nonresponse follow- up. Better application
of the Bureau*s quality assurance procedures could have produced more
reliable data initially and eliminated the need to correct the data once
nonresponse follow- up was completed.
Accurate productivity information is important for future planning
decisions. At the same time, the complexity and importance of nonresponse
follow- up requires that the Bureau place a premium on completing the
enterprise according to its operational plan. Nevertheless, with proper
planning, the two functions could be complementary, since the
Bureau*s personnel/ payroll and management information systems already
collect the raw data needed to assess enumerator productivity. Therefore,
as the Bureau plans for 2010 nonresponse follow- up, to ensure that the
Bureau develops a baseline of reliable productivity data for informing
future planning decisions consistent with the operational demands of
nonresponse follow- up, we recommend that the Secretary of Commerce direct
the Bureau to (1) determine the productivity measures needed to gauge
nonresponse follow- up and (2) design procedures, information systems, and
quality control mechanisms to reliably capture and analyze those measures
in accordance with nonresponse follow- up*s operational requirements.
Background In our January 2001 review, we described how the Bureau planned
to assess and refine its data on enumerator productivity because a
significant number of individuals on enumerator applicant lists at some
local census
offices were hired instead as crew leader assistants* a different
position. 3 In some instances, the position change was not reflected in
the Bureau*s personnel/ payroll system. To the extent that this occurred,
Bureau officials said that it would overstate the number of hours that
enumerators actually worked and understate productivity (it did not,
however, affect actual payments to employees because enumerators and crew
leader assistants were paid at the same wage rate). Bureau officials also
had questions
concerning the extent to which enumerators who worked on more than one
census operation charged the codes for these earlier operations rather
than the code for nonresponse follow- up. Our review also noted that
enumerator productivity rates could not be calculated for the 1940 through
1990 censuses because needed data on staffing levels and hours worked were
unavailable, incomplete, or not comparable. 4
Because productivity information will be important for informing the
Bureau*s planning and budgeting processes for the next national head count
in 2010, 5 we recommended that the Secretary of Commerce ensure that the
Bureau, in refining its productivity figures, identify the extent and
nature of any data anomalies, the impact they have on data quality, and
the extent to which the data can be compared by type of local census
office. To help ensure the comparability of data for the 2000 and future
censuses, we also recommended that the Bureau fully document how it
calculates enumerator productivity rates, and report the data by type of
local census 3 GAO- 01- 208R. 4 GAO- 01- 208R. 5 GAO- 01- 208R.
office. In response to our recommendations, as part of its analysis of
enumerator productivity data, the Bureau included information on
enumerator production rates at the local office and national levels. This
information included an explanation of the extent and nature of certain
anomalies in the data, the impact they had on data quality, and the
adjustments made for them. In addition, the Bureau*s analysis included a
detailed methodology that documented how these rates were adjusted and
calculated. Enumerator The Bureau issued preliminary enumerator
productivity data in May 2001. Productivity Rates
Nationally, enumerators completed 1. 04 housing units per hour* slightly
exceeding the 1.03 housing units per hour that the Bureau estimated for
budgetary purposes. The Bureau calculated productivity by dividing its
nonresponse follow- up workload (42. 4 million housing units nationally)
by enumerator production hours (40. 7 million hours nationally). The
Bureau derived this information from its personnel/ payroll and management
information systems.
Enumerator production hours refer to the total time enumerators spent in
the field collecting data and meeting with supervisors, and covers the
actual duration of the operation (April 24 through July 2, 2000). The
hours exclude time spent in training and conducting follow- up activities
after the Bureau completed the initial operation. The Bureau also said it
excluded
subsequent fieldwork that it believes was inadvertently charged to the
nonresponse follow- up code after its completion. Bureau officials noted
that the productivity rates are subject to minor change as the Bureau
further refines its data, in part by conducting a comprehensive analysis
of payroll and personnel data.
According to the Bureau, productivity varied by type of local office-
ranging from 0.90 to 1.21 housing units per hour. For the 2000 Census, the
Bureau had four primary types of local census offices (referred to as
types A, B, C, and D)-- which differed by enumeration methods used and
geographic makeup. Type A offices, located in the hardest- to- enumerate,
inner- city, and urban areas, used mailout/ mailback and urban/ update
leave
enumeration methodologies. 6 Type B offices, located in urban and
metropolitan areas, also used mailout/ mailback and urban/ update leave
methodologies. Type C offices, located in suburban areas, small and
medium- size cities, towns, and rural areas, used update/ leave, mailout/
mailback, and rural update/ enumerate methodologies. 7 Type D offices,
located in more rural areas, used list/ enumerate, update/ leave, and some
mailout/ mailback methodologies. 8 Type E offices* a fifth office type*
were located in Puerto Rico and used update/ leave methodology.
As shown in table 1, according to the Bureau, actual enumerator
productivity was 20 percent higher than expected at the Bureau's urban or
type A offices. Productivity was about 11 percent lower than expected at
rural or type D offices. A Bureau official told us that, as part of its
evaluation of the 2000 Census, the Bureau is studying reasons for these
variations.
6 For the mailout/ mailback methodology, the Bureau used U. S. Postal
Service (USPS) letter carriers to deliver questionnaires to the vast
majority of housing units that had city- style addresses (house number and
street name) for household members to mail back. For the urban update/
leave methodology, in pre- identified census blocks in urban areas-- where
the USPS might usually deliver a quantity of questionnaires to a building
lobby-- enumerators delivered questionnaires to each unit and updated the
address list. 7 The Bureau conducted an update/ leave methodology in areas
with primarily non- city- style addresses. For this methodology,
enumerators delivered questionnaires to housing units and updated their
address list in their assignment areas at the same time. For the rural
update/ enumerate operation, in pre- identified census blocks, enumerators
canvassed an area, updated the address list and associated maps, and
completed census questionnaires for all occupied and vacant housing units.
8 For the list/ enumerate methodology, in very remote or very sparsely
populated areas, enumerators visited every household to update census
maps, conduct interviews, and list
each address or location.
Table 1: Enumerator Productivity During Nonresponse Follow- up Varied by
Type of Local Census Office Difference between Enumerator
Budgeted budgeted
Total Total housing
production housing units
Actual housing and actual
Percentage Office type offices unit workload
hours a per hour units per hour a
productivity difference
National level 520 42, 382, 492 40, 698, 936 1. 03 1. 04 0.01 0. 97% Type
A 102 6, 347, 900 7, 078,897 0. 75 0. 90 0. 15 20 Type B 51 4, 080, 754 4,
012,296 1. 11 1. 02 (0.09) (8.1) Type C 316 28, 008, 736 26, 073, 280 1.
11 1. 07 (0. 04) (3.6) Type D 42 3, 247, 754 2, 960,354 1. 24 1. 10 (0.14)
(11.3) Type E 9 697, 348 574,109 N/ A 1. 21 N/ A N/ A
N/ A = Data was not available from the Bureau at the time of our review. a
Preliminary data.
Source: U. S. Bureau of Census data.
Bureau officials cautioned against comparing the productivity of
individual offices because of location- specific circumstances. The
officials said that the data are more reliable when aggregated by local
census office type and at the national level. For the 1990 Census, the
Bureau reported that enumerators completed 1.56 housing units per hour.
However, Bureau officials said the 1990 figure should not be directly
compared to the 2000 rate because the Bureau does not have documentation
on how the 1990 figure was calculated, and thus does not know if a direct
comparison would be valid. How the Bureau According to Bureau officials,
the Bureau made a *coarse* adjustment for Refined Its
what it considered to be the most significant anomaly-- the
misclassification of crew leader assistant hours. As noted earlier, some
local census offices Productivity Data
hired a substantial number of crew leader assistants from lists of
applicants for enumerator positions. In some cases, the position change
was not entered into the Bureau*s personnel/ payroll system. To the extent
this happened, Bureau officials said that it would overstate the number of
hours that enumerators actually worked and thus understate enumerator
productivity. To adjust for the misclassified crew leader assistant hours,
the Bureau analyzed productivity rates to distinguish between those hours
that
belonged to crew leader assistants and those hours that belonged to
enumerators. Based on its analysis of daily productivity rates at six
local census offices (three that had reliable crew leader assistant data
and three with no crew leader assistant data), employees with a production
rate greater or equal to 0.2 cases per hour were considered to be
enumerators, and their noncase hours (production in which no households
were completed) were included in the Bureau*s adjusted production hour
variable. Employees with a production rate less than 0.2 cases per hour
were considered to be crew leader assistants, and their noncase hours were
excluded from the adjusted production hour variable.
Compared to the original, unadjusted data, using the refined figures,
enumerators completed nonresponse follow- up in 40.7 million production
hours versus 44.3 million production hours, a difference of 3.6 million
production hours or 8.1 percent. This revised estimate of production hours
could be important for future budgeting and planning purposes. For
example, using the lowest enumerator wage rate of about $8. 25 per hour,
the 3.6 million production hour difference would change the Bureau*s cost
estimates by about $29.7 million.
The Bureau has not yet made any corrections for a second problem
identified in its productivity data: employees who worked on earlier
census operations who continued to charge their time to those codes rather
than nonresponse follow- up. According to Bureau officials, the extent and
impact of this problem is unknown. With both problems, better application
of quality assurance procedures at the time the productivity data were
collected could have produced more
reliable data initially, and eliminated the need to correct the data later
in the census cycle. Indeed, enumerators charged codes other than
nonresponse follow- up despite the fact that supervisors were to review
payroll forms to ensure that enumerators entered the correct task codes.
Conclusions Accurate productivity information is important for gauging the
performance of nonresponse follow- up, validating planning assumptions,
preparing and justifying budgets, and devising more cost- effective
censustaking techniques for the future. However, in past censuses, the
Bureau has encountered problems obtaining complete and comparable
productivity data, and the Bureau appears to have repeated this pattern in
2000. As a result, the Bureau needed to spend additional resources to
refine the information, and even then, the adjustment was coarse and
addressed just one of the two known problems.
At the same time, given the size and decentralization of nonresponse
follow- up, a certain amount of procedural error and unreliable data is
unavoidable. Moreover, because of the complexity and importance of
nonresponse follow- up, it is important for the Bureau to emphasize
completing the endeavor according to its operational plan.
As the Bureau develops nonresponse follow- up plans for the 2010 Census, a
key challenge will be developing systems and procedures for collecting
reliable productivity data needed for evaluation and planning, without
detracting from higher- priority operational demands. However, these goals
are not necessarily incompatible. In fact, as the Bureau*s personnel/
payroll and management information systems already collect the raw data
needed
to assess enumerator productivity, with proper planning, the two functions
can be complementary. Consequently, it will be important for the Bureau to
(1) determine the productivity information it needs to evaluate
nonresponse follow- up during the 2010 decennial and plan for future
censuses and (2) integrate those evaluative requirements into its
operational plans and management information systems.
Recommendations for To ensure that the Bureau develops a baseline of
reliable productivity data Executive Action
for evaluating nonresponse follow- up and informing future planning
decisions, we recommend that the Secretary of Commerce direct the Bureau
to, as part of its planning effort for the 2010 Census, (1) determine the
productivity measures needed to assess nonresponse follow- up and (2)
design procedures, information systems, and quality control mechanisms to
capture and analyze those measures consistent with nonresponse follow-
up*s operational requirements. Possible measures
include average number of hours worked by enumerators, cases completed per
production hour, and number of cases completed per enumerator. The data
should, at a minimum, support the analysis of variation in these measures
by census region and type of local census office (type A, B, C, D, or E).
We are sending copies of this report to the Chairman of the House
Committee on Government Reform; Chairman and Ranking Minority Member of
the Subcommittee on the Census, House Committee on
Government Reform; Secretary of Commerce; and Acting Director of the
Bureau of the Census. Copies will be made available to others upon
request. Robert Goldenkoff and Victoria E. Miller made major contributions
to this letter. If you have any questions concerning this letter, please
contact me at (202) 512- 6806. J. Christopher Mihm
Director Strategic Issues
(450066) Lett er
a
GAO United States General Accounting Office
Page 1 GAO- 02- 4 Census Productivity Follow- up United States General
Accounting Office
Washington, D. C. 20548 Page 1 GAO- 02- 4 Census Productivity Follow- up
A
Page 2 GAO- 02- 4 Census Productivity Follow- up
Page 3 GAO- 02- 4 Census Productivity Follow- up
Page 4 GAO- 02- 4 Census Productivity Follow- up
Page 5 GAO- 02- 4 Census Productivity Follow- up
Page 6 GAO- 02- 4 Census Productivity Follow- up
Page 7 GAO- 02- 4 Census Productivity Follow- up
Page 8 GAO- 02- 4 Census Productivity Follow- up
Page 9 GAO- 02- 4 Census Productivity Follow- up
The General Accounting Office, the investigative arm of Congress, exists
to support Congress in meeting its constitutional responsibilities and to
help improve the performance and accountability of the federal government
for the American people. GAO examines the use of public funds; evaluates
federal programs and policies; and provides analyses, recommendations, and
other assistance to help Congress make informed oversight, policy, and
funding decisions. GAO*s commitment to good government is reflected in its
core values of accountability, integrity, and reliability.
The fastest and easiest way to obtain copies of GAO documents is through
the Internet. GAO*s Web site (www. gao. gov) contains abstracts and full-
text files of current reports and testimony and an expanding archive of
older products. The Web site features a search engine to help you locate
documents using key words and phrases. You can print these documents in
their entirety, including charts and other graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as *Today*s Reports,* on its
Web site daily. The list contains links to the full- text document files.
To have GAO E- mail this list to you every afternoon, go to our home page
and complete the easy- to- use electronic order form found under *To Order
GAO Products.*
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent of
Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more
copies mailed to a single address are discounted 25 percent. Orders should
be sent to:
U. S. General Accounting Office P. O. Box 37050 Washington, D. C. 20013
To order by Phone: Voice: (202) 512- 6000 TDD: (301) 413- 0006 Fax: (202)
258- 4066
GAO Building Room 1100, 700 4th Street, NW (corner of 4th and G Streets,
NW) Washington, D. C. 20013
Contact: Web site: www. gao. gov/ fraudnet/ fraudnet. htm, E- mail:
fraudnet@ gao. gov, or 1- 800- 424- 5454 (automated answering system).
GAO*s Mission
Obtaining Copies of GAO Reports and Testimony
Order by Mail or Phone Visit GAO*s Document Distribution Center
To Report Fraud, Waste, and Abuse in Federal Programs
United States General Accounting Office Washington, D. C. 20548- 0001
Official Business Penalty for Private Use $300
Address Correction Requested Presorted Standard
Postage & Fees Paid GAO Permit No. GI00
*** End of document. ***