[Analytical Perspectives]
[Crosscutting Programs]
[4. Strengthening Federal Statistics]
[From the U.S. Government Printing Office, www.gpo.gov]


[[Page 41]]

 
                  4.  STRENGTHENING FEDERAL STATISTICS

  Federal statistical programs produce key information about a range of 
topics of interest to public and private decision makers, including the 
economy, the population, agriculture, crime, education, energy, the 
environment, health, science, and transportation. The ability of 
governments, businesses, and citizens to make appropriate decisions 
about budgets, employment, investments, taxes, and a host of other 
important matters depends critically on the ready availability of 
relevant, accurate, and timely Federal statistics.
  Moreover, for Federal statistical programs to meet the needs of a wide 
range of users, the underlying data systems must be viewed as credible. 
In order to foster this credibility, Federal statistical programs seek 
to adhere to high quality standards and to maintain integrity and 
efficiency in the production of statistics. As the collectors and 
providers of these basic data, Federal agencies act as data stewards--
balancing public and private decision makers' needs for information with 
legal and ethical obligations to minimize reporting burden, respect 
respondents' privacy, and protect the confidentiality of the data 
provided to the Government. This chapter discusses the development of 
standards that principal statistical programs can use to assess their 
performance and presents highlights of their 2005 budget proposals.

                          Performance Standards

  Agencies maintain the quality of their data or information products as 
well as their credibility by setting high performance standards for 
their activities. The statistical agencies and statistical units 
represented on the Interagency Council on Statistical Policy (ICSP) have 
collaborated on developing an initial set of common performance 
standards for use under the Government Performance and Results Act and 
in completing the Administration's new Program Assessment Rating Tool 
(PART). Federal statistical agencies have agreed that there are six 
conceptual dimensions within two general areas of focus that are key to 
measuring and monitoring statistical programs. The first area of focus 
is Product Quality, encompassing the traditional dimensions of 
relevance, accuracy, and timeliness. The second area of focus is Program 
Performance, encompassing the dimensions of cost, dissemination, and 
mission achievement.
  Statistical agencies historically have focused on measuring 
performance in the area of product quality, especially the dimensions 
most amenable to quantitative measurement, specifically accuracy and 
timeliness. Relevance, also an accepted measure of quality, can be 
either a qualitative description of the usefulness of products or a 
quantitative measure such as a customer satisfaction score. Relevance is 
more difficult to measure, and the indicators that do exist are more 
varied.
  Program performance standards form the basis for evaluating 
effectiveness. They address questions such as: Are taxpayer dollars 
spent most effectively? Are products made available to those who need 
them? Are agencies meeting their mission requirements or making it 
possible for other agencies to meet their missions? The indicators 
available to measure program performance for statistical activities 
currently are less well developed than those for product quality.
  Product quality and program performance standards are designed to 
serve as indicators when answering specific questions in the 
Administration's PART process. (Please refer to Chapter 2 of this volume 
for a description of the PART.) Figure 4-1 presents each principal 
Federal statistical agency's assessment of the status of its current and 
planned use of indicators on the six dimensions. Use of the indicators 
may be for internal management, strategic planning, or annual 
performance reporting. The dimensions shown in the figure reflect an 
overall set of indicators for statistical activities but the specific 
measures vary among the individual programs depending on their unique 
characteristics and requirements. Annual performance reports and PARTs 
contain these specific measures as well as additional information about 
performance goals and targets and whether a program is meeting, or 
making measurable progress toward meeting, its performance goals. The 
examples below illustrate different ways agencies track their 
performance on each dimension.

  Product Quality: Statistical agencies agree that product quality 
encompasses many attributes, including (but not limited to) relevance, 
accuracy, and timeliness. The basic measures in this group relate to the 
quality of specific products, thereby providing actionable information 
to managers. These are ``outcome-oriented'' measures and are key to the 
usability of information products. Statistical agencies or units 
establish targets and monitor how well targets are met. In some sense, 
relevance relates to ``doing the right things,'' while accuracy and 
timeliness relate to ``doing things right.''

            Relevance: Qualitative or quantitative descriptions of the 
          degree to which products are useful and responsive to users' 
          needs. Relevance of data products and analytic reports may be 
          assessed through a professional review process and ongoing 
          contacts with data users. Product relevance may be indicated 
          by customer satisfaction with product content, information 
          from customers about product use, demonstration of product 
          improvements, comparability with other data series, agency 
          responses to customer suggestions for improvement, new or

[[Page 42]]

          customized products/services, frequency of use, or responses 
          to data requests from users (including policy makers). Through 
          a variety of professional review activities, agencies maintain 
          the relevance, accuracy, and validity of programs, and 
          encourage data users and other stakeholders to contribute to 
          the agency's data collection and dissemination program. 
          Striving for relevance requires monitoring to ensure that 
          information systems anticipate change and evolve to 
          appropriately measure our dynamic society and economy.

            Accuracy: Qualitative or quantitative measures of important 
          features of correctness, validity, and reliability of data and 
          information products measured as degree of closeness to target 
          values. For statistical data, accuracy measures include 
          sampling error and various aspects of nonsampling error (e.g., 
          response rates, size of revisions, coverage, edit 
          performance). For analysis products, accuracy may be the 
          quality of the reasoning, reasonableness of assumptions, and 
          clarity of the exposition, typically measured and monitored 
          through review processes. In addition, accuracy is assessed 
          and improved by external and internal reviews, comparisons of 
          data among different surveys, linkages of survey data to 
          administrative records, redesigns of surveys, or expansions of 
          sample sizes.

            Timeliness: Qualitative or quantitative measure of the 
          timing of information releases. May be measured as time from 
          the collection of data or the close of the reference period to 
          the release of information, or customer satisfaction with 
          timeliness. May also be measured as how well agencies meet 
          scheduled and publicized release dates, expressed as a percent 
          of release dates met.

  Program Performance: Statistical agencies agree that program 
performance encompasses balancing the dimensions of cost, dissemination, 
and mission accomplishment for the agency as a whole; operating 
efficiently and effectively; ensuring that customers receive the 
information they need; and serving the information needs of the Nation. 
Costs of products or programs may be used to develop efficiency 
measures. Dissemination involves making sure customers receive the 
information they need via the most appropriate mechanisms. Mission 
achievement means that the information program makes a difference. 
Hence, three key dimensions are being used to indicate program 
performance: cost (input), dissemination (output), and mission 
achievement (outcome).

            Cost: Quantitative measure of the dollar amount used to 
          produce data products and services. The development and use of 
          financial performance measures within the Federal Government 
          is an established goal, and the intent of such measures is to 
          determine the ``true costs'' of various programs or 
          alternative modes of operation at the Federal level. Examples 
          of cost data include full costs of products or programs, 
          return on investment, dollar value of efficiencies, and ratios 
          of cost to products distributed.

            Dissemination: Qualitative or quantitative information on 
          the availability, accessibility, and distribution of products 
          and services. Most agencies have goals to improve product 
          accessibility, particularly through the Internet. Typical 
          measures include: on-demand requests fulfilled, product 
          downloads, degree of accessibility, customer satisfaction with 
          ease of use, number of participants at user conferences, 
          citations of agency data in the media, number of Internet user 
          sessions, number of formats in which data are available, 
          amount of technical support provided to data users, exhibits 
          to inform the public about information products, issuance of 
          newsletters describing products, usability testing of 
          websites, and assessing compliance with Section 508 of the 
          Rehabilitation Act which requires Federal agencies to make 
          their electronic and information technology accessible to 
          people with disabilities.

            Mission Achievement: Qualitative or quantitative information 
          about the impact of or satisfaction with statistical programs. 
          For Federal statistical programs, this dimension responds to 
          the question--have we achieved our objectives and met the 
          expectations of our stakeholders? Under this dimension, 
          statistical programs document their contributions to the goals 
          and missions of parent departments and other agencies, the 
          Administration, the Congress, and information users in the 
          private sector. For statistical programs, this broad dimension 
          involves meeting recognized societal information needs and 
          also addresses the linkage between statistical outputs and 
          programmatic outcomes.

            However, identifying this linkage is far from 
          straightforward. It is sometimes difficult to trace the impact 
          of information products on the public good. Such products 
          often are necessary intermediate inputs in the creation of a 
          high visibility product whose societal benefit is clearly 
          recognized. For example, the economic statistics produced by a 
          variety of agencies are directly used by the Bureau of 
          Economic Analysis in the calculation of the Gross Domestic 
          Product (GDP), which analysts use to assess changes in the 
          level of domestic economic activity. Similarly, statistics 
          from specific surveys are directly used by the Bureau of Labor 
          Statistics in the calculation of the Consumer Price Index 
          (CPI), which is widely used in diverse applications, such as 
          indexing pensions for retirees. As a result, a number of 
          statistical agencies contribute to the GDP and/or the CPI and 
          to the many uses of these information products. In addition, 
          the data produced by statistical agencies are used to track 
          the performance of programs managed by their parent agencies 
          or other

[[Page 43]]

          organizations in areas such as crime, education, energy, the 
          environment, health, science, and transportation.

            Moreover, beyond the direct and focused uses of statistical 
          products and programs, the statistical agencies and their 
          products serve a diverse and dispersed set of data users 
          working on a broad range of applications. Users include senior 
          government policy makers at the Federal, State, and local 
          levels, business leaders, households, academic researchers, 
          analysts at public policy institutes and trade groups, 
          marketers and planners in the private sector, and many others. 
          Information produced by statistical agencies often is combined 
          with other information for use in the decision-making process. 
          Thus, as with many nonstatistical programs, the relationship 
          between statistical program outputs and their beneficial uses 
          and outcomes is often complex and difficult to track.

            In the absence of preferred quantitative indicators, 
          qualitative narratives can indicate how statistical agency 
          products contribute to and evaluate progress toward important 
          goals established for government or private programs. In 
          particular, narratives can highlight how statistical agencies 
          measure the Nation's social and economic structure, and how 
          the availability of the information influences changes in 
          policies and programs. These narratives contribute to 
          demonstrating mission accomplishment, particularly in response 
          to questions in Section I of the PART, ``program purpose and 
          design.'' Narratives may describe the impact of measuring 
          agency policy or change of policy, supporting research focused 
          on policy issues, furnishing information to inform debate on 
          policy issues, or providing in-house consulting support.

            In addition to narratives, quantitative measures may be used 
          to reflect mission achievement. For example, customer 
          satisfaction with the statistical agency or unit indicates if 
          the agency or unit has met the expectations of its 
          stakeholders.
          
          

                                            Description of Dimensions
 
 
 
Product Quality
 
Relevance: Qualitative or quantitative description of the degree to which products and
 services are useful to users and responsive to their needs.
 
Accuracy: Qualitative or quantitative measure of important features of correctness,
 validity, and reliability of data and information products measured as degree of closeness
 to target values.
 
Timeliness: Qualitative or quantitative measure of the timing of information releases.
 


[[Page 44]]


                                      Description of Dimensions--Continued
 
 
 
Program Performance
 
Cost: Quantitative measure of the dollar amount used to produce data products and services.
 
Dissemination: Qualitative or quantitative information on the availability, accessibility,
 and distribution of products and services.
 
Mission Achievement: Qualitative or quantitative information about the impact of, or
 satisfaction with, statistical programs.
 
Key to Statistical Agencies
 
BEA = Bureau of Economic Analysis, Department of Commerce
BJS = Bureau of Justice Statistics, Department of Justice
BLS = Bureau of Labor Statistics, Department of Labor
BTS = Bureau of Transportation Statistics, Department of Transportation
Census = Census Bureau, Department of Commerce
EIA = Energy Information Administration, Department of Energy
ERS = Economic Research Service, Department of Agriculture
NASS = National Agricultural Statistics Service, Department of Agriculture
NCES = National Center for Education Statistics, Department of Education
NCHS = National Center for Health Statistics, Department of Health and Human Services
ORES = Office of Research, Evaluation, and Statistics, Social Security Administration
SOI = Statistics of Income, Internal Revenue Service, Department of the Treasury
SRS = Division of Science Resources Statistics, National Science Foundation
 

  Of the 14 principal Federal statistical agencies that are members of 
the ICSP, four agencies have programs that have been assessed using the 
PART process. These agencies' programs have received PART summary 
ratings of Effective or Moderately Effective, as shown in Figure 4-2. As 
additional ICSP agencies have an opportunity to undergo the PART 
process, the agencies plan to use the results of the collaborative 
performance standards development effort to help maintain and extend 
their generally well-received assessments.


     Figure 4-2.  2005 PART SUMMARY RATINGS FOR STATISTICAL PROGRAMS
------------------------------------------------------------------------
                                                Summary Rating
------------------------------------------------------------------------
Bureau of Economic Analysis          Effective
 
Bureau of Labor Statistics           Effective
 
Census Bureau
  Current Demographic Statistics     Moderately Effective
  Decennial Census                   Moderately Effective
  Intercensal Demographic Estimates  Moderately Effective
  Survey Sample Redesign             Effective
 
National Center for Education        ...................................
 Statistics
  Statistics                         Effective
  Assessment                         Effective
------------------------------------------------------------------------

               Highlights of 2005 Program Budget Proposals

  The programs that provide essential statistical information for use by 
governments, businesses, researchers, and the public are carried out by 
some 70 agencies spread across every department and several independent 
agencies. Approximately 40 percent of the funding for these programs 
provides resources for twelve agencies or agency units that have 
statistical activities as their principal mission. (Please see Table 4-
1.) The remaining funding supports work in 60-plus agencies that carry 
out statistical activities in conjunction with other missions such as 
providing services or enforcing regulations. More comprehensive budget 
and program information about the Federal statistical system will be 
available in OMB's annual report, Statistical Programs of the United 
States Government, Fiscal Year 2005, when it is published later this 
year. The following highlights elaborate on the Administration's 
proposals to strengthen the programs of the principal Federal 
statistical agencies.

  Bureau of Economic Analysis: Funding is requested to complete work 
begun in 2003 to: (1) accelerate the release of some of the Nation's 
most important economic statistics to dramatically increase their 
usefulness to policy makers, business leaders, and other users; (2) meet 
U.S. statistical obligations to international organizations on the 
Special Data Dissemination Standards and complete the incorporation of 
the North American Industry Classification System into BEA accounts; (3) 
improve the economic accounts by acquiring monthly real-time data from 
private sources to fill data gaps in current measures as well as conduct 
a quarterly survey of large and volatile international services such as 
telecommunications, finance, and insurance; and (4) produce more current 
business investment data that include associated employment and 
compensation estimates on an annual basis in order to provide data 
needed to conduct analyses of tax policy, business investment, and 
productivity in manufacturing and service industries.
  Bureau of Justice Statistics: Funding is requested to continue 
conversion of the National Crime Victimization Survey from primarily a 
paper and pencil operation to a fully automated data collection process. 
The BJS base program increase will provide for the maintenance of BJS's 
core statistical programs, including: (1) the National Crime 
Victimization Survey, the Nation's primary source of information on 
criminal victimization; (2) cybercrime statistics on the incidence, 
magnitude,

[[Page 45]]

and consequences of electronic and computer crime to households and 
businesses; (3) law enforcement data from over 3,000 agencies on the 
organization and administration of police and sheriffs' departments; (4) 
nationally representative prosecution data on resources, policies, and 
practices of local prosecutors; (5) court and sentencing statistics, 
including Federal and State case processing data; and (6) data on 
correctional populations and facilities from Federal, State, and local 
governments.
  Bureau of Labor Statistics: Funding is requested to support current 
program operations to measure the economy through producing, 
disseminating, and improving BLS economic measures, including: (1) 
modernizing the computing systems for monthly processing of the Producer 
Price Index (PPI) and U.S. Import and Export Price Indexes, and 
producing new data outputs, such as experimental PPI's for goods and 
services that will provide the first economy-wide measures of changes in 
producer prices; (2) maintaining continuous updating of the Consumer 
Price Index (CPI) by updating the expenditure and population weights 
biennially, the superlative index annually, outlet samples on a four-
year cycle, and item samples in key categories on a two-year cycle, in 
lieu of performing major revisions about every ten years; and (3) 
continuing with a multi-year effort to enhance core BLS information 
technology infrastructure through a central Department of Labor 
appropriation.
  Bureau of Transportation Statistics: Funding is requested to: (1) 
develop the American Freight Data Program, a continuous source of 
freight data from shippers, carriers, and receivers, to replace the 
current five-year Commodity Flow Survey; (2) move the Airfare Price 
Index, an input to GDP and CPI indices, from experimental to production 
mode; and (3) develop more timely and comprehensive local and long-
distance travel data.
  Census Bureau: Funding is requested for the Census Bureau's economic 
and demographic programs and for a re-engineered 2010 Census. For the 
Census Bureau's economic and demographic programs, funding is requested 
to: (1) support the release of all remaining data products from the 2002 
Economic Census; (2) begin planning for the 2007 Economic Census and 
Census of Governments; (3) continue efforts begun in 2003 to eliminate 
data gaps by measuring migration across U.S. borders; (4) improve 
measurement of services by expanding key source data for critical 
quarterly and annual estimates of our Nation's Gross Domestic Product; 
(5) continue efforts to offer electronic reporting for almost 100 
current economic surveys; and (6) support the Automated Export System 
and accelerate release of trade statistics. For 2010 Census planning, 
funding is requested to continue to: (1) conduct extensive planning, 
testing, and development activities to support a re-engineered 2010 
Census; (2) complete map feature accuracy within 7.6 meters of true GPS 
location for 48 percent of all counties in the U.S., Puerto Rico, and 
island areas; and (3) conduct the first full year of the American 
Community Survey program to provide data on an ongoing basis rather than 
waiting for once-a-decade censuses.
  Economic Research Service: Funding is requested to develop an 
integrated and comprehensive data and analysis framework of the food 
system beyond the farm-gate to provide a basis for understanding, 
monitoring, tracking, and identifying changes in food supply and 
consumption patterns.
  Energy Information Administration: Funding is requested to: (1) 
continue the improvement of natural gas and electricity survey data; (2) 
undertake development work on a liquefied natural gas storage survey and 
a natural gas production survey; (3) enhance the National Energy 
Modeling System's transportation modeling; and (4) revise the Voluntary 
Greenhouse Gases survey to support the President's Initiative on 
Greenhouse Gases.
  National Agricultural Statistics Service: Funding is requested to: (1) 
continue restoration and modernization of the agricultural estimates 
program to ensure State, regional, and national level agricultural 
estimates of sufficient precision, quality, and detail to meet the needs 
of a broad customer base; and (2) support Government-wide and 
departmental E-Government initiatives.
  National Center for Education Statistics: Funding is requested to: (1) 
support the second wave of data collection of the Early Childhood 
Longitudinal Study--Birth Cohort and data release in Spring 2005; (2) 
continue efforts to improve electronic data collection and data 
dissemination; (3) support the ongoing data collection efforts for the 
Schools and Staffing Survey, the principal collection on national and 
State level indicators of teacher and school quality; (4) continue U.S. 
participation in data collections, analyses, and reporting on 
international assessments that compare educational performance and 
progress across countries; and (5) continue support for the National 
Assessment of Educational Progress (NAEP) program and its role in 
benchmarking national and State performance.
  National Center for Health Statistics: Funding is requested to: (1) 
maintain and transform HHS' core health statistics capacity; (2) 
preserve and modernize the Nation's vital statistics system; (3) fortify 
and transform basic operations for the National Health and Nutrition 
Examination Survey; (4) maintain and redesign systems for tracking the 
health care delivery system; and (5) redesign the sample for the 
National Health Interview Survey.
  Science Resources Statistics Division, NSF: Funding is requested to: 
(1) implement ongoing programs on the science and engineering (S&E) 
enterprise; (2)

[[Page 46]]

continue implementing quality improvements to surveys on the S&E 
workforce; (3) begin research on methods to implement necessary 
enhancements to the Industry Research and Development survey; (4) 
develop an ongoing data collection program on research instrumentation 
stocks, as mandated by Congress; and (5) continue activities to 
establish an ongoing data series on postdoctorates.
  Statistics of Income Division, IRS: Funding is requested to: (1) 
maintain and modernize core data collection systems, including several 
major statistical programs for the Treasury Department, the 
Congressional Joint Committee on Taxation, the Bureau of Economic 
Analysis, and SOI's many other customers; (2) implement a databank 
repository for SOI and IRS population file data to more efficiently 
build longitudinal databases and enable sub-national estimates; (3) 
examine means to more effectively mask individual records to minimize 
the possibility of identification in the Individual Public Use sample 
files; and (4) modernize and expedite dissemination of data and 
publications, including a reengineered Internet website.
  

                    Table 4-1.  2003-2005 BUDGET AUTHORITY FOR PRINCIPAL STATISTICAL AGENCIES
                                            (in millions of dollars)
----------------------------------------------------------------------------------------------------------------
                                                                                                Estimate
                                                                               2003    -------------------------
                                                                              Actual        2004         2005
----------------------------------------------------------------------------------------------------------------
Bureau of Economic Analysis..............................................          66           67           82
 
Bureau of Justice Statistics.............................................          32           32           39
 
Bureau of Labor Statistics...............................................         492          518          534
 
Bureau of Transportation Statistics......................................          30           31           34
 
Census Bureau \1\........................................................         571          632          848
   Salaries and Expenses \1\.............................................         202          213          240
   Periodic Censuses and Programs........................................         369          419          608
 
Economic Research Service................................................          69           71           80
 
Energy Information Administration........................................          80           81           85
 
National Agricultural Statistics Service \2\.............................         138          128          138
 
National Center for Education Statistics.................................         184          187          187
   Statistics............................................................          89           92           92
   Assessment............................................................          95           95           95
 
National Center for Health Statistics....................................         126          128          150
   PHS Evaluation Funds..................................................         126          128          150
   Budget Authority......................................................           0            0            0
 
Science Resources Statistics Division, NSF...............................          31           32           32
 
Statistics of Income Division, IRS.......................................          32           36           36
----------------------------------------------------------------------------------------------------------------
\1\ Includes mandatory appropriations of $20 million for each year for the Survey of Program Dynamics and
  collection of data related to the allocation to States of State Children's Health Insurance Program funds.
 
\2\ Includes funds for the periodic Census of Agriculture of $41, $25, and $23 million in 2003, 2004, and 2005,
  respectively.