[Analytical Perspectives]
[Crosscutting Programs]
[4. Strengthening Federal Statistics]
[From the U.S. Government Printing Office, www.gpo.gov]



[[Page 53]]


 
                  4.  STRENGTHENING FEDERAL STATISTICS

  Federal statistical programs produce key information about a range of 
topics of interest to public and private decision makers, including the 
economy, the population, agriculture, crime, education, energy, the 
environment, health, science, and transportation. The ability of 
governments, businesses, and citizens to make appropriate decisions 
about budgets, employment, investments, taxes, and a host of other 
important matters depends critically on the ready availability of 
relevant, accurate, and timely Federal statistics. The Federal 
statistical community remains on alert for opportunities to strengthen 
these measures of our Nation's performance. For example, during 2004, 
Federal statistical agencies launched the first new economic indicator 
survey in 40 years--the Quarterly Services Survey (Census Bureau); 
expanded regional economic data from 318 to 934 American communities 
(BEA); successfully adopted new collection and processing technologies 
that will greatly accelerate the release of data from the National 
Health Interview Survey (NCHS); and completed the first data collection 
on the cyberinfrastructure of academic and biomedical facilities (NSF's 
SRS).
  For Federal statistical programs to effectively benefit such a wide 
range of users, the underlying data systems must be viewed as credible. 
In order to foster this credibility, Federal statistical programs seek 
to adhere to high quality standards and to maintain integrity and 
efficiency in the production of statistics. As the collectors and 
providers of these basic data, Federal statistical agencies act as data 
stewards--balancing public and private decision makers' needs for 
information with legal and ethical obligations to minimize reporting 
burden, respect respondents' privacy, and protect the confidentiality of 
the data provided to the Government. This chapter discusses the 
development of standards that principal statistical programs can use to 
assess their performance and presents highlights of their 2006 budget 
proposals.

                          Performance Standards

  Statistical programs maintain the quality of their data or information 
products as well as their credibility by setting high performance 
standards for their activities. The statistical agencies and statistical 
units represented on the Interagency Council on Statistical Policy 
(ICSP) have collaborated on developing an initial set of common 
performance standards for use under the Government Performance and 
Results Act and in completing the Administration's Program Assessment 
Rating Tool (PART). Federal statistical agencies have agreed that there 
are six conceptual dimensions within two general areas of focus that are 
key to measuring and monitoring statistical programs. The first area of 
focus is Product Quality, encompassing the traditional dimensions of 
relevance, accuracy, and timeliness. The second area of focus is Program 
Performance, encompassing the dimensions of cost, dissemination, and 
mission achievement.
  Statistical agencies historically have focused on measuring 
performance in the area of product quality, especially the dimensions 
most amenable to quantitative measurement, specifically accuracy and 
timeliness. Relevance, also an accepted measure of quality, can be 
either a qualitative description of the usefulness of products or a 
quantitative measure such as a customer satisfaction score. Relevance is 
more difficult to measure, and the indicators that do exist are more 
varied.
  Program performance standards form the basis for evaluating 
effectiveness. They address questions such as: Are taxpayer dollars 
spent most effectively? Are products made available to those who need 
them? Are agencies meeting their mission requirements or making it 
possible for other agencies to meet their missions? The indicators 
available to measure program performance for statistical activities 
currently are less well developed.
  Product quality and program performance standards are designed to 
serve as indicators when answering specific questions in the 
Administration's PART process. Chart 4-1 presents each principal Federal 
statistical agency's assessment of the status of its current and planned 
use of indicators on the six dimensions. During the past year, four 
agencies (BTS, EIA, NCES, and SRS) have improved the status of their 
indicators. Use of the indicators may be for internal management, 
strategic planning, or annual performance reporting. The dimensions 
shown in the figure reflect an overall set of indicators for statistical 
activities but the specific measures vary among the individual programs 
depending on their unique characteristics and requirements. Annual 
performance reports and PARTs contain these specific measures as well as 
additional information about performance goals and targets and whether a 
program is meeting, or making measurable progress toward meeting, its 
performance goals. The examples below illustrate different ways agencies 
track their performance on each dimension.

[[Page 54]]

                                     



                                            Description of Dimensions
 
 
 
Product Quality
 
Relevance: Qualitative or quantitative descriptions of the degree to which products and
 services are useful to users and responsive to users' needs.
 
Accuracy: Qualitative or quantitative measure of important features of correctness,
 validity, and reliability of data and information products measured as degree of closeness
 to target values.
 
Timeliness: Qualitative or quantitative measure of the timing of information releases.
 
Program Performance
 
Cost: Quantitative measure of the dollar amount used to produce data products and services.
 
Dissemination: Qualitative or quantitative information on the availability, accessibility,
 and distribution of products and services.
 
Mission Achievement: Qualitative or quantitative information about the effect of, or
 satisfaction with, statistical programs.
 
Key to Statistical Agencies
 
BEA = Bureau of Economic Analysis, Department of Commerce
BJS = Bureau of Justice Statistics, Department of Justice
BLS = Bureau of Labor Statistics, Department of Labor
BTS = Bureau of Transportation Statistics, Department of Transportation
Census = Census Bureau, Department of Commerce
EIA = Energy Information Administration, Department of Energy
ERS = Economic Research Service, Department of Agriculture
NASS = National Agricultural Statistics Service, Department of Agriculture
NCES = National Center for Education Statistics, Department of Education
NCHS = National Center for Health Statistics, Department of Health and Human Services
ORES = Office of Research, Evaluation, and Statistics, Social Security Administration
SOI = Statistics of Income, Internal Revenue Service, Department of the Treasury
SRS = Division of Science Resources Statistics, National Science Foundation
 

  Product Quality: Statistical agencies agree that product quality 
encompasses many attributes, including (but not limited to) relevance, 
accuracy, and timeliness. The basic measures in this group relate to the 
quality of specific products, thereby providing actionable information 
to managers. These are ``outcome-oriented''

[[Page 55]]

measures and are key to the usability of information products. 
Statistical agencies or units establish targets and monitor how well 
targets are met. In some sense, relevance relates to ``doing the right 
things,'' while accuracy and timeliness relate to ``doing things 
right.''

            Relevance: Qualitative or quantitative descriptions of the 
          degree to which products and services are useful and 
          responsive to users' needs. Relevance of data products and 
          analytic reports may be monitored through a professional 
          review process and ongoing contacts with data users. Product 
          relevance may be indicated by customer satisfaction with 
          product content, information from customers about product use, 
          demonstration of product improvements, comparability with 
          other data series, agency responses to customer suggestions 
          for improvement, new or customized products/services, 
          frequency of use, or responses to data requests from users 
          (including policy makers). Through a variety of professional 
          review activities, agencies maintain the relevance, accuracy, 
          and validity of their products, and encourage data users and 
          other stakeholders to contribute to the agency's data 
          collection and dissemination programs. Striving for relevance 
          requires monitoring to ensure that information systems 
          anticipate change and evolve to appropriately measure our 
          dynamic society and economy.

            Accuracy: Qualitative or quantitative measures of important 
          features of correctness, validity, and reliability of data and 
          information products measured as degree of closeness to target 
          values. For statistical data, accuracy may be defined as the 
          degree of closeness to the target value and measured as 
          sampling error and various aspects of nonsampling error (e.g., 
          response rates, size of revisions, coverage, edit 
          performance). For analysis products, accuracy may be the 
          quality of the reasoning, reasonableness of assumptions, and 
          clarity of the exposition, typically measured and monitored 
          through review processes. In addition, accuracy is assessed 
          and improved by internal reviews, comparisons of data among 
          different surveys, linkages of survey data to administrative 
          records, redesigns of surveys, or expansions of sample sizes.

            Timeliness: Qualitative or quantitative measure of timing of 
          information releases. Timeliness may be measured as time from 
          the close of the reference period to the release of 
          information, or customer satisfaction with timeliness. 
          Timeliness may also be measured as how well agencies meet 
          scheduled and publicized release dates, expressed as a percent 
          of release dates met.

  Program Performance: Statistical agencies agree that program 
performance encompasses balancing the dimensions of cost, dissemination, 
and mission accomplishment for the agency as a whole; operating 
efficiently and effectively; ensuring that customers receive the 
information they need; and serving the information needs of the Nation. 
Costs of products or programs may be used to develop efficiency 
measures. Dissemination involves making sure customers receive the 
information they need via the most appropriate mechanisms. Mission 
achievement means that the information program makes a difference. 
Hence, three key dimensions are being used to indicate program 
performance: cost (input), dissemination (output), and mission 
achievement (outcome).

            Cost: Quantitative measure of the dollar amount used to 
          produce data products or services. The development and use of 
          financial performance measures within the Federal Government 
          is an established goal, and the intent of such measures is to 
          determine the ``true costs'' of various programs or 
          alternative modes of operation at the Federal level. Examples 
          of cost data include full costs of products or programs, 
          return on investment, dollar value of efficiencies, and ratios 
          of cost to products distributed.

            Dissemination: Qualitative or quantitative information on 
          the availability, accessibility, and distribution of products 
          and services. Most agencies have goals to improve product 
          accessibility, particularly through the Internet. Typical 
          measures include: on-demand requests fulfilled, product 
          downloads, degree of accessibility, customer satisfaction with 
          ease of use, number of participants at user conferences, 
          citations of agency data in the media, number of Internet user 
          sessions, number of formats in which data are available, 
          amount of technical support provided to data users, exhibits 
          to inform the public about information products, issuance of 
          newsletters describing products, usability testing of web 
          sites, and assessing compliance with Section 508 of the 
          Rehabilitation Act, which requires Federal agencies to make 
          their electronic and information technology accessible to 
          people with disabilities.

            Mission Achievement: Qualitative or quantitative information 
          about the effect of, or satisfaction with, statistical 
          programs. For Government statistical programs, this dimension 
          responds to the question--have we achieved our objectives and 
          met the expectations of our stakeholders? Under this 
          dimension, statistical programs document their contributions 
          to the goals and missions of parent departments and other 
          agencies, the Administration, the Congress, and information 
          users in the private sector and the general public. For 
          statistical programs, this broad dimension involves meeting 
          recognized societal information needs and also addresses the 
          linkage between statistical outputs and programmatic outcomes.

            However, identifying this linkage is far from 
          straightforward. It is frequently difficult to trace the 
          effects of information products on the public

[[Page 56]]

          good. Such products often are necessary intermediate inputs in 
          the creation of high visibility information whose societal 
          benefit is clearly recognized. For example, the economic 
          statistics produced by a variety of agencies are directly used 
          by the Bureau of Economic Analysis in the calculation of the 
          Gross Domestic Product (GDP), which analysts universally use 
          to assess changes in the level of domestic economic activity. 
          Similarly, statistics from specific surveys are directly used 
          by the Bureau of Labor Statistics in the calculation of the 
          Consumer Price Index (CPI), which is widely used in diverse 
          applications, such as indexing pensions for retirees. As a 
          result, a number of statistical agencies can claim credit for 
          contributing to the GDP and/or the CPI and to the many uses of 
          these information products. In addition, the statistics 
          produced by statistical agencies are used to track the 
          performance of programs managed by their parent or other 
          organizations related to topics such as crime, education, 
          energy, the environment, health, science, and transportation.

            Moreover, beyond the direct and focused uses of statistical 
          products and programs, the statistical agencies and their 
          products serve a diverse and dispersed set of data users 
          working on a broad range of applications. Users include 
          government policy makers at the Federal, State, and local 
          levels, business leaders, households, academic researchers, 
          analysts at public policy institutes and trade groups, 
          marketers and planners in the private sector, and many others. 
          Information produced by statistical agencies often is combined 
          with other information for use in the decision-making process. 
          Thus, the relationship between program outputs and their 
          beneficial uses and outcomes is often complex and difficult to 
          track. Consequently, agencies use both qualitative and 
          quantitative indicators to make this linkage as explicit as 
          feasible.

            In the absence of preferred quantitative indicators, 
          qualitative narratives can indicate how statistical agency 
          products contribute to and evaluate progress toward important 
          goals established for government or private programs. In 
          particular, narratives can highlight how statistical agencies 
          measure the Nation's social and economic structure, and how 
          the availability of the information influences changes in 
          policies and programs. These narratives contribute to 
          demonstrating mission accomplishment, particularly in response 
          to questions in Section I of the PART, ``program purpose and 
          design.'' Narratives may describe statistical information's 
          effects on measuring agency policy or change of policy, 
          supporting research focused on policy issues, informing debate 
          on policy issues, or providing in-house consulting support.

            In addition to narratives, quantitative measures may be used 
          to reflect mission achievement. For example, customer 
          satisfaction with the statistical agency or unit indicates if 
          the agency or unit has met the expectations of its 
          stakeholders.

  Of the 14 principal Federal statistical agencies that are members of 
the ICSP, six agencies have programs that have been assessed using the 
PART process. Most of these agencies' programs have received PART 
summary ratings of Effective or Moderately Effective, as shown in Chart 
4-2. While recognizing the strength of the Energy Information 
Administration's purpose and management, EIA's PART evaluation found 
that it lacks specific annual performance measures, baselines, and 
targets and should consider enhancing independent expert evaluation of 
its major program areas. EIA is correcting both of these shortcomings, 
which should bring its PART rating into line with those of its sister 
agencies. As additional ICSP agencies have an opportunity to undergo the 
PART process, the agencies plan to continue to use the results of the 
collaborative performance standards development effort to help maintain 
and extend their generally favorable assessments.


  Chart 4-2.  Most Recent PART Summary Ratings for Statistical Programs
------------------------------------------------------------------------
                                                Summary Rating
------------------------------------------------------------------------
Bureau of Economic Analysis          Effective
 
Bureau of Labor Statistics           Effective
 
Census Bureau
  Current Demographic Statistics     Effective
  Decennial Census                   Moderately Effective
  Economic Census                    Effective
  Intercensal Demographic Estimates  Moderately Effective
  Survey Sample Redesign             Effective
 
Energy Information Administration    Results Not Demonstrated
 
National Agricultural Statistics     Moderately Effective
 Service
 
National Center for Education        ...................................
 Statistics
  Statistics                         Effective
  Assessment                         Effective
------------------------------------------------------------------------



                    Table 4-1.  2004-2006 BUDGET AUTHORITY FOR PRINCIPAL STATISTICAL AGENCIES
                                            (in millions of dollars)
----------------------------------------------------------------------------------------------------------------
                                                                                                Estimate
                                                                               2004    -------------------------
                                                                              Actual      2005 \1\       2006
----------------------------------------------------------------------------------------------------------------
Bureau of Economic Analysis \2\..........................................          68           73           81
 
Bureau of Justice Statistics \3\.........................................          32           34           63
 
Bureau of Labor Statistics...............................................         518          529          543
 
Bureau of Transportation Statistics......................................          30           30           33
 
Census Bureau \4\........................................................         629          765          897
  Salaries and Expenses \4\..............................................         213          216          240
  Periodic Censuses and Programs.........................................         416          549          657
 
Economic Research Service................................................          71           74           81
 
Energy Information Administration........................................          81           84           86
 
National Agricultural Statistics Service \5\.............................         128          128          145
 
National Center for Education Statistics.................................         187          185          208
  Statistics.............................................................          92           91           91
  Assessment.............................................................          95           94          117
 
National Center for Health Statistics \6\................................          90          109          109
 
Science Resources Statistics Division, NSF...............................          31           32           32
 
Statistics of Income Division, IRS.......................................          36           39           39
----------------------------------------------------------------------------------------------------------------
\1\ Reflects any recissions.
 
\2\ 2005 estimate includes $2 million for a National Academy of Public Administration study of off-shoring.
 
\3\ The 2006 estimate includes funds for the Felony Arrestee Drug Use Reporting program (previously funded as
  the Arrestee Drug Abuse Monitoring program within the National Institute of Justice) as well as funds for
  management and administrative costs that were displayed separately in 2004 and 2005.
 
\4\ Includes Mandatory Appropriations of $20 million for each year for the Survey of Program Dynamics and
  collection of data related to the allocation to States of State Chidren's Health Insurance Program funds.
 
\5\ Includes funds for the periodic Census of Agriculture of $25, $22, and $29 million in 2004, 2005, and 2006,
  respectively. The 2006 Budget includes an increase of $6.5 million due to cyclical activities including
  finalizing content, developing mail lists, and streamlining and upgrading processing systems in preparation
  for the 2007 Census of Agriculture.
 
\6\ All funds from the Public Health Service Evaluation Fund. Funds for 2004 are shown comparably with 2005 and
  2006. Administrative costs for NCHS that previously were displayed as part of the NCHS budget line are now
  reflected in two consolidated CDC-wide budget lines for management and administrative costs.

               Highlights of 2006 Program Budget Proposals

  The programs that provide essential statistical information for use by 
governments, businesses, researchers, and the public are carried out by 
some 70 agencies spread across every department and several independent 
agencies. Approximately 40 percent of the funding for these programs 
provides resources for twelve agencies or units that have statistical 
activities as their principal mission. (Please see Table 4-1.) The 
remaining funding supports work in 60-plus agencies or units that carry 
out statistical activities in conjunction with other missions such as 
providing services or enforcing regulations. More comprehensive budget 
and program information about the Federal statistical system will be 
available in OMB's annual report, Statistical Programs of the United 
States Government, Fiscal Year 2006, when it is published later this 
year. The following highlights elaborate on the Administration's

[[Page 57]]

proposals to strengthen the programs of the principal Federal 
statistical agencies.

  Bureau of Economic Analysis: Funding is requested to: (1) make 
selected improvements to the timeliness and comprehensiveness of the 
Nation's international statistics on multinational corporations and 
trade in services; (2) complete work to accelerate the release of gross 
state product, metropolitan personal income, and county-level personal 
income; (3) enhance the accuracy of BEA statistics by acquiring and 
incorporating real-time data into core BEA accounts; (4) improve data on 
international financial transactions by working with the Department of 
the Treasury and the Federal Reserve Board to incorporate newly 
developed estimates of derivatives and other financial instruments; and 
(5) produce up-to-date, annual estimates of business investment spending 
by industry in order to more accurately discern where high-tech and 
other investments are being made in the manufacturing and service 
sectors.
  Bureau of Justice Statistics: Funding is requested to provide for the 
maintenance of BJS's core statistical programs, including: (1) the 
National Crime Victimization Survey, the Nation's primary source of 
information on criminal victimization; (2) cybercrime statistics on the 
incidence, magnitude, and consequences of electronic and computer crime 
to households and businesses; (3) law enforcement data from over 3,000 
agencies on the organization and administration of police and sheriffs' 
departments; (4) nationally representative prosecution data on 
resources, policies, and practices of local prosecutors; (5) court and 
sentencing statistics, including Federal and State case processing data; 
(6) data on correctional populations and facilities from Federal, State, 
and local governments; and (7) the Felony Arrestee Drug Use Reporting 
program (previously funded as the Arrestee Drug Abuse Monitoring program 
within the National Institute of Justice).
  Bureau of Labor Statistics: Funding is requested to support program 
operations to measure the economy through producing, disseminating, and 
improving BLS

[[Page 58]]

economic measures, including: (1) modernizing the computing systems for 
monthly processing of the Producer Price Index (PPI) and U.S. Import and 
Export Price Indexes (IPP), and producing new data outputs, such as 
indexes based on the North American Industry Classification System for 
the IPP; (2) maintaining continuous updating of the Consumer Price Index 
(CPI) by updating the expenditure and population weights biennially, the 
superlative index annually, outlet samples on a four-year cycle, and 
item samples in key categories on a two-year cycle; and (3) releasing 
the 2004-2014 Employment Projections and publishing the 2006-2007 
edition of the Occupational Outlook Handbook.
  Bureau of Transportation Statistics: Funding is requested to: (1) 
enhance the Freight Data Program, a continuous source of data from 
shippers, carriers, and receivers, to replace the current Commodity Flow 
Survey; (2) move the Air Transportation Price Index, an input to GDP and 
CPI indices, from experimental to production mode; and (3) develop more 
timely and comprehensive local and long-distance travel data.
  Census Bureau: Funding is requested for the Census Bureau's economic 
and demographic programs, and for a reengineered 2010 Census. For the 
Census Bureau's economic and demographic programs, funding is requested 
to: (1) plan for the 2007 Economic Census, (2) plan and implement the 
organizational phase of the 2007 Census of Governments and plan for the 
employment and finance phases, (3) improve measurement of services by 
expanding key source data for critical quarterly and annual estimates of 
our Nation's Gross Domestic Product, (4) support improved coverage and 
electronic reporting of trade statistics, (5) support the development of 
a database infrastructure to integrate State administrative data and 
Census Bureau data products in order to fill critical data gaps at the 
State and local levels, (6) continue efforts begun in 2003 to eliminate 
data gaps by measuring migration across U.S. borders, and (7) purchase 
furniture and relocate operations and employees to the new headquarters 
facility to avoid disruption of mission-critical operations necessary 
for the successful completion of Census Bureau surveys. For 2010 Census 
planning, funding is requested to continue to: (1) conduct planning, 
testing, and development activities to support a reengineered 2010 
Census; (2) complete map feature accuracy within 7.6 meters of true GPS 
location for 700 of the Nation's counties; and (3) continue to conduct 
the American Community Survey program to provide data on an ongoing 
basis rather than waiting for once-a-decade censuses.
  Economic Research Service: Funding is requested to support ongoing 
programs and to continue the development of an integrated and 
comprehensive data and analysis framework of the food system beyond the 
farm-gate to provide a basis for understanding, monitoring, tracking, 
and identifying changes in the food supply and consumption patterns.
  Energy Information Administration: Funding is requested to continue 
ongoing operations, with a focus on: (1) improving petroleum and natural 
gas data security, reliability, and quality; (2) conducting the 
commercial, manufacturing, and residential energy consumption surveys; 
(3) implementing the enhanced Voluntary Reporting of Greenhouse Gases 
program to support the President's Climate Change Initiative; and (4) 
developing a program performance prototype to assess EIA's data 
collection and operations costs at a more disaggregated level.
  National Agricultural Statistics Service: Funding is requested to: (1) 
continue restoration and modernization of the agricultural estimates 
program to ensure State, regional, and national level agricultural 
estimates of sufficient precision, quality, and detail to meet the needs 
of a broad customer base; (2) continue development and implementation of 
a locality-based agricultural county estimates/small area estimation 
program; and (3) continue preparations for the 2007 Census of 
Agriculture.
  National Center for Education Statistics: Funding is requested to: (1) 
support the ongoing data collection and analysis of the Early Childhood 
Longitudinal Study Birth and Kindergarten Cohorts, which provide data to 
inform child development practices and early education; (2) continue the 
Integrated Education Postsecondary Data System, which collects 
information on enrollment, completions, and finances from postsecondary 
institutions; (3) sustain the ongoing data collection efforts for the 
Beginning Postsecondary Students Longitudinal Study; (4) maintain U.S. 
participation in international assessments that compare educational 
achievement in the United States with that in other countries; and (5) 
continue the National Assessment of Educational Progress (NAEP) program, 
including funding to support the expansion of State NAEP to grade 12.
  National Center for Health Statistics: Funding is requested to: (1) 
increase timeliness by upgrading electronic systems for data collection 
and processing; (2) expand the content of surveys, particularly those 
addressing the health care delivery system; (3) redesign the sample for 
the National Health Interview Survey, NCHS' largest population survey; 
and (4) work collaboratively with States and other agencies on upgrading 
the technology for collecting data from State birth and death 
certificates.
  Science Resources Statistics Division, NSF: Funding is requested to: 
(1) continue to implement the results of prior methodological, 
analytical, and planning activities directed toward improving the 
relevance, accuracy, timeliness, and accessibility of SRS products, 
including the suite of Research and Development surveys and the Survey 
of Graduate Students and Postdoctorates in Science and Engineering; and 
(2) lead a cross-agency effort to examine and revise current

[[Page 59]]

taxonomies used for classifying academic fields of study, including the 
development of crosswalks between existing taxonomies and any potential 
new taxonomy, as well as strengthen methods to enhance the 
identification and description of cross-disciplinary and multi-
disciplinary fields.
  Statistics of Income Division, IRS: Funding is requested to: (1) 
maintain and modernize core data collection systems, including several 
major statistical programs for the Department of the Treasury, the 
Congressional Joint Committee on Taxation, the Bureau of Economic 
Analysis, and SOI's many other customers; (2) implement a databank 
repository for SOI and IRS population file data to more efficiently 
build longitudinal databases and enable sub-national estimates; (3) 
examine means to more effectively mask individual records to minimize 
the possibility of identification in the Individual Public Use sample 
files; and (4) modernize and expedite dissemination of data and 
publications, including a reengineered Internet website.




[[Page 61]]