    [Analytical Perspectives]
    [Crosscutting Programs]
    [4. Strengthening Federal Statistics]
    [From the U.S. Government Printing Office, www.gpo.gov]

    [[Page 41]]

                      4.  STRENGTHENING FEDERAL STATISTICS

      Federal statistical programs produce key information about a range of
    topics of interest to public and private decision makers, including the
    economy, the population, agriculture, crime, education, energy, the
    environment, health, science, and transportation. The ability of
    governments, businesses, and citizens to make appropriate decisions
    about budgets, employment, investments, taxes, and a host of other
    important matters depends critically on the ready availability of
    relevant, accurate, and timely Federal statistics.
      Moreover, for Federal statistical programs to meet the needs of a wide
    range of users, the underlying data systems must be viewed as credible.
    In order to foster this credibility, Federal statistical programs seek
    to adhere to high quality standards and to maintain integrity and
    efficiency in the production of statistics. As the collectors and
    providers of these basic data, Federal agencies act as data stewards--
    balancing public and private decision makers' needs for information with
    legal and ethical obligations to minimize reporting burden, respect
    respondents' privacy, and protect the confidentiality of the data
    provided to the Government. This chapter discusses the development of
    standards that principal statistical programs can use to assess their
    performance and presents highlights of their 2005 budget proposals.

                              Performance Standards

      Agencies maintain the quality of their data or information products as
    well as their credibility by setting high performance standards for
    their activities. The statistical agencies and statistical units
    represented on the Interagency Council on Statistical Policy (ICSP) have
    collaborated on developing an initial set of common performance
    standards for use under the Government Performance and Results Act and
    in completing the Administration's new Program Assessment Rating Tool
    (PART). Federal statistical agencies have agreed that there are six
    conceptual dimensions within two general areas of focus that are key to
    measuring and monitoring statistical programs. The first area of focus
    is Product Quality, encompassing the traditional dimensions of
    relevance, accuracy, and timeliness. The second area of focus is Program
    Performance, encompassing the dimensions of cost, dissemination, and
    mission achievement.
      Statistical agencies historically have focused on measuring
    performance in the area of product quality, especially the dimensions
    most amenable to quantitative measurement, specifically accuracy and
    timeliness. Relevance, also an accepted measure of quality, can be
    either a qualitative description of the usefulness of products or a
    quantitative measure such as a customer satisfaction score. Relevance is
    more difficult to measure, and the indicators that do exist are more
    varied.
      Program performance standards form the basis for evaluating
    effectiveness. They address questions such as: Are taxpayer dollars
    spent most effectively? Are products made available to those who need
    them? Are agencies meeting their mission requirements or making it
    possible for other agencies to meet their missions? The indicators
    available to measure program performance for statistical activities
    currently are less well developed than those for product quality.
      Product quality and program performance standards are designed to
    serve as indicators when answering specific questions in the
    Administration's PART process. (Please refer to Chapter 2 of this volume
    for a description of the PART.) Figure 4-1 presents each principal
    Federal statistical agency's assessment of the status of its current and
    planned use of indicators on the six dimensions. Use of the indicators
    may be for internal management, strategic planning, or annual
    performance reporting. The dimensions shown in the figure reflect an
    overall set of indicators for statistical activities but the specific
    measures vary among the individual programs depending on their unique
    characteristics and requirements. Annual performance reports and PARTs
    contain these specific measures as well as additional information about
    performance goals and targets and whether a program is meeting, or
    making measurable progress toward meeting, its performance goals. The
    examples below illustrate different ways agencies track their
    performance on each dimension.

      Product Quality: Statistical agencies agree that product quality
    encompasses many attributes, including (but not limited to) relevance,
    accuracy, and timeliness. The basic measures in this group relate to the
    quality of specific products, thereby providing actionable information
    to managers. These are ``outcome-oriented'' measures and are key to the
    usability of information products. Statistical agencies or units
    establish targets and monitor how well targets are met. In some sense,
    relevance relates to ``doing the right things,'' while accuracy and
    timeliness relate to ``doing things right.''

                Relevance: Qualitative or quantitative descriptions of the
              degree to which products are useful and responsive to users'
              needs. Relevance of data products and analytic reports may be
              assessed through a professional review process and ongoing
              contacts with data users. Product relevance may be indicated
              by customer satisfaction with product content, information
              from customers about product use, demonstration of product
              improvements, comparability with other data series, agency
              responses to customer suggestions for improvement, new or

    [[Page 42]]

              customized products/services, frequency of use, or responses
              to data requests from users (including policy makers). Through
              a variety of professional review activities, agencies maintain
              the relevance, accuracy, and validity of programs, and
              encourage data users and other stakeholders to contribute to
              the agency's data collection and dissemination program.
              Striving for relevance requires monitoring to ensure that
              information systems anticipate change and evolve to
              appropriately measure our dynamic society and economy.

                Accuracy: Qualitative or quantitative measures of important
              features of correctness, validity, and reliability of data and
              information products measured as degree of closeness to target
              values. For statistical data, accuracy measures include
              sampling error and various aspects of nonsampling error (e.g.,
              response rates, size of revisions, coverage, edit
              performance). For analysis products, accuracy may be the
              quality of the reasoning, reasonableness of assumptions, and
              clarity of the exposition, typically measured and monitored
              through review processes. In addition, accuracy is assessed
              and improved by external and internal reviews, comparisons of
              data among different surveys, linkages of survey data to
              administrative records, redesigns of surveys, or expansions of
              sample sizes.

                Timeliness: Qualitative or quantitative measure of the
              timing of information releases. May be measured as time from
              the collection of data or the close of the reference period to
              the release of information, or customer satisfaction with
              timeliness. May also be measured as how well agencies meet
              scheduled and publicized release dates, expressed as a percent
              of release dates met.

      Program Performance: Statistical agencies agree that program
    performance encompasses balancing the dimensions of cost, dissemination,
    and mission accomplishment for the agency as a whole; operating
    efficiently and effectively; ensuring that customers receive the
    information they need; and serving the information needs of the Nation.
    Costs of products or programs may be used to develop efficiency
    measures. Dissemination involves making sure customers receive the
    information they need via the most appropriate mechanisms. Mission
    achievement means that the information program makes a difference.
    Hence, three key dimensions are being used to indicate program
    performance: cost (input), dissemination (output), and mission
    achievement (outcome).

                Cost: Quantitative measure of the dollar amount used to
              produce data products and services. The development and use of
              financial performance measures within the Federal Government
              is an established goal, and the intent of such measures is to
              determine the ``true costs'' of various programs or
              alternative modes of operation at the Federal level. Examples
              of cost data include full costs of products or programs,
              return on investment, dollar value of efficiencies, and ratios
              of cost to products distributed.

                Dissemination: Qualitative or quantitative information on
              the availability, accessibility, and distribution of products
              and services. Most agencies have goals to improve product
              accessibility, particularly through the Internet. Typical
              measures include: on-demand requests fulfilled, product
              downloads, degree of accessibility, customer satisfaction with
              ease of use, number of participants at user conferences,
              citations of agency data in the media, number of Internet user
              sessions, number of formats in which data are available,
              amount of technical support provided to data users, exhibits
              to inform the public about information products, issuance of
              newsletters describing products, usability testing of
              websites, and assessing compliance with Section 508 of the
              Rehabilitation Act which requires Federal agencies to make
              their electronic and information technology accessible to
              people with disabilities.

                Mission Achievement: Qualitative or quantitative information
              about the impact of or satisfaction with statistical programs.
              For Federal statistical programs, this dimension responds to
              the question--have we achieved our objectives and met the
              expectations of our stakeholders? Under this dimension,
              statistical programs document their contributions to the goals
              and missions of parent departments and other agencies, the
              Administration, the Congress, and information users in the
              private sector. For statistical programs, this broad dimension
              involves meeting recognized societal information needs and
              also addresses the linkage between statistical outputs and
              programmatic outcomes.

                However, identifying this linkage is far from
              straightforward. It is sometimes difficult to trace the impact
              of information products on the public good. Such products
              often are necessary intermediate inputs in the creation of a
              high visibility product whose societal benefit is clearly
              recognized. For example, the economic statistics produced by a
              variety of agencies are directly used by the Bureau of
              Economic Analysis in the calculation of the Gross Domestic
              Product (GDP), which analysts use to assess changes in the
              level of domestic economic activity. Similarly, statistics
              from specific surveys are directly used by the Bureau of Labor
              Statistics in the calculation of the Consumer Price Index
              (CPI), which is widely used in diverse applications, such as
              indexing pensions for retirees. As a result, a number of
              statistical agencies contribute to the GDP and/or the CPI and
              to the many uses of these information products. In addition,
              the data produced by statistical agencies are used to track
              the performance of programs managed by their parent agencies
              or other

    [[Page 43]]

              organizations in areas such as crime, education, energy, the
              environment, health, science, and transportation.

                Moreover, beyond the direct and focused uses of statistical
              products and programs, the statistical agencies and their
              products serve a diverse and dispersed set of data users
              working on a broad range of applications. Users include senior
              government policy makers at the Federal, State, and local
              levels, business leaders, households, academic researchers,
              analysts at public policy institutes and trade groups,
              marketers and planners in the private sector, and many others.
              Information produced by statistical agencies often is combined
              with other information for use in the decision-making process.
              Thus, as with many nonstatistical programs, the relationship
              between statistical program outputs and their beneficial uses
              and outcomes is often complex and difficult to track.

                In the absence of preferred quantitative indicators,
              qualitative narratives can indicate how statistical agency
              products contribute to and evaluate progress toward important
              goals established for government or private programs. In
              particular, narratives can highlight how statistical agencies
              measure the Nation's social and economic structure, and how
              the availability of the information influences changes in
              policies and programs. These narratives contribute to
              demonstrating mission accomplishment, particularly in response
              to questions in Section I of the PART, ``program purpose and
              design.'' Narratives may describe the impact of measuring
              agency policy or change of policy, supporting research focused
              on policy issues, furnishing information to inform debate on
              policy issues, or providing in-house consulting support.

                In addition to narratives, quantitative measures may be used
              to reflect mission achievement. For example, customer
              satisfaction with the statistical agency or unit indicates if
              the agency or unit has met the expectations of its
              stakeholders.

                                                Description of Dimensions

    Product Quality

    Relevance: Qualitative or quantitative description of the degree to which products and
     services are useful to users and responsive to their needs.

    Accuracy: Qualitative or quantitative measure of important features of correctness,
     validity, and reliability of data and information products measured as degree of closeness
     to target values.

    Timeliness: Qualitative or quantitative measure of the timing of information releases.

    [[Page 44]]

                                          Description of Dimensions--Continued

    Program Performance

    Cost: Quantitative measure of the dollar amount used to produce data products and services.

    Dissemination: Qualitative or quantitative information on the availability, accessibility,
     and distribution of products and services.

    Mission Achievement: Qualitative or quantitative information about the impact of, or
     satisfaction with, statistical programs.

    Key to Statistical Agencies

    BEA = Bureau of Economic Analysis, Department of Commerce
    BJS = Bureau of Justice Statistics, Department of Justice
    BLS = Bureau of Labor Statistics, Department of Labor
    BTS = Bureau of Transportation Statistics, Department of Transportation
    Census = Census Bureau, Department of Commerce
    EIA = Energy Information Administration, Department of Energy
    ERS = Economic Research Service, Department of Agriculture
    NASS = National Agricultural Statistics Service, Department of Agriculture
    NCES = National Center for Education Statistics, Department of Education
    NCHS = National Center for Health Statistics, Department of Health and Human Services
    ORES = Office of Research, Evaluation, and Statistics, Social Security Administration
    SOI = Statistics of Income, Internal Revenue Service, Department of the Treasury
    SRS = Division of Science Resources Statistics, National Science Foundation

      Of the 14 principal Federal statistical agencies that are members of
    the ICSP, four agencies have programs that have been assessed using the
    PART process. These agencies' programs have received PART summary
    ratings of Effective or Moderately Effective, as shown in Figure 4-2. As
    additional ICSP agencies have an opportunity to undergo the PART
    process, the agencies plan to use the results of the collaborative
    performance standards development effort to help maintain and extend
    their generally well-received assessments.

         Figure 4-2.  2005 PART SUMMARY RATINGS FOR STATISTICAL PROGRAMS
    ------------------------------------------------------------------------
                                                    Summary Rating
    ------------------------------------------------------------------------
    Bureau of Economic Analysis          Effective

    Bureau of Labor Statistics           Effective

    Census Bureau
      Current Demographic Statistics     Moderately Effective
      Decennial Census                   Moderately Effective
      Intercensal Demographic Estimates  Moderately Effective
      Survey Sample Redesign             Effective

    National Center for Education        ...................................
     Statistics
      Statistics                         Effective
      Assessment                         Effective
    ------------------------------------------------------------------------

                   Highlights of 2005 Program Budget Proposals

      The programs that provide essential statistical information for use by
    governments, businesses, researchers, and the public are carried out by
    some 70 agencies spread across every department and several independent
    agencies. Approximately 40 percent of the funding for these programs
    provides resources for twelve agencies or agency units that have
    statistical activities as their principal mission. (Please see Table 4-
    1.) The remaining funding supports work in 60-plus agencies that carry
    out statistical activities in conjunction with other missions such as
    providing services or enforcing regulations. More comprehensive budget
    and program information about the Federal statistical system will be
    available in OMB's annual report, Statistical Programs of the United
    States Government, Fiscal Year 2005, when it is published later this
    year. The following highlights elaborate on the Administration's
    proposals to strengthen the programs of the principal Federal
    statistical agencies.

      Bureau of Economic Analysis: Funding is requested to complete work
    begun in 2003 to: (1) accelerate the release of some of the Nation's
    most important economic statistics to dramatically increase their
    usefulness to policy makers, business leaders, and other users; (2) meet
    U.S. statistical obligations to international organizations on the
    Special Data Dissemination Standards and complete the incorporation of
    the North American Industry Classification System into BEA accounts; (3)
    improve the economic accounts by acquiring monthly real-time data from
    private sources to fill data gaps in current measures as well as conduct
    a quarterly survey of large and volatile international services such as
    telecommunications, finance, and insurance; and (4) produce more current
    business investment data that include associated employment and
    compensation estimates on an annual basis in order to provide data
    needed to conduct analyses of tax policy, business investment, and
    productivity in manufacturing and service industries.
      Bureau of Justice Statistics: Funding is requested to continue
    conversion of the National Crime Victimization Survey from primarily a
    paper and pencil operation to a fully automated data collection process.
    The BJS base program increase will provide for the maintenance of BJS's
    core statistical programs, including: (1) the National Crime
    Victimization Survey, the Nation's primary source of information on
    criminal victimization; (2) cybercrime statistics on the incidence,
    magnitude,

    [[Page 45]]

    and consequences of electronic and computer crime to households and
    businesses; (3) law enforcement data from over 3,000 agencies on the
    organization and administration of police and sheriffs' departments; (4)
    nationally representative prosecution data on resources, policies, and
    practices of local prosecutors; (5) court and sentencing statistics,
    including Federal and State case processing data; and (6) data on
    correctional populations and facilities from Federal, State, and local
    governments.
      Bureau of Labor Statistics: Funding is requested to support current
    program operations to measure the economy through producing,
    disseminating, and improving BLS economic measures, including: (1)
    modernizing the computing systems for monthly processing of the Producer
    Price Index (PPI) and U.S. Import and Export Price Indexes, and
    producing new data outputs, such as experimental PPI's for goods and
    services that will provide the first economy-wide measures of changes in
    producer prices; (2) maintaining continuous updating of the Consumer
    Price Index (CPI) by updating the expenditure and population weights
    biennially, the superlative index annually, outlet samples on a four-
    year cycle, and item samples in key categories on a two-year cycle, in
    lieu of performing major revisions about every ten years; and (3)
    continuing with a multi-year effort to enhance core BLS information
    technology infrastructure through a central Department of Labor
    appropriation.
      Bureau of Transportation Statistics: Funding is requested to: (1)
    develop the American Freight Data Program, a continuous source of
    freight data from shippers, carriers, and receivers, to replace the
    current five-year Commodity Flow Survey; (2) move the Airfare Price
    Index, an input to GDP and CPI indices, from experimental to production
    mode; and (3) develop more timely and comprehensive local and long-
    distance travel data.
      Census Bureau: Funding is requested for the Census Bureau's economic
    and demographic programs and for a re-engineered 2010 Census. For the
    Census Bureau's economic and demographic programs, funding is requested
    to: (1) support the release of all remaining data products from the 2002
    Economic Census; (2) begin planning for the 2007 Economic Census and
    Census of Governments; (3) continue efforts begun in 2003 to eliminate
    data gaps by measuring migration across U.S. borders; (4) improve
    measurement of services by expanding key source data for critical
    quarterly and annual estimates of our Nation's Gross Domestic Product;
    (5) continue efforts to offer electronic reporting for almost 100
    current economic surveys; and (6) support the Automated Export System
    and accelerate release of trade statistics. For 2010 Census planning,
    funding is requested to continue to: (1) conduct extensive planning,
    testing, and development activities to support a re-engineered 2010
    Census; (2) complete map feature accuracy within 7.6 meters of true GPS
    location for 48 percent of all counties in the U.S., Puerto Rico, and
    island areas; and (3) conduct the first full year of the American
    Community Survey program to provide data on an ongoing basis rather than
    waiting for once-a-decade censuses.
      Economic Research Service: Funding is requested to develop an
    integrated and comprehensive data and analysis framework of the food
    system beyond the farm-gate to provide a basis for understanding,
    monitoring, tracking, and identifying changes in food supply and
    consumption patterns.
      Energy Information Administration: Funding is requested to: (1)
    continue the improvement of natural gas and electricity survey data; (2)
    undertake development work on a liquefied natural gas storage survey and
    a natural gas production survey; (3) enhance the National Energy
    Modeling System's transportation modeling; and (4) revise the Voluntary
    Greenhouse Gases survey to support the President's Initiative on
    Greenhouse Gases.
      National Agricultural Statistics Service: Funding is requested to: (1)
    continue restoration and modernization of the agricultural estimates
    program to ensure State, regional, and national level agricultural
    estimates of sufficient precision, quality, and detail to meet the needs
    of a broad customer base; and (2) support Government-wide and
    departmental E-Government initiatives.
      National Center for Education Statistics: Funding is requested to: (1)
    support the second wave of data collection of the Early Childhood
    Longitudinal Study--Birth Cohort and data release in Spring 2005; (2)
    continue efforts to improve electronic data collection and data
    dissemination; (3) support the ongoing data collection efforts for the
    Schools and Staffing Survey, the principal collection on national and
    State level indicators of teacher and school quality; (4) continue U.S.
    participation in data collections, analyses, and reporting on
    international assessments that compare educational performance and
    progress across countries; and (5) continue support for the National
    Assessment of Educational Progress (NAEP) program and its role in
    benchmarking national and State performance.
      National Center for Health Statistics: Funding is requested to: (1)
    maintain and transform HHS' core health statistics capacity; (2)
    preserve and modernize the Nation's vital statistics system; (3) fortify
    and transform basic operations for the National Health and Nutrition
    Examination Survey; (4) maintain and redesign systems for tracking the
    health care delivery system; and (5) redesign the sample for the
    National Health Interview Survey.
      Science Resources Statistics Division, NSF: Funding is requested to:
    (1) implement ongoing programs on the science and engineering (S&E)
    enterprise; (2)

    [[Page 46]]

    continue implementing quality improvements to surveys on the S&E
    workforce; (3) begin research on methods to implement necessary
    enhancements to the Industry Research and Development survey; (4)
    develop an ongoing data collection program on research instrumentation
    stocks, as mandated by Congress; and (5) continue activities to
    establish an ongoing data series on postdoctorates.
      Statistics of Income Division, IRS: Funding is requested to: (1)
    maintain and modernize core data collection systems, including several
    major statistical programs for the Treasury Department, the
    Congressional Joint Committee on Taxation, the Bureau of Economic
    Analysis, and SOI's many other customers; (2) implement a databank
    repository for SOI and IRS population file data to more efficiently
    build longitudinal databases and enable sub-national estimates; (3)
    examine means to more effectively mask individual records to minimize
    the possibility of identification in the Individual Public Use sample
    files; and (4) modernize and expedite dissemination of data and
    publications, including a reengineered Internet website.

                        Table 4-1.  2003-2005 BUDGET AUTHORITY FOR PRINCIPAL STATISTICAL AGENCIES
                                                (in millions of dollars)
    ----------------------------------------------------------------------------------------------------------------
                                                                                                    Estimate
                                                                                   2003    -------------------------
                                                                                  Actual        2004         2005
    ----------------------------------------------------------------------------------------------------------------
    Bureau of Economic Analysis..............................................          66           67           82

    Bureau of Justice Statistics.............................................          32           32           39

    Bureau of Labor Statistics...............................................         492          518          534

    Bureau of Transportation Statistics......................................          30           31           34

    Census Bureau \1\........................................................         571          632          848
       Salaries and Expenses \1\.............................................         202          213          240
       Periodic Censuses and Programs........................................         369          419          608

    Economic Research Service................................................          69           71           80

    Energy Information Administration........................................          80           81           85

    National Agricultural Statistics Service \2\.............................         138          128          138

    National Center for Education Statistics.................................         184          187          187
       Statistics............................................................          89           92           92
       Assessment............................................................          95           95           95

    National Center for Health Statistics....................................         126          128          150
       PHS Evaluation Funds..................................................         126          128          150
       Budget Authority......................................................           0            0            0

    Science Resources Statistics Division, NSF...............................          31           32           32

    Statistics of Income Division, IRS.......................................          32           36           36
    ----------------------------------------------------------------------------------------------------------------
    \1\ Includes mandatory appropriations of $20 million for each year for the Survey of Program Dynamics and
      collection of data related to the allocation to States of State Children's Health Insurance Program funds.

    \2\ Includes funds for the periodic Census of Agriculture of $41, $25, and $23 million in 2003, 2004, and 2005,
      respectively.