    [Analytical Perspectives]
    [Crosscutting Programs]
    [4. Strengthening Federal Statistics]
    [From the U.S. Government Printing Office, www.gpo.gov]

    [[Page 53]]

                      4.  STRENGTHENING FEDERAL STATISTICS

      Federal statistical programs produce key information about a range of
    topics of interest to public and private decision makers, including the
    economy, the population, agriculture, crime, education, energy, the
    environment, health, science, and transportation. The ability of
    governments, businesses, and citizens to make appropriate decisions
    about budgets, employment, investments, taxes, and a host of other
    important matters depends critically on the ready availability of
    relevant, accurate, and timely Federal statistics. The Federal
    statistical community remains on alert for opportunities to strengthen
    these measures of our Nation's performance. For example, during 2004,
    Federal statistical agencies launched the first new economic indicator
    survey in 40 years--the Quarterly Services Survey (Census Bureau);
    expanded regional economic data from 318 to 934 American communities
    (BEA); successfully adopted new collection and processing technologies
    that will greatly accelerate the release of data from the National
    Health Interview Survey (NCHS); and completed the first data collection
    on the cyberinfrastructure of academic and biomedical facilities (NSF's
    SRS).
      For Federal statistical programs to effectively benefit such a wide
    range of users, the underlying data systems must be viewed as credible.
    In order to foster this credibility, Federal statistical programs seek
    to adhere to high quality standards and to maintain integrity and
    efficiency in the production of statistics. As the collectors and
    providers of these basic data, Federal statistical agencies act as data
    stewards--balancing public and private decision makers' needs for
    information with legal and ethical obligations to minimize reporting
    burden, respect respondents' privacy, and protect the confidentiality of
    the data provided to the Government. This chapter discusses the
    development of standards that principal statistical programs can use to
    assess their performance and presents highlights of their 2006 budget
    proposals.

                              Performance Standards

      Statistical programs maintain the quality of their data or information
    products as well as their credibility by setting high performance
    standards for their activities. The statistical agencies and statistical
    units represented on the Interagency Council on Statistical Policy
    (ICSP) have collaborated on developing an initial set of common
    performance standards for use under the Government Performance and
    Results Act and in completing the Administration's Program Assessment
    Rating Tool (PART). Federal statistical agencies have agreed that there
    are six conceptual dimensions within two general areas of focus that are
    key to measuring and monitoring statistical programs. The first area of
    focus is Product Quality, encompassing the traditional dimensions of
    relevance, accuracy, and timeliness. The second area of focus is Program
    Performance, encompassing the dimensions of cost, dissemination, and
    mission achievement.
      Statistical agencies historically have focused on measuring
    performance in the area of product quality, especially the dimensions
    most amenable to quantitative measurement, specifically accuracy and
    timeliness. Relevance, also an accepted measure of quality, can be
    either a qualitative description of the usefulness of products or a
    quantitative measure such as a customer satisfaction score. Relevance is
    more difficult to measure, and the indicators that do exist are more
    varied.
      Program performance standards form the basis for evaluating
    effectiveness. They address questions such as: Are taxpayer dollars
    spent most effectively? Are products made available to those who need
    them? Are agencies meeting their mission requirements or making it
    possible for other agencies to meet their missions? The indicators
    available to measure program performance for statistical activities
    currently are less well developed.
      Product quality and program performance standards are designed to
    serve as indicators when answering specific questions in the
    Administration's PART process. Chart 4-1 presents each principal Federal
    statistical agency's assessment of the status of its current and planned
    use of indicators on the six dimensions. During the past year, four
    agencies (BTS, EIA, NCES, and SRS) have improved the status of their
    indicators. Use of the indicators may be for internal management,
    strategic planning, or annual performance reporting. The dimensions
    shown in the figure reflect an overall set of indicators for statistical
    activities but the specific measures vary among the individual programs
    depending on their unique characteristics and requirements. Annual
    performance reports and PARTs contain these specific measures as well as
    additional information about performance goals and targets and whether a
    program is meeting, or making measurable progress toward meeting, its
    performance goals. The examples below illustrate different ways agencies
    track their performance on each dimension.

    [[Page 54]]

                                                Description of Dimensions

    Product Quality

    Relevance: Qualitative or quantitative descriptions of the degree to which products and
     services are useful to users and responsive to users' needs.

    Accuracy: Qualitative or quantitative measure of important features of correctness,
     validity, and reliability of data and information products measured as degree of closeness
     to target values.

    Timeliness: Qualitative or quantitative measure of the timing of information releases.

    Program Performance

    Cost: Quantitative measure of the dollar amount used to produce data products and services.

    Dissemination: Qualitative or quantitative information on the availability, accessibility,
     and distribution of products and services.

    Mission Achievement: Qualitative or quantitative information about the effect of, or
     satisfaction with, statistical programs.

    Key to Statistical Agencies

    BEA = Bureau of Economic Analysis, Department of Commerce
    BJS = Bureau of Justice Statistics, Department of Justice
    BLS = Bureau of Labor Statistics, Department of Labor
    BTS = Bureau of Transportation Statistics, Department of Transportation
    Census = Census Bureau, Department of Commerce
    EIA = Energy Information Administration, Department of Energy
    ERS = Economic Research Service, Department of Agriculture
    NASS = National Agricultural Statistics Service, Department of Agriculture
    NCES = National Center for Education Statistics, Department of Education
    NCHS = National Center for Health Statistics, Department of Health and Human Services
    ORES = Office of Research, Evaluation, and Statistics, Social Security Administration
    SOI = Statistics of Income, Internal Revenue Service, Department of the Treasury
    SRS = Division of Science Resources Statistics, National Science Foundation

      Product Quality: Statistical agencies agree that product quality
    encompasses many attributes, including (but not limited to) relevance,
    accuracy, and timeliness. The basic measures in this group relate to the
    quality of specific products, thereby providing actionable information
    to managers. These are ``outcome-oriented''

    [[Page 55]]

    measures and are key to the usability of information products.
    Statistical agencies or units establish targets and monitor how well
    targets are met. In some sense, relevance relates to ``doing the right
    things,'' while accuracy and timeliness relate to ``doing things
    right.''

                Relevance: Qualitative or quantitative descriptions of the
              degree to which products and services are useful and
              responsive to users' needs. Relevance of data products and
              analytic reports may be monitored through a professional
              review process and ongoing contacts with data users. Product
              relevance may be indicated by customer satisfaction with
              product content, information from customers about product use,
              demonstration of product improvements, comparability with
              other data series, agency responses to customer suggestions
              for improvement, new or customized products/services,
              frequency of use, or responses to data requests from users
              (including policy makers). Through a variety of professional
              review activities, agencies maintain the relevance, accuracy,
              and validity of their products, and encourage data users and
              other stakeholders to contribute to the agency's data
              collection and dissemination programs. Striving for relevance
              requires monitoring to ensure that information systems
              anticipate change and evolve to appropriately measure our
              dynamic society and economy.

                Accuracy: Qualitative or quantitative measures of important
              features of correctness, validity, and reliability of data and
              information products measured as degree of closeness to target
              values. For statistical data, accuracy may be defined as the
              degree of closeness to the target value and measured as
              sampling error and various aspects of nonsampling error (e.g.,
              response rates, size of revisions, coverage, edit
              performance). For analysis products, accuracy may be the
              quality of the reasoning, reasonableness of assumptions, and
              clarity of the exposition, typically measured and monitored
              through review processes. In addition, accuracy is assessed
              and improved by internal reviews, comparisons of data among
              different surveys, linkages of survey data to administrative
              records, redesigns of surveys, or expansions of sample sizes.

                Timeliness: Qualitative or quantitative measure of timing of
              information releases. Timeliness may be measured as time from
              the close of the reference period to the release of
              information, or customer satisfaction with timeliness.
              Timeliness may also be measured as how well agencies meet
              scheduled and publicized release dates, expressed as a percent
              of release dates met.

      Program Performance: Statistical agencies agree that program
    performance encompasses balancing the dimensions of cost, dissemination,
    and mission accomplishment for the agency as a whole; operating
    efficiently and effectively; ensuring that customers receive the
    information they need; and serving the information needs of the Nation.
    Costs of products or programs may be used to develop efficiency
    measures. Dissemination involves making sure customers receive the
    information they need via the most appropriate mechanisms. Mission
    achievement means that the information program makes a difference.
    Hence, three key dimensions are being used to indicate program
    performance: cost (input), dissemination (output), and mission
    achievement (outcome).

                Cost: Quantitative measure of the dollar amount used to
              produce data products or services. The development and use of
              financial performance measures within the Federal Government
              is an established goal, and the intent of such measures is to
              determine the ``true costs'' of various programs or
              alternative modes of operation at the Federal level. Examples
              of cost data include full costs of products or programs,
              return on investment, dollar value of efficiencies, and ratios
              of cost to products distributed.

                Dissemination: Qualitative or quantitative information on
              the availability, accessibility, and distribution of products
              and services. Most agencies have goals to improve product
              accessibility, particularly through the Internet. Typical
              measures include: on-demand requests fulfilled, product
              downloads, degree of accessibility, customer satisfaction with
              ease of use, number of participants at user conferences,
              citations of agency data in the media, number of Internet user
              sessions, number of formats in which data are available,
              amount of technical support provided to data users, exhibits
              to inform the public about information products, issuance of
              newsletters describing products, usability testing of web
              sites, and assessing compliance with Section 508 of the
              Rehabilitation Act, which requires Federal agencies to make
              their electronic and information technology accessible to
              people with disabilities.

                Mission Achievement: Qualitative or quantitative information
              about the effect of, or satisfaction with, statistical
              programs. For Government statistical programs, this dimension
              responds to the question--have we achieved our objectives and
              met the expectations of our stakeholders? Under this
              dimension, statistical programs document their contributions
              to the goals and missions of parent departments and other
              agencies, the Administration, the Congress, and information
              users in the private sector and the general public. For
              statistical programs, this broad dimension involves meeting
              recognized societal information needs and also addresses the
              linkage between statistical outputs and programmatic outcomes.

                However, identifying this linkage is far from
              straightforward. It is frequently difficult to trace the
              effects of information products on the public

    [[Page 56]]

              good. Such products often are necessary intermediate inputs in
              the creation of high visibility information whose societal
              benefit is clearly recognized. For example, the economic
              statistics produced by a variety of agencies are directly used
              by the Bureau of Economic Analysis in the calculation of the
              Gross Domestic Product (GDP), which analysts universally use
              to assess changes in the level of domestic economic activity.
              Similarly, statistics from specific surveys are directly used
              by the Bureau of Labor Statistics in the calculation of the
              Consumer Price Index (CPI), which is widely used in diverse
              applications, such as indexing pensions for retirees. As a
              result, a number of statistical agencies can claim credit for
              contributing to the GDP and/or the CPI and to the many uses of
              these information products. In addition, the statistics
              produced by statistical agencies are used to track the
              performance of programs managed by their parent or other
              organizations related to topics such as crime, education,
              energy, the environment, health, science, and transportation.

                Moreover, beyond the direct and focused uses of statistical
              products and programs, the statistical agencies and their
              products serve a diverse and dispersed set of data users
              working on a broad range of applications. Users include
              government policy makers at the Federal, State, and local
              levels, business leaders, households, academic researchers,
              analysts at public policy institutes and trade groups,
              marketers and planners in the private sector, and many others.
              Information produced by statistical agencies often is combined
              with other information for use in the decision-making process.
              Thus, the relationship between program outputs and their
              beneficial uses and outcomes is often complex and difficult to
              track. Consequently, agencies use both qualitative and
              quantitative indicators to make this linkage as explicit as
              feasible.

                In the absence of preferred quantitative indicators,
              qualitative narratives can indicate how statistical agency
              products contribute to and evaluate progress toward important
              goals established for government or private programs. In
              particular, narratives can highlight how statistical agencies
              measure the Nation's social and economic structure, and how
              the availability of the information influences changes in
              policies and programs. These narratives contribute to
              demonstrating mission accomplishment, particularly in response
              to questions in Section I of the PART, ``program purpose and
              design.'' Narratives may describe statistical information's
              effects on measuring agency policy or change of policy,
              supporting research focused on policy issues, informing debate
              on policy issues, or providing in-house consulting support.

                In addition to narratives, quantitative measures may be used
              to reflect mission achievement. For example, customer
              satisfaction with the statistical agency or unit indicates if
              the agency or unit has met the expectations of its
              stakeholders.

      Of the 14 principal Federal statistical agencies that are members of
    the ICSP, six agencies have programs that have been assessed using the
    PART process. Most of these agencies' programs have received PART
    summary ratings of Effective or Moderately Effective, as shown in Chart
    4-2. While recognizing the strength of the Energy Information
    Administration's purpose and management, EIA's PART evaluation found
    that it lacks specific annual performance measures, baselines, and
    targets and should consider enhancing independent expert evaluation of
    its major program areas. EIA is correcting both of these shortcomings,
    which should bring its PART rating into line with those of its sister
    agencies. As additional ICSP agencies have an opportunity to undergo the
    PART process, the agencies plan to continue to use the results of the
    collaborative performance standards development effort to help maintain
    and extend their generally favorable assessments.

      Chart 4-2.  Most Recent PART Summary Ratings for Statistical Programs
    ------------------------------------------------------------------------
                                                    Summary Rating
    ------------------------------------------------------------------------
    Bureau of Economic Analysis          Effective

    Bureau of Labor Statistics           Effective

    Census Bureau
      Current Demographic Statistics     Effective
      Decennial Census                   Moderately Effective
      Economic Census                    Effective
      Intercensal Demographic Estimates  Moderately Effective
      Survey Sample Redesign             Effective

    Energy Information Administration    Results Not Demonstrated

    National Agricultural Statistics     Moderately Effective
     Service

    National Center for Education        ...................................
     Statistics
      Statistics                         Effective
      Assessment                         Effective
    ------------------------------------------------------------------------

                        Table 4-1.  2004-2006 BUDGET AUTHORITY FOR PRINCIPAL STATISTICAL AGENCIES
                                                (in millions of dollars)
    ----------------------------------------------------------------------------------------------------------------
                                                                                                    Estimate
                                                                                   2004    -------------------------
                                                                                  Actual      2005 \1\       2006
    ----------------------------------------------------------------------------------------------------------------
    Bureau of Economic Analysis \2\..........................................          68           73           81

    Bureau of Justice Statistics \3\.........................................          32           34           63

    Bureau of Labor Statistics...............................................         518          529          543

    Bureau of Transportation Statistics......................................          30           30           33

    Census Bureau \4\........................................................         629          765          897
      Salaries and Expenses \4\..............................................         213          216          240
      Periodic Censuses and Programs.........................................         416          549          657

    Economic Research Service................................................          71           74           81

    Energy Information Administration........................................          81           84           86

    National Agricultural Statistics Service \5\.............................         128          128          145

    National Center for Education Statistics.................................         187          185          208
      Statistics.............................................................          92           91           91
      Assessment.............................................................          95           94          117

    National Center for Health Statistics \6\................................          90          109          109

    Science Resources Statistics Division, NSF...............................          31           32           32

    Statistics of Income Division, IRS.......................................          36           39           39
    ----------------------------------------------------------------------------------------------------------------
    \1\ Reflects any recissions.

    \2\ 2005 estimate includes $2 million for a National Academy of Public Administration study of off-shoring.

    \3\ The 2006 estimate includes funds for the Felony Arrestee Drug Use Reporting program (previously funded as
      the Arrestee Drug Abuse Monitoring program within the National Institute of Justice) as well as funds for
      management and administrative costs that were displayed separately in 2004 and 2005.

    \4\ Includes Mandatory Appropriations of $20 million for each year for the Survey of Program Dynamics and
      collection of data related to the allocation to States of State Chidren's Health Insurance Program funds.

    \5\ Includes funds for the periodic Census of Agriculture of $25, $22, and $29 million in 2004, 2005, and 2006,
      respectively. The 2006 Budget includes an increase of $6.5 million due to cyclical activities including
      finalizing content, developing mail lists, and streamlining and upgrading processing systems in preparation
      for the 2007 Census of Agriculture.

    \6\ All funds from the Public Health Service Evaluation Fund. Funds for 2004 are shown comparably with 2005 and
      2006. Administrative costs for NCHS that previously were displayed as part of the NCHS budget line are now
      reflected in two consolidated CDC-wide budget lines for management and administrative costs.

                   Highlights of 2006 Program Budget Proposals

      The programs that provide essential statistical information for use by
    governments, businesses, researchers, and the public are carried out by
    some 70 agencies spread across every department and several independent
    agencies. Approximately 40 percent of the funding for these programs
    provides resources for twelve agencies or units that have statistical
    activities as their principal mission. (Please see Table 4-1.) The
    remaining funding supports work in 60-plus agencies or units that carry
    out statistical activities in conjunction with other missions such as
    providing services or enforcing regulations. More comprehensive budget
    and program information about the Federal statistical system will be
    available in OMB's annual report, Statistical Programs of the United
    States Government, Fiscal Year 2006, when it is published later this
    year. The following highlights elaborate on the Administration's

    [[Page 57]]

    proposals to strengthen the programs of the principal Federal
    statistical agencies.

      Bureau of Economic Analysis: Funding is requested to: (1) make
    selected improvements to the timeliness and comprehensiveness of the
    Nation's international statistics on multinational corporations and
    trade in services; (2) complete work to accelerate the release of gross
    state product, metropolitan personal income, and county-level personal
    income; (3) enhance the accuracy of BEA statistics by acquiring and
    incorporating real-time data into core BEA accounts; (4) improve data on
    international financial transactions by working with the Department of
    the Treasury and the Federal Reserve Board to incorporate newly
    developed estimates of derivatives and other financial instruments; and
    (5) produce up-to-date, annual estimates of business investment spending
    by industry in order to more accurately discern where high-tech and
    other investments are being made in the manufacturing and service
    sectors.
      Bureau of Justice Statistics: Funding is requested to provide for the
    maintenance of BJS's core statistical programs, including: (1) the
    National Crime Victimization Survey, the Nation's primary source of
    information on criminal victimization; (2) cybercrime statistics on the
    incidence, magnitude, and consequences of electronic and computer crime
    to households and businesses; (3) law enforcement data from over 3,000
    agencies on the organization and administration of police and sheriffs'
    departments; (4) nationally representative prosecution data on
    resources, policies, and practices of local prosecutors; (5) court and
    sentencing statistics, including Federal and State case processing data;
    (6) data on correctional populations and facilities from Federal, State,
    and local governments; and (7) the Felony Arrestee Drug Use Reporting
    program (previously funded as the Arrestee Drug Abuse Monitoring program
    within the National Institute of Justice).
      Bureau of Labor Statistics: Funding is requested to support program
    operations to measure the economy through producing, disseminating, and
    improving BLS

    [[Page 58]]

    economic measures, including: (1) modernizing the computing systems for
    monthly processing of the Producer Price Index (PPI) and U.S. Import and
    Export Price Indexes (IPP), and producing new data outputs, such as
    indexes based on the North American Industry Classification System for
    the IPP; (2) maintaining continuous updating of the Consumer Price Index
    (CPI) by updating the expenditure and population weights biennially, the
    superlative index annually, outlet samples on a four-year cycle, and
    item samples in key categories on a two-year cycle; and (3) releasing
    the 2004-2014 Employment Projections and publishing the 2006-2007
    edition of the Occupational Outlook Handbook.
      Bureau of Transportation Statistics: Funding is requested to: (1)
    enhance the Freight Data Program, a continuous source of data from
    shippers, carriers, and receivers, to replace the current Commodity Flow
    Survey; (2) move the Air Transportation Price Index, an input to GDP and
    CPI indices, from experimental to production mode; and (3) develop more
    timely and comprehensive local and long-distance travel data.
      Census Bureau: Funding is requested for the Census Bureau's economic
    and demographic programs, and for a reengineered 2010 Census. For the
    Census Bureau's economic and demographic programs, funding is requested
    to: (1) plan for the 2007 Economic Census, (2) plan and implement the
    organizational phase of the 2007 Census of Governments and plan for the
    employment and finance phases, (3) improve measurement of services by
    expanding key source data for critical quarterly and annual estimates of
    our Nation's Gross Domestic Product, (4) support improved coverage and
    electronic reporting of trade statistics, (5) support the development of
    a database infrastructure to integrate State administrative data and
    Census Bureau data products in order to fill critical data gaps at the
    State and local levels, (6) continue efforts begun in 2003 to eliminate
    data gaps by measuring migration across U.S. borders, and (7) purchase
    furniture and relocate operations and employees to the new headquarters
    facility to avoid disruption of mission-critical operations necessary
    for the successful completion of Census Bureau surveys. For 2010 Census
    planning, funding is requested to continue to: (1) conduct planning,
    testing, and development activities to support a reengineered 2010
    Census; (2) complete map feature accuracy within 7.6 meters of true GPS
    location for 700 of the Nation's counties; and (3) continue to conduct
    the American Community Survey program to provide data on an ongoing
    basis rather than waiting for once-a-decade censuses.
      Economic Research Service: Funding is requested to support ongoing
    programs and to continue the development of an integrated and
    comprehensive data and analysis framework of the food system beyond the
    farm-gate to provide a basis for understanding, monitoring, tracking,
    and identifying changes in the food supply and consumption patterns.
      Energy Information Administration: Funding is requested to continue
    ongoing operations, with a focus on: (1) improving petroleum and natural
    gas data security, reliability, and quality; (2) conducting the
    commercial, manufacturing, and residential energy consumption surveys;
    (3) implementing the enhanced Voluntary Reporting of Greenhouse Gases
    program to support the President's Climate Change Initiative; and (4)
    developing a program performance prototype to assess EIA's data
    collection and operations costs at a more disaggregated level.
      National Agricultural Statistics Service: Funding is requested to: (1)
    continue restoration and modernization of the agricultural estimates
    program to ensure State, regional, and national level agricultural
    estimates of sufficient precision, quality, and detail to meet the needs
    of a broad customer base; (2) continue development and implementation of
    a locality-based agricultural county estimates/small area estimation
    program; and (3) continue preparations for the 2007 Census of
    Agriculture.
      National Center for Education Statistics: Funding is requested to: (1)
    support the ongoing data collection and analysis of the Early Childhood
    Longitudinal Study Birth and Kindergarten Cohorts, which provide data to
    inform child development practices and early education; (2) continue the
    Integrated Education Postsecondary Data System, which collects
    information on enrollment, completions, and finances from postsecondary
    institutions; (3) sustain the ongoing data collection efforts for the
    Beginning Postsecondary Students Longitudinal Study; (4) maintain U.S.
    participation in international assessments that compare educational
    achievement in the United States with that in other countries; and (5)
    continue the National Assessment of Educational Progress (NAEP) program,
    including funding to support the expansion of State NAEP to grade 12.
      National Center for Health Statistics: Funding is requested to: (1)
    increase timeliness by upgrading electronic systems for data collection
    and processing; (2) expand the content of surveys, particularly those
    addressing the health care delivery system; (3) redesign the sample for
    the National Health Interview Survey, NCHS' largest population survey;
    and (4) work collaboratively with States and other agencies on upgrading
    the technology for collecting data from State birth and death
    certificates.
      Science Resources Statistics Division, NSF: Funding is requested to:
    (1) continue to implement the results of prior methodological,
    analytical, and planning activities directed toward improving the
    relevance, accuracy, timeliness, and accessibility of SRS products,
    including the suite of Research and Development surveys and the Survey
    of Graduate Students and Postdoctorates in Science and Engineering; and
    (2) lead a cross-agency effort to examine and revise current

    [[Page 59]]

    taxonomies used for classifying academic fields of study, including the
    development of crosswalks between existing taxonomies and any potential
    new taxonomy, as well as strengthen methods to enhance the
    identification and description of cross-disciplinary and multi-
    disciplinary fields.
      Statistics of Income Division, IRS: Funding is requested to: (1)
    maintain and modernize core data collection systems, including several
    major statistical programs for the Department of the Treasury, the
    Congressional Joint Committee on Taxation, the Bureau of Economic
    Analysis, and SOI's many other customers; (2) implement a databank
    repository for SOI and IRS population file data to more efficiently
    build longitudinal databases and enable sub-national estimates; (3)
    examine means to more effectively mask individual records to minimize
    the possibility of identification in the Individual Public Use sample
    files; and (4) modernize and expedite dissemination of data and
    publications, including a reengineered Internet website.

    [[Page 61]]