Managing for Results: Views on Ensuring the Usefulness of Agency
Performance Information to Congress (Letter Report, 01/26/2000,
GAO/GGD-00-35).

Pursuant to a congressional request, GAO provided information on
ensuring the usefulness of agency performance information to congress,
focusing on: (1) which aspects of congressional information needs were
met by the agency's annual performance plan or some other source; (2)
where those needs were not met, and what accounted for the discrepancies
or gaps in the information provided; and (3) what options agencies could
use to practically and efficiently provide the desired performance
information.

GAO noted that: (1) the congressional staff GAO interviewed identified a
great diversity of information they would like to have to address key
questions about program performance; (2) the agencies GAO studied met
some, but not all, of these recurring and ad hoc congressional
information needs through both formal and informal means; (3) the
congressional staffs were looking for recurring information on spending
priorities within programs, the quality, quantity, and efficiency of
program operations, the populations served or regulated, as well as the
program's progress in meeting its objectives; (4) some of these
recurring needs were met through formal agency documents, such as annual
budget request justification materials, annual performance plans, or
other recurring reports; (5) other congressional information needs were
ad hoc, requiring more detailed information or analysis as issues arose
for congressional consideration; (6) information needs that the
congressional staffs reported as unmet were similar in content to, but
often more specific or detailed than, those that were met; (7) several
factors accounted for the gaps in meeting congressional information
needs; (8) some information the agencies provided did not fully meet the
congressional staffs' needs because the presentation was not clear,
directly relevant, or sufficiently detailed; (9) other information was
not readily available to the congressional staffs; (10) in some cases,
the agencies said they did not have the information because it was
either too soon or too difficult to obtain it; (11) improved
communication between congressional staff and agency officials might
help ensure that congressional information needs are understood, and
that arrangements are made to meet them; (12) greater consultation on
how best to distribute agency documents might improve congressional
access to existing reports; (13) posting publications on Internet sites
can increase congressional staffs' access to agency information without
their having to specifically request it, but staff still need to learn
that the information exists and where to look for it; and (14) agencies'
annual Government Performance and Results Act performance plans and
other reports might be more useful to congressional committees if they
addressed the issues congressional staff said they wanted addressed on a
recurring bases, and if agency staff consulted with the committees on
their choice of performance measures.

--------------------------- Indexing Terms -----------------------------

 REPORTNUM:  GGD-00-35
     TITLE:  Managing for Results: Views on Ensuring the Usefulness of
	     Agency Performance Information to Congress
      DATE:  01/26/2000
   SUBJECT:  Reporting requirements
	     Government information dissemination
	     Program management
	     Performance measures
	     Agency missions
	     Accountability
	     Information resources management
	     Strategic planning
	     Congressional/executive relations
IDENTIFIER:  Government Performance and Results Act
	     GPRA
	     Federal Family Education Loan Program
	     William D. Ford Federal Direct Loan Program
	     HHS Fiscal Year 2000 Performance Plan
	     CDC National Breast and Cervical Cancer Early Detection
	     Program

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO report.  Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved.                 **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
** A printed copy of this report may be obtained from the GAO   **
** Document Distribution Center.  For further details, please   **
** send an e-mail message to:                                   **
**                                                              **
**                                            **
**                                                              **
** with the message 'info' in the body.                         **
******************************************************************

United States General Accounting Office
GAO

Report to the Chairman, Committee on Health,

Education, Labor, and Pensions

U.S. Senate

January 2000

GAO/GGD-00-35

MANAGING FOR RESULTS
Views on Ensuring the Usefulness of Agency

Performance Information to Congress

Ordering Information

The first copy of each GAO report and testimony
is free. Additional copies are $2 each. Orders
should be sent to the following address,
accompanied by a check or money order made out
to the Superintendent of Documents, when
necessary. VISA and MasterCard credit cards are
accepted, also. Orders for 100 or more copies to
be mailed to a single address are discounted 25
percent.

Order by mail:

U.S. General Accounting Office
P.O. Box 37050
Washington, DC 20013

or visit:

Room 1100
700 4th St. NW (corner of 4th and G Sts. NW)
U.S. General Accounting Office
Washington, DC

Orders may also be placed by calling (202) 512-
6000 or by using fax number (202) 512-6061, or
TDD (202) 512-2537.

Each day, GAO issues a list of newly available
reports and testimony. To receive facsimile
copies of the daily list or any list from the
past 30 days, please call (202) 512-6000 using a
touch-tone phone. A recorded menu will provide
information on how to obtain these lists.

For information on how to access GAO reports on
the INTERNET, send e-mail message with "info" in
the body to:

[email protected]

or visit GAO's World Wide Web Home Page at:

http://www.gao.gov

United States
General Accounting Office
Washington, D.C. 20548-0001
Official Business
Penalty for Private Use $300
Address Correction Requested

                    Bulk Rate
               Postage & Fees Paid
                       GAO
                 Permit No. G100

(966713)

Contents
Page 241GAO/GGD-00-35 Performance Information to Congress
Letter                                                                      1
                                                                             
Appendix I                                                                 26
Core Questions to Assist
Program Oversight
                                                                             
Appendix II                                                                27
Comments From the
Department of Health and
Human Services
                                                                             
Appendix III                                                               30
Comments From the
Department of Labor
                                                                             
Related GAO Products                                                       34
                                                                             

     B-282487

Page 21GAO/GGD-00-35 Performance Information to Co
ngress
B-282487

     January 26, 2000

The Honorable James M. Jeffords
Chairman, Committee on Health, Education,
 Labor, and Pensions
United States Senate
 
 Dear Mr. Chairman:

     Congressional committees have cited the need
for a variety of information about federal
programs and their performance to help the
committees make decisions about the programs they
oversee. Seeking to provide Congress and federal
managers with more objective information on the
results of federal programs and thus to improve
government performance and accountability,
Congress enacted the Government Performance and
Results Act of 1993 (GPRA). The act established
governmentwide requirements for agencies to set
strategic and annual performance goals and,
beginning in March 2000, to report annually on
their results in achieving their goals.

     In reviewing agencies' second annual
performance plans (setting goals for fiscal year
2000), the Committee was concerned that the
performance measures selected by some agencies did
not meet the Committee's needs for oversight
information and that it had insufficient input
into some agencies' performance plans. Recognizing
that agencies are still in the early, learning
stages of GPRA implementation, you asked us to
explore, in three case studies, how agencies might
better meet congressional needs for information on
program and agency performance-through whatever
source. The three agencies we studied are
responsible for health surveillance, postsecondary
student loans, and pension oversight.1

     We interviewed staff from these agencies'
authorizing and appropriations committees about
their information needs and whether those needs
were met, and we interviewed agency officials
about how that information might be obtained. As
agreed with your office, this report addresses the
following questions: (1) Which aspects of
congressional information needs were met by the
agency's annual performance plan or some other
source? (2) Where those needs were not met, what
accounts for the discrepancies or gaps in the
information provided? (3) What options could the
agencies use to practically and efficiently
provide the desired performance information?

Results in Brief
The congressional staff we interviewed identified
a great diversity of information they would like
to have to address key questions about program
performance-either on a regular or an ad hoc
basis. The agencies we studied met some, but not
all, of these recurring and ad hoc congressional
information needs through both formal and informal
means. The congressional staffs were looking for
recurring information on spending priorities
within programs; the quality, quantity, and
efficiency of program operations; the populations
served or regulated; as well as the program's
progress in meeting its objectives. For example,
learning who benefits from a program can help in
addressing questions about how well services are
targeted to those most in need. Some of these
recurring needs were met through formal agency
documents, such as annual budget request
justification materials, annual performance plans,
or other recurring reports.

Other congressional information needs were ad hoc,
requiring more detailed information or analysis as
"hot" issues arose for congressional
consideration. The congressional staffs were
seeking more detailed information about program
authority and scope, news of impending change,
assessments of emerging issues, projected effects
of proposed changes, and the effects and side
effects of existing programs. For example, on
several occasions when a negative incident
occurred, they wanted to know how frequently it
occurred, how well the public was protected
against it, and whether a congressional response
was warranted. Some of these needs were met
through both formal and informal means, such as
reports, hearings, agency referrals to other
documents, formal briefings, or informal
consultations.

     Information needs that the congressional
staffs reported as unmet were similar in content
to, but often more specific or detailed than,
those that were met. Several factors accounted for
the gaps in meeting congressional information
needs. Some information the agencies provided did
not fully meet the congressional staffs' needs
because the presentation was not clear, directly
relevant, or sufficiently detailed. For example,
congressional staffs wanted to see more direct
linkages among the agencies' resources,
strategies, and goals. In other cases, the
information was not readily available to the
congressional staffs, either because it had not
been requested or reported, or because staff were
not informed that it was available. Some of the
authorizing committee staff had not seen the
annual performance plan that one agency had
prepared separately from its department's plan,
because the agency had not submitted it to that
committee. Although all three agencies placed
documents on their Internet sites, they did not
typically inform congressional staff about them
unless they specifically asked for the
information. Finally, in some cases, the agencies
said they did not have the information-such as the
use of new student loan repayment options-because
it was either too soon or too difficult to obtain
it. They said that special studies would be
required to obtain data not contained in program
records or to assess long-term effects.

     These experiences suggest a variety of
options for increasing the congressional staffs'
access to the information they need to address key
policy questions about program performance,
depending on the circumstances. Foremost, improved
communication between congressional staff and
agency officials about those needs might help
ensure that congressional information needs are
understood, and that, where feasible, arrangements
are made to meet them. Agency officials indicated
that improved communication could have avoided
some of these unmet needs; they believed that, if
requested to do so, they could have provided, or
arranged to obtain, most of the information the
congressional staff wanted. Although the
congressional staffs were able to obtain answers
to many of their inquiries by going through
official agency channels, a few noted that
informal discussions, when they occurred, were
particularly helpful. Improved two-way
communication might also make clear what
information is and is not available.

     Greater consultation on how best to
distribute agency documents might improve
congressional access to existing reports. Posting
publications on Internet sites can increase
congressional staffs' access to agency information
without their having to specifically request it,
but staff still need to learn that the information
exists and where to look for it. Depending on the
circumstances, agencies may want to broadly
distribute some documents of general interest and
simply alert key contacts within the committees to
new publications and events of more specific
potential interest.

     The agencies' annual GPRA performance plans
and other reports might be more useful to
congressional committees if they addressed the
issues congressional staff said they wanted
addressed on a recurring basis, and if agency
staff consulted with the committees on their
choice of performance measures. Developing
performance plans for major bureaus or programs
might help clarify the links between their
resources, strategies, and goals at the program
level-the level at which these committee staffs
were interested. Agencies could opt either to
incorporate this information in their departmental
performance plans or to refer readers to
supplementary documents. However, without feedback
from congressional staff on where presentations
were unclear, or where additional detail or
content was desired, future reports might still
not meet congressional information needs.

     Finally, to obtain new information about
subpopulations or emerging issues, congressional
staff would have to make direct requests of the
agency, and the agency officials we interviewed
said they welcomed these requests. However, as we
stated in a previous report, two-way communication
is critical in obtaining special studies, to
ensure a mutual understanding of information needs
and how they can be met.2

Background
In 1995, we reported on a study of how three
agencies collected and reported evaluative
information about their programs to this
Committee.3 We found that the agencies collected a
great deal of useful information about their
programs, but much of it was not requested and
thus did not reach the Committee, and much of what
the Committee did receive was not as useful as it
could have been. We also found that communication
between the Committee and agency staff on
information issues was limited and afforded little
opportunity to build a shared understanding of the
Committee's needs and how to meet them. At that
time, we proposed a strategy for obtaining
information to assist program oversight and
reauthorization review: (1) select descriptive and
evaluative questions to be asked about a program
at reauthorization and in interim years, (2)
explicitly arrange to obtain oversight information
and results of evaluation studies at
reauthorization, and (3) provide for increased
communication with agency program and evaluation
officials to ensure that information needs are
understood and requests and reports are suitably
framed.

At the time, GPRA had recently been enacted,
requiring agencies to develop multiyear strategic
plans and annual performance plans and reports
over a 7-year implementation period. In our 1995
report, we noted that annual reporting under GPRA
was expected to fill some of the information gaps
we described and that GPRA also emphasized the
importance of consultation with Congress as
evaluation strategies are planned, goals and
objectives are identified, and indicators are
selected. We suggested that our proposed process
for identifying questions would be useful as
agencies prepared to meet GPRA requirements and
that consultation with Congress would help ensure
that data collected to meet GPRA reporting
requirements could also be used to meet the
Committee's special needs (for example, to
disaggregate performance data in ways important to
the Committee). We also saw a need for a useful
complement to GPRA reports (and their focus on
progress towards goals) that would provide
additional categories of information, such as
program description, side effects, and comparative
advantage to other programs. The Committee had
found such information to be useful, especially in
connection with major program reauthorizations and
policy reviews.

Since its enactment, we have been tracking federal
agencies' progress in implementing GPRA by
identifying promising practices in performance
measurement and results-based management, as well
as by evaluating agencies' strategic plans and the
first two rounds of performance plans.4 We found
that although agencies' fiscal year 2000
performance plans, on the whole, showed moderate
improvements over the fiscal year 1999 plans, key
weaknesses remained and important opportunities
existed to improve future plans to make them more
useful to Congress. Overall, the fiscal year 2000
plans provided general, rather than clear,
pictures of intended performance, but they had
increased their use of results-oriented goals and
quantifiable measures. Although some agencies made
useful linkages between their budget requests and
performance goals, many needed to more directly
explain how programs and initiatives would achieve
their goals. Finally, many agencies offered only
limited indications that their performance data
would be credible, a source of major concern about
the usefulness of the plans.

     This report does not directly evaluate the
three agencies' performance plans but rather looks
more broadly at the types of information that
authorizing and appropriations committees need
from the agencies and how their unmet needs could
be met, either through performance plans or
through other means. We included program
performance information available from sources
other than annual performance plans because
agencies communicate with congressional committees
using a variety of modes-reports, agency Internet
sites, hearings, briefings, telephone
consultations, e-Mail messages, and other means.
We did not assume that annual GPRA performance
plans or performance reports are the best or only
vehicle for conveying all kinds of performance
information to Congress.

     We conducted our work between May and
November 1999 in accordance with generally
accepted government auditing standards. We
requested comments on a draft of this report from
the Secretaries of Education, Labor, and Health
and Human Services and the Director of the Office
of Management and Budget. HHS and Labor provided
written comments that are reprinted in appendixes
II and III. The other agencies either had no
comments or provided technical comments. The
agencies' comments are discussed at the end of
this letter. We also requested comments from the
congressional staff members we interviewed on our
characterization of their concerns, and we
incorporated the clarifying changes they
suggested.

Program Descriptions
     Health Surveillance. The Centers for Disease
Control and Prevention (CDC) in the Department of
Health and Human Services (HHS) supports-through a
number of programs-a system of health surveillance
activities to monitor, and help prevent and
control, infectious and chronic diseases. By
working with the states and other partners,
CDC-primarily the National Center for Infectious
Diseases and the National Center for Chronic
Disease Prevention and Health Promotion-provides
leadership and funding through grants to state and
local public health departments. Grants support
research to develop diagnostic tests, prevention
interventions, local and state public health
laboratories, and information sharing and other
infrastructure to facilitate a nationwide
surveillance system. CDC centers support critical
disease registries (such as the cancer registries)
and surveillance tools (such as the Behavioral
Risk Factor Survey) and disseminate public health
surveillance data.

Pensions Oversight. In the Department of Labor
(DOL), the Pension and Welfare Benefits
Administration (PWBA) oversees the integrity of
private sector pensions (as well as health and
other welfare benefits) and seeks to increase
employer-sponsored pension coverage in the
workforce. The Employee Retirement Income Security
Act (ERISA) sets minimum standards to ensure that
private employee pension plans are established and
maintained in a fair and financially sound manner.
Employers also have an obligation to provide
promised benefits and to satisfy ERISA
requirements for managing and administering
private pension plans. PWBA tracks and collects
annual reports by plan managers on the plan
operations, funding, assets, and investments. It
develops regulations and conducts enforcement
investigations and compliance reviews to deter
pension fund mismanagement. PWBA also provides
information and customer assistance, such as
brochures targeted to women, small businesses, and
minorities with low participation rates in pension
plans, to encourage the growth of employment-based
benefits.

Postsecondary Student Loans. The Department of
Education's Office of Student Financial Assistance
(OSFA), a newly created performance-based
organization, manages operations of the direct
loan program (William D. Ford Federal Direct
Student Loan Program) and guaranteed loan program
(Federal Family Education Loan Program) that are
major student financial assistance programs.5
These and other programs under the Higher
Education Act of 1965, as amended, aim to help
undergraduate and graduate students meet the cost
of their education. The agency provides loans to
students (or families) either directly through the
direct loan program or under the guaranteed loan
program, through private banks that lend the money
at a federally subsidized rate.

In the direct loan program, the student applies
through the school to the agency that transfers
funds to the school. Later, a loan servicer (under
agency contract) tracks and collects payments on
the loan. In the guaranteed loan program, the
student applies for the loan through a private
lender that then tracks and collects the loan
payments. The agency subsidizes the interest rate
paid by the borrower. If a borrower defaults, a
local guaranty agency reimburses the bank for the
defaulted loan, and the department pays the
guaranty agency.

Some Congressional Information Needs Were Met
Through Formal and Informal Means
     Congressional staff identified a great
diversity of information they wanted to have to
enable them to address key questions about program
performance-either on a regular basis, to answer
recurring questions, or in response to ad hoc
inquiries as issues arose. Agencies met some, but
not all, of these information needs through a
variety of formal and informal means, such as
formal reports and hearings and informal
consultations.

Formal Annual Reports Met Some Recurring
Information Needs
     Congressional staff identified a number of
recurring information needs, some of which were
met through annual documents, such as agencies'
budget justification materials, GPRA annual
performance plans, or other annual reports. The
recurring information needs fell into four broad
categories:

ï¿½    allocation of program personnel and
expenditures across activities;
ï¿½    data on the quantity, quality, and efficiency
of operations or services;
ï¿½    characteristics of the populations or
entities served or regulated; and
ï¿½    indicators of progress in meeting objectives
and side effects.

Both authorizing and appropriations staff wanted
regular information on how personnel and
expenditures were allocated across activities,
both for the purpose of learning what was actually
spent on a program or activity as well as to
understand priorities within a program. This
information was typically provided to their
appropriations committees in the detailed budget
justification documents that agencies submit each
year with their budget requests. An appropriations
staff member indicated that the routine data he
wanted on PWBA's program staffing and expenditures
were provided by the agency's budget justification
documents, and that the agency was forthcoming in
responding to requests for additional information.

Congressional staff also described wanting
information on the quantity, quality, and
efficiency of the activities or services provided.
This information was needed to inform them of the
nature and scope of a program's activities, as
well as to address questions about how well a
program was being implemented or administered.
They said they found this kind of information in
both agency budget justification documents and
performance plans. For example, both authorizing
and appropriations staff members noted that the
Department of Education's budget justification
documents and its departmental performance plan
met their needs for basic information on trends in
program expenditures and the volume and size of
student loans and grants-in-aid over time. This
data provided them with information about the
change over time in the use of different financing
options, revealing the potential for an increase
in student debt burden.

In addition, the department's performance plan
included performance indicators and targets for
OSFA's response times in processing loan
applications, an issue of concern to congressional
staff because backlogs in loans being consolidated
under the direct loan program had been identified
and targeted for increased attention. In this
case, Education officials said that a committee
report required a biweekly report for 18 months on
its loan processing so that the committee could
monitor their progress in resolving the backlog.
Officials said that this report was provided to a
total of six committees-the authorizing,
appropriations, and budget committees-in both the
Senate and House. All three agencies also
described their major programs (with some
information on program activities and services
provided) on their agency Internet sites.6

Similarly, congressional staff also wanted regular
information on the characteristics of the persons
or entities the programs serve or regulate. In
addition to providing a picture of who benefits
from the program, such information can help answer
questions about how well program services are
targeted to the population most in need of service
and how well those targeted populations are
reached. The congressional staff described PWBA as
good at providing statistics on the private
pension plans and participants covered by ERISA in
an annual report issued separately from the GPRA
requirements. This report, the Private Pension
Plan Bulletin, provides their most recent as well
as historical data on plans and participants and
detailed data on employee coverage and other
characteristics by employer size.

     Finally, the congressional staff also wanted
regular information on the program's progress in
meeting its objectives and any important side
effects that the program might have. The
Department of Labor's fiscal year 2000 performance
plan supplied information on one of PWBA' s
goals-to increase the number of employees covered
by private pension plans-derived from a survey
conducted by the Bureau of the Census (Census).
Congressional staff noted their satisfaction with
the inclusion of program data on the student loan
default rate and default recovery rate as
performance measures in the Department of
Education's performance plan. The plan also
provided data on whether low- and middle-income
students' access to postsecondary education was
improving over time relative to high-income
students' access. These and other measures in the
plan of unmet need for student financial aid,
college enrollment rates, and size of debt
repayments were derived from special surveys
conducted by the Department of Education or by
Census.

Some Ad Hoc Information Needs Were Met Through
Formal Reports and Informal Consultation
     Congressional staff identified a number of ad
hoc information needs that arose periodically as
"hot issues" came up for congressional
consideration. Some of the needs were met through
existing documents, and many others through
informal consultations in response to a request
from congressional staff, while still other needs
were not met. The ad hoc information needs were
similar to but somewhat different from recurring
information needs and fell into five broad
categories:

ï¿½    details about a program's activities and
authority,
ï¿½    news of impending change in the program,
ï¿½    assessments of emerging issues,
ï¿½    projected effects of proposed program
changes, and
ï¿½    effects and side effects of existing
programs.

 Congressional staff often wanted details about the
scope of a program's activities and authority that
were not readily available from the general
documents they had. Questions might have been
raised by a constituent request or a legislative
proposal, in which case the staff member wanted a
fairly rapid response to a targeted question. In
such cases, congressional staff said they often
called the agency's congressional liaison office,
which either handled the request itself or
forwarded it to knowledgeable program officials
who, in turn, either returned the call to the
requester or forwarded the information through the
liaison. CDC officials also described referring
requesters to the brief program descriptions they
maintain on their Internet site.

     Congressional staff noted that they wanted
the agency to proactively inform them, in advance,
when there was news of significant impending
change in their Member's district or to a program
in which they had been involved. In one case, they
wanted to have an opportunity to influence the
policy discussions; in another case, they wanted
to be prepared when the news appeared in the
press. An authorizing committee staff member found
that CDC's targeted distribution of "alerts"
provided a very useful "heads up" before the
agency issued a press release about a public
health concern. The alerts were distributed by e-
Mail or faxed to the interested committee staff
member or congressional members. During the recent
appearance of a rare form of encephalitis in New
York City, for example, CDC said that it informed
congressional members and interested staff members
from that region (as well as their authorizing and
appropriations committees) about its findings
regarding the source of the disease and explained
what CDC was doing about it.

     Another type of ad hoc information request
was for assessments of an issue's potential
threat. Congressional staff described several
occasions when a negative incident-such as a
disease outbreak-occurred that raised questions
about how frequently such incidents occur, how
well the public is protected against them, and
whether a congressional or legislative response
was warranted. Because of the highly specific
nature of such requests, the staff said they were
usually made by telephone to the agency's
congressional liaison and responded to with a
brief, informal consultation or a formal briefing.

     On one occasion, CDC officials testified at a
congressional hearing summarizing their research
into antimicrobial-resistant diseases and how
CDC's surveillance programs track and respond to
the problem. In another example, in response to a
proposed merger of two large private corporations,
a staff member wanted to know what the new owner's
obligations were to its holdover employees and how
this would affect those employees' pension
benefits. In addition, in order to ensure the
protection of those employees' rights, the staff
member wanted to know what enforcement options
were available to the agency. The staff member
indicated that PWBA officials provided this
technical assessment and consultation in a timely
manner.

     As either the legislative or executive branch
proposed changes to a program, congressional staff
wanted projections of the effects of those
proposed changes, not only as to whether (and how)
the change would fix the problem identified, but
also whether it would have undesired side effects.
As committee staff discuss proposals, they said
they often asked agency officials for informal
consultations. If hearings or other more formal
deliberations were planned, some kind of formal
document might be requested. When an agency
proposed a regulation or amended regulation, the
agency prepared a formal document for public
comment that provided a justification for the
change. For example, to reduce the cost of loans
to student borrowers, a congressional committee
considered reducing the interest rate. However,
some lenders expressed concern that a rate
reduction would cut into their profit margins,
forcing some to drop out of the program. To assess
the likelihood of this projected result, the
committee staff turned to the estimates of
lenders' profit margins produced by the Office of
Management and Budget (OMB) and the Treasury
Department.

     Similarly, as new provisions are implemented,
congressional staff might have questions about
whether the provisions are operating as planned
and having the effects hoped for or the side
effects feared. In December 1998, OSFA was
designated a performance-based organization (PBO),
given increased administrative flexibility, and
charged with modernizing the Department's
information systems and improving day-to-day
operations. OSFA has provided authorizing and
appropriations committee staff with regular
reports on its Interim Performance Objectives
(also available on its Internet site) that provide
measures of efficiency in processing loan and loan
consolidation applications and measures of
borrower and institutional satisfaction. OSFA has
also initiated cost accounting improvements to
obtain better data on loans made, serviced, and
collected under both the direct and guaranteed
loan programs in order to provide baseline data
against which to measure its progress in improving
operational efficiency.

Several Factors Accounted for Gaps in Meeting
Congressional Information Needs
Information needs that congressional staff
reported as unmet were similar in content to, but
often more specific or detailed than, those that
were met. The information needs that congressional
staff described as having been met tended to be
general, descriptive information about a program's
activities and expenditures (such as those that
might support their budget request) or descriptive
information about the agency's activities in
response to a specific, often emerging, issue.
This information was often provided in a formal
report or presentation (such as a briefing or
hearing). The information needs that congressional
staff described as typically unmet were detailed
information on the allocation of funds for
activities, descriptive information about the
program's strategies and the issues they
addressed, and analyses showing the program's
effects on its objectives.

The key factors accounting for the gaps in meeting
congressional information needs were the
following:

ï¿½    the presentations of information were not
clear, sufficiently detailed, or directly
relevant;
ï¿½    the information was not readily available to
congressional staff; or
ï¿½    the information was not available to the
agency.

Some Information Presentations Were Not Clear,
Detailed Enough, or Directly Relevant
In some cases, information on the topics was
available or provided, but its presentation was
not as useful as it could have been. Congressional
staff members noted that neither the budget
submission nor the departmental strategic plan
demonstrated the link between a CDC cancer
screening program, the dollars appropriated for it
in the budget, and how this program contributed to
meeting the department's strategic objectives. A
CDC official noted that, in combination, CDC's
performance plan and budget submission did link
the strategic objectives with the budget. They
explained that this was in part due to CDC's
budget being structured differently from its
organization of centers and institutes. A CDC
budget work group, formed in early 1999 in
response to similar concerns, met with its
congressional stakeholders and program partners
and is developing a revised budget display that
the group hopes will make this information more
understandable in CDC's next budget submission.

In another situation, congressional staff looked
to the performance plan for a clear presentation
of PWBA's regulatory strategy that showed how the
agency planned to balance its various
activities-litigation, enforcement, guidelines,
regulations, assistance, and employee
education-and how those activities would meet
PWBA's strategic goals. The congressional staff
wanted to know what PWBA's regulatory priorities
were, as well as how PWBA expected the different
activities to achieve its goals. However, the
departmental plan did not provide a comprehensive
picture of PWBA and described only isolated PWBA
activities to the extent that they supported
departmental goals.

     Some agency reports did not provide enough
detail on issues of concern to the committee.
Congressional staff members concerned about PWBA's
enforcement efforts wanted detailed information on
the patterns of violations to show how many were
serious threats to plans and their financial
assets, rather than paperwork filing problems. A
PWBA official indicated that PWBA could
disaggregate its data on violations to show the
distribution of various types of violations, but
that there would need to be some discussion with
the committee staff about what constituted a
"paperwork" rather than a "serious" violation.7

     In another case, a congressional staff member
was concerned that some patients were experiencing
significant delays in obtaining cancer treatment
after being screened under the National Breast and
Cervical Cancer Early Detection Program. The
program focuses on screening and diagnosis, while
participating health agencies are to identify and
secure other resources to obtain treatment for
women in need. Staff wanted to see the
distribution of the number of days between
screening and beginning treatment, in addition to
the median period, in order to assess how many
women experienced significant delays. When this
issue was raised in a hearing, CDC officials
provided the median periods as well as the results
of surveillance data that showed that 92 percent
of the women diagnosed with breast cancer and
invasive cervical cancer had initiated treatment.

Some responses to congressional inquiries were not
adequately tailored to meet congressional staff's
concerns. For example, in preparing legislation, a
congressional staff member needed immediately very
specific information about the scope and authority
of a program in order to assess whether a proposed
legislative remedy was needed. However, he said he
received documents containing general descriptive
information on the issue instead, which he did not
consider relevant to his question. An agency
official indicated that this response suggested
that the congressional query may not have been
specific enough, or that the responding agency
official did not have the answer and hoped that
those documents would satisfy the requestor. In
other cases, staff indicated they obtained this
type of information succinctly through a telephone
call to the agency's congressional affairs office,
which might direct them to a brief description of
the program's authority, scope, and activities on
the agency's Internet site or refer them to a
knowledgeable agency official.

One authorizing committee staff person noted that,
although the committee staff assigned to an issue
develops background on these programs over time,
there is rapid turnover in Members' staff
representatives to a committee. Moreover, because
these staff are expected to cover a broad range of
topics, she thought that they would find
particularly useful brief documents that
articulate the program's authority, scope, and
major issues, to draw upon as needed.

Some Information Was Not Readily Available
Some congressional information needs were unmet
because the information was not readily available,
either because it was not requested or reported,
or because staff were not informed that it was
available. In one instance, concerned about the
safety of multiemployer pension plans,
congressional staff wanted disaggregated data on
the results of enforcement reviews for that type
of plan. PWBA officials explained that the ERISA
Annual Report to Congress does not highlight
enforcement results for particular types of plans.
However, they said that they could provide this
information if congressional staff specifically
requested it.

In several cases, the agencies thought that they
had made information available by placing a
document on the agency's Internet site, but they
had not informed all interested committee staff of
the existence or specific location of those
documents. For instance, an authorizing committee
staff member had heard of long delays in PWBA's
responses to requests for assistance and wanted to
know how frequently these delays occurred. In its
own agency performance and strategic plans, PWBA
included performance measures of its response
times to customers requesting assistance and
interpretations. But, because those measures were
not adopted as part of the departmental
performance plan and PWBA did not provide its own
performance plan to the authorizing committee
staff, this information was not available to those
staff. Agency officials said that this information
was available because they had posted their
strategic plan on the agency's Internet site.
However, the committee staff person was unaware of
this document's presence on the site and thus was
unaware that such a measure existed.

Some Information Was Not Available to the Agency
In some instances, the desired information was not
available to the agency. This was because either
special data collection was required, it was too
early to get the information, the data were
controlled by another agency, or some forms of
information were difficult to obtain.

Where congressional questions extend across
program or agency boundaries, special studies,
coordinated at the department level, might be
required to obtain the answers. For example, to
address a policy question about how well prenatal
services were directed to pockets of need,
congressional staff wanted a comparison of the
geographic distribution of the incidence of low
birth-weight babies with areas served by prenatal
programs and with the availability of ultrasound
testing. HHS officials explained that although CDC
and the National Center for Health Statistics had
information on the regional incidence of low birth-
weight babies through birth certificate data,
these agencies did not have the information on the
availability of prenatal services. The Health
Resources and Services Administration (another HHS
agency), which is concerned with such services,
does not have information on the location of all
prenatal programs or the availability of
ultrasound equipment to link with the birth
certificate data on low birth-weight. HHS
officials indicated that, if this analysis were
requested, the department would need to initiate a
special study to collect data on the availability
of services to match with existing vital
statistics.

Some congressional information needs extend beyond
what a program collects as part of its operations
and thus would require supplemental information or
a special data collection effort to obtain. For
example, because a student's race is not collected
as part of loan applications, the Department of
Education supplements its own records on the use
of different student finance options with periodic
special studies of student borrowers that do
collect racial information. Because the different
student loan programs maintain their records in
separate databases, the office relies on special
studies, conducted every 3 years since school year
1986-1987, to examine the full package of
financial options students and their families use
to pay for postsecondary education. The
congressional staff also wanted to obtain trend
data on the extent to which all forms of student
aid received (e.g., grants, loans, and tax
credits) cover the cost of school attendance for
low-income students. Education officials said that
if published data from these special studies were
not adequate, specialized data tabulations could
be obtained. In the meantime, OSFA issued a 5-year
performance plan in October 1999 that showed how
it plans to improve the information systems for
the student loan programs in order to improve
operations and interconnectivity among the
programs.

As programs are revised, questions naturally arise
about whether the new provisions are operating as
planned and having the desired effects or unwanted
side effects. Congressional staff identified
several questions of this type for the student
loan programs due to changes created by the 1998
reauthorization of the Higher Education Act and
the separate enactment of a new tuition tax
credit: How many students will select each of the
new loan repayment options? Which students benefit
more from the new tax credit, low- or middle-
income? Will the need to verify a family's
educational expenses create a new burden for
schools' financial aid offices?

In our discussions with OSFA, officials told us
that they will report information on use of the
new repayment options in their next annual budget
submission, and that they believed the Internal
Revenue Service (IRS) would include analyses of
who used the tuition tax credit (similar to its
analyses of other personal income tax credits) in
its publication series, Statistics of Income.
Because OSFA does not administer the tax credit,
OSFA officials suggested to us that IRS would be
responsible for estimates of any reporting burden
for schools related to the tax credit.

     Lastly, some information was not available
because it is difficult to obtain. There has been
congressional interest in whether a provision that
cancels loan obligations for those who enter
public school teaching or other public service
leads more student borrowers to choose public
service careers. Education officials said that a
design for a special evaluation had been prepared,
but that they had discovered that, because only a
small number of student borrowers benefited from
this provision, they were unable to obtain a
statistically valid sample of these borrowers
through national surveys.

     Determining the effectiveness of federally
funded state and local projects in achieving
federal goals can be challenging for federal
agencies. A CDC official told us that CDC conducts
many studies evaluating whether a specific health
prevention or promotion practice is effective or
not, but that it expects it will take a
combination of such practices to produce
populationwide health effects. However, it is much
more difficult to measure the effects of a
combination of practices, especially when such
practices are carried out in the context of other
state and local health initiatives, than to test
the efficacy of one specific health practice at a
time. In addition, measuring the effectiveness of
health promotion and disease prevention programs
related to chronic disease can be difficult in the
short term, given the nature of chronic diseases.

Improved Communication Offers Several Options for
Filling Unmet Information Needs
To help ensure that congressional stakeholders
obtain the information they want requires
communication and planning-to understand the form
and content of the desired information as well as
what can feasibly be obtained, and to arrange to
obtain the information. Our analysis uncovered a
range of options that agency and congressional
staffs could choose from-depending on the
circumstances-to improve the usefulness of agency
performance information to these congressional
staffs. Improved communication might help increase
congressional access to existing information,
improve the quality and usefulness of existing
reports, and plan for obtaining supplemental data
in the future.

Options for Increasing Access to Existing
Information
     Agency officials said that increased
communication between agency and congressional
staff could have prevented some of the unmet
information needs because they believed that, if
requested, they could have provided most of the
information congressional staff said they wanted,
or arranged for the special analysis required.
Increased two-way communication might also make
clear what information is and is not available.

     Each agency has protocols for communication
between congressional staff and agency officials,
typically requiring the involvement of
congressional liaison offices to ensure
departmental review and coordination of policy.
Agency congressional liaisons and other officials
said that they answered some ad hoc inquiries
directly or referred congressional staff to
existing documents or program specialists.
Congressional staff said that they were generally
able to get responses to their formal and informal
inquiries through these channels, but several
noted that communication was often very formal and
controlled in these settings.

     Some congressional staff and agency officials
found that the informal discussions they had had
were very helpful. In one case, agency officials
were asked to discuss their program informally
with appropriations committee staff; in another
case, the incoming agency director scheduled a
visit with a subcommittee chair and his staff to
describe his plans and learn of their interests.
It is our opinion that when key agency or
committee staff changes occur, introductory
briefings or discussions might help ensure
continuity of understanding and open lines of
communication that could help smooth the process
of obtaining information on a recurring and on an
ad hoc basis.

Discussion of what might be the most appropriate
distribution options for different types of
documents might help ensure that the information
agencies make available is actually found. For
example, authorizing committees might want to
routinely receive agencies' annual budget
justification documents, which contain detailed
information on allocations of resources. Also,
although the three agencies aimed to increase the
volume of material that was publicly available by
posting it on their Internet sites, the
information was often not available to
congressional staff unless they knew that it
existed and where to look for it. For relatively
brief and broadly applicable material, like CDC's
summary of cost-effective health promotion
practices, an agency may decide, as CDC did, to
send copies to all congressional offices.
Alternatively, to avoid overwhelming congressional
staffs with publications, CDC officials sent e-
Mail or fax alerts to contacts at relevant
committees about newly released publications and
other recent or upcoming events of potential
interest.

Options for Improving GPRA and Other Existing
Reports
Our analysis of the types of information the
congressional staffs said they wanted on a
recurring basis suggests ways the agencies might
improve the usefulness of their performance plans
and other reports to these committees. In
addition, increased communication about the
specifics of congressional information needs might
help ensure that those needs are understood and
addressed.

The congressional staff said that they wanted a
clear depiction at the program level of the
linkages between program resources, strategies,
and the objectives they aim to achieve. Of our
three case studies, congressional staff indicated
that only the Education Department's performance
plan provided adequate detail at the program
level-the level that they were interested in. As
we previously reported, most federal agencies'
fiscal year 2000 plans do not consistently show
how the program activity funding in their budget
accounts would be allocated to agencies'
performance goals.8 And, although most agencies
attempted to relate strategies and program goals,
few agencies indicated how the strategies would
contribute to accomplishing the expected level of
performance.

One option would be for agencies to consider
developing performance plans for their major
bureaus or programs and incorporating this
information in their department's plan. For
example, the HHS Fiscal Year 2000 Performance Plan
consisted of a departmentwide summary as well as
the annual performance plans developed by its
component agencies and submitted as part of the
agencies' budget justifications. Alternatively,
departments that prefer to submit a consolidated
plan keyed to departmentwide goals could refer
readers to where more specific data could be found
in supplementary documents. OMB's Circular No. A-
11 guidance asks agencies to develop a single plan
covering an entire agency but notes that, for some
agencies, the plan will describe performance on a
macro scale by summarizing more detailed
information available at different levels in the
agency. In these instances, OMB instructs agencies
to have ready their more detailed plans specific
to a program or component to respond to inquiries
for more refined levels of performance
information.

The congressional staff also said that they
wanted, on a recurring basis, data on the
quantity, quality, and efficiency of a program's
activities; the characteristics of the population
served; and indicators of a program's progress in
meeting its objectives. These categories are
consistent with those identified in our 1995
report as the information Congress wants on a
routine basis.9 (Appendix I contains the
categories of information and the list of core
questions that we proposed committees select from
and adapt to meet their needs when requesting
information.) Although all three agencies
consulted with congressional committees on their
strategic plans as required by GPRA, only one
consulted with our congressional interviewees on
the development of its performance plan and choice
of indicators. As we previously reported, agency
consultation with both authorizing and
appropriations committees as performance measures
are selected is likely to make the agencies'
performance plans more useful to those committees.

The three agencies' planned and ongoing efforts in
data collection and analysis improvements may
improve the quality and responsiveness of their
reported information. However, without feedback
from the congressional staffs on where
presentations were unclear, or where additional
detail or content is desired, the reports may
still not meet congressional needs. Discussing
information needs could also help identify which
needs could be addressed in an annual or other
recurring report and which could be addressed more
feasibly through some other means.

In addition to performance plans and reports, the
congressional staff also described a need for
readily accessible background information on
individual programs' authority, scope, and major
issues. Committee staff noted that rapid turnover
in Members' staff representatives to a committee
results in some of their colleagues needing a
quick introduction to complex programs and their
issues. Some of the program and agency
descriptions on agency Internet sites were
designed for the general public and were not
detailed enough to meet the congressional staffs'
needs.

Options for Arranging to Obtain New Information
To obtain new information about special
subpopulations or emerging issues, congressional
staff would have to make direct requests of the
agency. Agency officials told us that they
welcomed these requests and would do what they
could to meet them. However, depending on the
information requested and the time period in which
a response is needed, it might not be possible for
the agency to obtain it in time. Therefore,
discussion between congressional staff and agency
officials concerning the information needed is
important to clarify what is desired and what is
feasible to obtain, as well as to arrange for
obtaining the information.

In some cases, the agencies said that they were
able to conduct special tabulations to obtain the
desired information. In other cases, they said
that more data collection or analysis efforts
might be required and that they would need some
initial planning to determine how much time and
resources it would take to obtain the requested
information. Because it can be costly to obtain
some information, advance agreement on the
information content and format might avoid some
frustration on both sides by clarifying
expectations. In a couple of cases, when
congressional staff members learned that the
information was not readily available and would be
costly to obtain, they were satisfied to accept a
less precise or less detailed response.

Where congressional staff expect certain
information will be important in future
congressional considerations, advance planning for
its collection would help ensure its availability
in the desired format when it is needed. In some
cases, agencies may be able to alter their
information systems to track some new provision;
in others, they may have to plan new data
collection efforts. As stated in our 1995 report,
communication is critical at two points in
obtaining special studies:10

ï¿½    when a Committee frames a request for
information, to ensure that the agency understands
what is wanted and thus can alert the Committee to
issues of content or feasibility that need
resolution; and
ï¿½    as report drafting begins, to assist the
agency in understanding the issues that will be
before the Committee and what kind of presentation
format is thus likely to be most useful.

Agency Comments and Our Evaluation
The Departments of Health and Human Services and
Labor provided written comments on a draft of this
report, which are reprinted in appendixes II and
III. Both HHS and Labor stated that, in general,
the report is balanced and contains useful ideas
for improving communications between federal
agencies and congressional committees.

HHS also expressed two concerns. One concern was
that the report suggested that the Department did
not provide performance information at the program
level. It said its component agencies provided
this information in their own performance plans,
which are presented as part of their congressional
budget justifications. We have changed the text to
clarify that the HHS Fiscal Year 2000 Performance
Plan consisted of a departmentwide summary as well
as the performance plans submitted as part of its
component agencies' congressional budget
justifications. However, because we understand
that these budget justifications were not widely
distributed beyond the appropriations committees,
we remain concerned that this performance
information was not made readily available to
authorizing committee staff.

HHS' other concern was that the opening paragraphs
of the report implied that it would emphasize GPRA
as the primary medium for disseminating agency
performance information although, it noted, the
scope of the report is appropriately much broader.
The Committee's expectations for and concerns
about agencies' performance plans prepared under
GPRA were the impetus for this report. However,
the Committee also recognized that these plans and
reports are only one mechanism to provide
performance information to Congress and thus
broadened the focus of our work.

Officials at the Department of Education suggested
no changes and said that they appreciated
recognition of their efforts to work
collaboratively with Congress and provide good
management for the department's programs. OMB,
HHS, and PWBA provided technical comments that we
incorporated where appropriate throughout the
text.

Scope and Methodology
     To explore how agencies might improve the
usefulness of the performance information they
provide Congress, we conducted case studies of the
extent to which the relevant authorizing and
appropriations committee staffs obtained the
information they wanted about three program areas.
These cases were selected in consultation with the
requesting committee's staff to represent programs
whose performance information they felt could be
improved and to represent a range of program
structures and departments under the Committee's
jurisdiction. For example, one selection (pension
oversight) is a regulatory program in the
Department of Labor; the other two (student loans
and health surveillance) represent service
programs in the Departments of Education and
Health and Human Services. Pension oversight
represents the direct operations of a federal
agency, while the other cases operate through
state and local agencies or the private sector.
Each case represents a program or cluster of
programs administered by an agency within these
departments.

     To identify congressional information needs
and the extent to which they were met, we
interviewed staff members recommended by the
minority and majority staff directors of the
authorizing and appropriation committees for the
selected agencies.11 We asked the staffs to
identify what information they needed to address
the key policy questions or decisions they faced
over the preceding 2 years, and whether their
information needs were met. To identify the
reasons for the information gaps and how in
practice the agencies might better meet those
congressional information needs, we interviewed
both agency officials and congressional staff;
reviewed agency materials; and drew upon our
experience with various data collection, analysis,
and reporting strategies.

     We are sending copies of this report to
Senator Edward Kennedy, Ranking Minority Member of
your committee; Senator Ted Stevens, Chairman, and
Senator Robert Byrd, Ranking Minority Member,
Senate Committee on Appropriations; Representative
William Goodling, Chairman, and Representative
William Clay, Ranking Minority Member, House
Committee on Education and the Workforce;
Representative Tom Bliley, Chairman, and
Representative John Dingell, Ranking Minority
Member, House Committee on Commerce; and
Representative Bill Young, Chairman, and
Representative David Obey, Ranking Minority
Member, House Committee on Appropriations. We are
also sending copies of this report to the
Honorable Alexis Herman, Secretary of Labor; the
Honorable Donna Shalala, Secretary of Health and
Human Services; the Honorable Richard Wiley,
Secretary of Education; and the Honorable Jacob
Lew, Director, Office of Management and Budget. We
will also make copies available to others on
request.

If you have any questions concerning this report,
please call me or Stephanie Shipman at (202) 512-
7997. Another major contributor to this report was
Elaine Vaurio, Project Manager.

Sincerely yours,

Nancy R. Kingsbury
Acting Assistant Comptroller General

_______________________________
1 The three agencies are the Centers for Disease
Control and Prevention (CDC), Department of Health
and Human Services (HHS); the Office of Student
Financial Assistance (OSFA), Department of
Education; and the Pension and Welfare Benefits
Administration (PWBA), Department of Labor (DOL).
2 Program Evaluation: Improving the Flow of
Information to the Congress (GAO/PEMD-95-1, Jan.
30, 1995).
3 GAO/PEMD-95-1.
4 Executive Guide: Effectively Implementing the
Government Performance and Results Act (GAO/GGD-96-
118, June 1996); Managing for Results: Analytic
Challenges in Measuring Performance (GAO/HEHS/GGD-
97-138, May 30, 1997); Managing for Results:
Critical Issues for Improving Federal Agencies'
Strategic Plans (GAO/GGD-97-180, Sept. 16, 1997);
Managing for Results: An Agenda to Improve the
Usefulness of Agencies' Annual Performance Plans
(GAO/GGD/AIMD-98-228, Sept. 8, 1998); Managing for
Results: Measuring Results That Are Under Limited
Federal Control (GAO/GGD-99-16, Dec. 11, 1998);
Managing for Results: Opportunities for Continued
Improvements in Agencies' Performance Plans
(GAO/GGD/AIMD-99-215, July 20, 1999); and
Performance Budgeting: Fiscal Year 2000 Progress
in Linking Plans With Budgets (GAO/AIMD-99-239R,
July 30, 1999).
5 The 1998 reauthorization of the Higher Education
Act resulted in the creation of the performance-
based organization to help modernize and improve
the student aid delivery system that previously
was managed by the Office of Postsecondary
Education.
6 For PWBA, see http://www.dol.gov/dol/pwba; for
CDC, see http://www.cdc.gov; for OSFA, see
http://www.ed.gov.
7 PWBA did produce an Enforcement Strategy
Implementation Plan in 1994 that discussed its
enforcement efforts, but officials told us that it
was not for external distribution.
8 Managing for Results: Opportunities for
Continued Improvements in Agencies' Performance
Plans (GAO/GGD/AIMD-99-215, July 20, 1999).
9 GAO/PEMD-95-1.
10 GAO/PEMD-95-1.
11 We interviewed staffs on the Senate Health,
Education, Labor and Pensions Committee; the House
Committees on Commerce and on Education and the
Workforce; and the House Appropriations
Subcommittee on Labor, Health and Human Services,
Education, and Related Agencies. Staff on the
Senate Appropriations Subcommittee on Labor,
Health and Human Services, Education, and Related
Agencies were unavailable during the time frame of
this review because of the ongoing appropriations
process.

Appendix I
Core Questions to Assist Program Oversight
Page 26GAO/GGD-00-35 Performance Information to Co
ngress

Category of Core question
information 
Description Overall, what activities are
            conducted? By whom? How extensive and
            costly are the activities, and whom do
            they reach?
            
            If conditions, activities, and
           purposes are not uniform throughout
           the program, in what significant
           respects do they vary across program
           components, providers, or subgroups of
           clients?
Implementat What progress has been made in
ion         implementing new provisions?
            
            Have feasibility or management
            problems become evident?
            
            If activities and products are
           expected to conform to professional
           standards or to program
           specifications, have they done so?
Targeting   Have program activities or products
            focused on appropriate issues or
            problems?
            
            To what extent have they reached the
            appropriate people or organizations?
            
            Do current targeting practices leave
           significant needs unmet (problems not
           addressed, clients not reached)?
Impact      Overall, has the program led to
            improvements consistent with its
            purpose?
            
            If impact has not been uniform, how
            has it varied across program
            components, approaches, providers, or
            client subgroups?
           
           Are there components or providers that
           consistently have failed to show an
           impact?
Side-       Have program activities had important
effects     positive or negative side effects,
            either for program participants or
           outside the program?
Comparative Is this program's strategy more
advantage   effective in relation to its costs
           than others that serve the same
           purpose?
Source: GAO/PEMD-95-1, page 26.

Appendix II
Comments From the Department of Health and Human
Services
Page 29GAO/GGD-00-35 Performance Information to Co
ngress

Appendix III
Comments From the Department of Labor
Page 32GAO/GGD-00-35 Performance Information to Co
ngress

Related GAO Products
Page 35GAO/GGD-00-35 Performance Information to Co
ngress
Performance Budgeting: Fiscal Year 2000 Progress
in Linking Plans With Budgets (GAO/AIMD-99-239R,
July 30, 1999).

Performance Plans: Selected Approaches for
Verification and Validation of Agency Performance
Information (GAO/GGD-99-139, July 30, 1999).

Managing for Results: Opportunities for Continued
Improvements in Agencies' Performance Plans
(GAO/GGD/AIMD-99-215, July 20, 1999).

Regulatory Accounting: Analysis of OMB's Reports
on the Costs and Benefits of Federal Regulation
(GAO/GGD-99-59, Apr. 20, 1999).

Performance Budgeting: Initial Experiences Under
the Results Act in Linking Plans With Budgets
(GAO/AIMD-99-67, Apr. 12, 1999).

Emerging Infectious Diseases: Consensus on Needed
Laboratory Capacity Could Strengthen Surveillance
(GAO/HEHS-99-26, Feb. 5, 1999).

Managing for Results: Measuring Program Results
That Are Under Limited Federal Control (GAO/GGD-99-
16, Dec. 11, 1998).

Pension Benefit Guaranty Corporation: Financial
Condition Improving, but Long-Term Risks Remain
(GAO/HEHS-99-5, Oct. 16, 1998).

Managing for Results: An Agenda to Improve the
Usefulness of Agencies' Annual Performance Plans
(GAO/GGD/AIMD-98-228, Sept. 8, 1998).

Student Loans: Characteristics of Students and
Default Rates at Historically Black Colleges and
Universities (GAO/HEHS-98-90, Apr. 9, 1998).

Credit Reform: Greater Effort Needed to Overcome
Persistent Cost Estimation Problems (GAO/AIMD-98-
14, Mar. 30, 1998).

Managing for Results: Critical Issues for
Improving Federal Agencies' Strategic Plans
(GAO/GGD-97-180, Sept. 16, 1997).

Direct Student Loans: Analyses of the Income
Contingent Repayment Option (GAO/HEHS-97-155, Aug.
21, 1997).

Student Financial Aid Information: Systems
Architecture Needed to Improve Programs'
Efficiency (GAO/AIMD-97-122, July 29, 1997).

Managing for Results: Analytic Challenges in
Measuring Performance (GAO/HEHS/GGD-97-138, May
30, 1997).

High-Risk Series: Student Financial Aid (GAO/HR-97-
11, Feb. 1997).

Executive Guide: Effectively Implementing the
Government Performance and Results Act (GAO/GGD-96-
118, June 1996).

Program Evaluation: Improving the Flow of
Information to the Congress (GAO/PEMD-95-1, Jan.
30, 1995).

*** End of Document ***