Performance Measurement and Evaluation: Definitions and
Relationships (02-MAY-05, GAO-05-739SP).
Both the executive branch and congressional committees need
evaluative information to help them make decisions about the
programs they oversee--information that tells them whether, and
in what important ways, a program is working well or poorly, and
why. In enacting the Government Performance and Results Act of
1993 (GPRA), Congress expressed frustration that executive branch
and congressional decisionmaking was often hampered by the lack
of good information on the results of federal program efforts.
Seeking to promote improved federal management and the increased
efficiency and effectiveness of federal programs, GPRA instituted
a governmentwide requirement for agencies to set goals and report
annually on program performance. Many analytic approaches have
been employed over the years by the agencies and others to assess
the operations and results of federal programs, policies,
activities, and organizations. Periodically, individual
evaluation studies are designed to answer specific questions
about how well a program is working, and thus such studies may
take several forms. GPRA explicitly recognizes and encourages a
complementary role for these types of program assessment: annual
performance reports are to include both performance measurement
results and program evaluation findings. Both performance
measures and program evaluation play key roles in the Program
Assessment Rating Tool that the Office of Management and Budget
introduced in 2002 to examine federal programs in the budget
formulation process. This glossary describes and explains the
relationship between two common types of systematic program
assessments: performance measures and program evaluations. It is
based on GAO publications and program evaluation literature and
was first prepared in 1998.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-05-739SP
ACCNO: A26358
TITLE: Performance Measurement and Evaluation: Definitions and
Relationships
DATE: 05/02/2005
SUBJECT: Comparative analysis
Performance measures
Program evaluation
Glossaries
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-05-739SP
Program PerformanceAssessment
Both the executive branch and congressional committeesneed evaluative
information to help them make decisionsabout the programs they
oversee--information that tellsthem whether, and in what important ways, a
program isworking well or poorly, and why. In enacting the
GovernmentPerformance and Results Act of 1993 (GPRA), Congressexpressed
frustration that executive branch and congressionaldecisionmaking was
often hampered by the lack of goodinformation on the results of federal
program efforts. Seeking to promote improved federal management and the
increasedefficiency and effectiveness of federal programs, GPRAinstituted
a governmentwide requirement for agencies toset goals and report annually
on program performance.
Many analytic approaches have been employed over theyears by the agencies
and others to assess the operationsand results of federal programs,
policies, activities, and organizations. Periodically, individual
evaluation studies are designed to answer specific questions about how
wella program is working, and thus such studies may takeseveral forms.
GPRA explicitly recognizes and encourages a complementary role for these
types of program assessment:annual performance reports are to include both
performancemeasurement results and program evaluation findings.
Bothperformance measures and program evaluation play keyroles in the
Program Assessment Rating Tool that the Office of Management and Budget
introduced in 2002 to examinefederal programs in the budget formulation
process.
This glossary describes and explains the relationship betweentwo common
types of systematic program assessments:performance measures and program
evaluations. It is basedon GAO publications and program evaluation
literature and was first prepared in 1998. Major contributors
wereStephanie Shipman and JosephWholey. Please address
Nancy R. Kingsbury, Managing Director Applied Research and Methods
Types of Program PerformanceAssessment
Performance Measurement
ProgramEvaluation
Performance measurement is the ongoing monitoring and reporting ofprogram
accomplishments, particularly progress toward preestablishedgoals. It is
typically conducted by program or agency management.
Performance measures may address the type or level of program activities
conducted (process), the directproducts and services delivered bya program
(outputs), or the resultsof those products and services(outcomes).
A "program" may be any activity, project, function, or policy that hasan
identifiable purpose or set ofobjectives.
Program evaluations are individualsystematic studies conductedperiodically
or on an ad hoc basisto assess how well a program isworking. They are
often conductedby experts external to the program,either inside or outside
the agency, as well as by program managers.
A program evaluation typicallyexamines achievement of program objectives
in the context of otheraspects of program performance orin the context in
which it occurs. Four main types can be identified, all of which use
measures of program performance, along with otherinformation, to learn the
benefits of a program or how to improve it.
Relationship betweenPerformance Measurement and Program Evaluation
Different Focus
Performance measurement focuses on whether a program has achievedits
objectives, expressed as measurable performance standards. Program
evaluations typically examinea broader range of information onprogram
performance and its context than is feasible to monitor on an ongoing
basis.
Depending on their focus, evaluations may examine aspects ofprogram
operations (such as in aprocess evaluation) or factors inthe program
environment that mayimpede or contribute to its success,to help explain
the linkages betweenprogram inputs, activities, outputs,and outcomes.
Alternatively, evaluations may assess the program's effects beyond its
intended objectives, or estimate what would have occurred in the absence
of the program, in order to assess theprogram's net impact. Additionally,
program evaluations may systematically compare the effectiveness of
alternative programs aimed at thesame objective.
Different Use
Both forms of assessment aim to support resource allocation andother
policy decisions to improveservice delivery and programeffectiveness. But
performancemeasurement, because of its ongoing nature, can serve as an
earlywarning system to managementand as a vehicle for
improvingaccountability to the public.
A program evaluation's typicallymore in-depth examination of program
performance and contextallows for an overall assessment of whether the
program worksand identification of adjustmentsthat may improve its
results.
Types of Program Evaluation
Process (orImplementation)Evaluation
Outcome Evaluation
ImpactEvaluation
Cost-Benefit and Cost-Effectiveness Analyses
This form of evaluation assesses the extent to which a program is
operating as it was intended. It typicallyassesses program activities'
conformance to statutory and regulatoryrequirements, program design,
andprofessional standards or customer expectations.
This form of evaluation assesses the extent to which a program achievesits
outcome-oriented objectives. Itfocuses on outputs and outcomes(including
unintended effects) tojudge program effectiveness but may also assess
program processto understand how outcomes are produced.
Impact evaluation is a form of outcome evaluation that assesses the net
effect of a program by comparingprogram outcomes with an estimateof what
would have happened in theabsence of the program. This form of evaluation
is employed when external factors are known to influence the program's
outcomes, in order to isolate the program's contribution to achievement of
its objectives.
These analyses compare a program's outputs or outcomes with the
costs(resources expended) to producethem. When applied to
existingprograms, they are also considereda form of program
evaluation.Cost-effectiveness analysis assessesthe cost of meeting a
single goal orobjective and can be used to identify the least costly
alternative for meeting that goal. Cost-benefit analysisaims to identify
all relevant costsand benefits, usually expressed indollar terms.
*** End of document. ***