Performance Measurement and Evaluation: Definitions and 	 
Relationships (02-MAY-05, GAO-05-739SP).			 
                                                                 
Both the executive branch and congressional committees need	 
evaluative information to help them make decisions about the	 
programs they oversee--information that tells them whether, and  
in what important ways, a program is working well or poorly, and 
why. In enacting the Government Performance and Results Act of	 
1993 (GPRA), Congress expressed frustration that executive branch
and congressional decisionmaking was often hampered by the lack  
of good information on the results of federal program efforts.	 
Seeking to promote improved federal management and the increased 
efficiency and effectiveness of federal programs, GPRA instituted
a governmentwide requirement for agencies to set goals and report
annually on program performance. Many analytic approaches have	 
been employed over the years by the agencies and others to assess
the operations and results of federal programs, policies,	 
activities, and organizations. Periodically, individual 	 
evaluation studies are designed to answer specific questions	 
about how well a program is working, and thus such studies may	 
take several forms. GPRA explicitly recognizes and encourages a  
complementary role for these types of program assessment: annual 
performance reports are to include both performance measurement  
results and program evaluation findings. Both performance	 
measures and program evaluation play key roles in the Program	 
Assessment Rating Tool that the Office of Management and Budget  
introduced in 2002 to examine federal programs in the budget	 
formulation process. This glossary describes and explains the	 
relationship between two common types of systematic program	 
assessments: performance measures and program evaluations. It is 
based on GAO publications and program evaluation literature and  
was first prepared in 1998.					 
-------------------------Indexing Terms------------------------- 
REPORTNUM:   GAO-05-739SP					        
    ACCNO:   A26358						        
  TITLE:     Performance Measurement and Evaluation: Definitions and  
Relationships							 
     DATE:   05/02/2005 
  SUBJECT:   Comparative analysis				 
	     Performance measures				 
	     Program evaluation 				 
	     Glossaries 					 

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO Product.                                                 **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
******************************************************************
GAO-05-739SP

                         Program PerformanceAssessment

Both the executive branch and congressional committeesneed evaluative
information to help them make decisionsabout the programs they
oversee--information that tellsthem whether, and in what important ways, a
program isworking well or poorly, and why. In enacting the
GovernmentPerformance and Results Act of 1993 (GPRA), Congressexpressed
frustration that executive branch and congressionaldecisionmaking was
often hampered by the lack of goodinformation on the results of federal
program efforts. Seeking to promote improved federal management and the
increasedefficiency and effectiveness of federal programs, GPRAinstituted
a governmentwide requirement for agencies toset goals and report annually
on program performance.

Many analytic approaches have been employed over theyears by the agencies
and others to assess the operationsand results of federal programs,
policies, activities, and organizations. Periodically, individual
evaluation studies are designed to answer specific questions about how
wella program is working, and thus such studies may takeseveral forms.
GPRA explicitly recognizes and encourages a complementary role for these
types of program assessment:annual performance reports are to include both
performancemeasurement results and program evaluation findings.
Bothperformance measures and program evaluation play keyroles in the
Program Assessment Rating Tool that the Office of Management and Budget
introduced in 2002 to examinefederal programs in the budget formulation
process.

This glossary describes and explains the relationship betweentwo common
types of systematic program assessments:performance measures and program
evaluations. It is basedon GAO publications and program evaluation
literature and was first prepared in 1998. Major contributors
wereStephanie Shipman and JosephWholey. Please address

Nancy R. Kingsbury, Managing Director Applied Research and Methods

                     Types of Program PerformanceAssessment

Performance Measurement

ProgramEvaluation

Performance measurement is the ongoing monitoring and reporting ofprogram
accomplishments, particularly progress toward preestablishedgoals. It is
typically conducted by program or agency management.

Performance measures may address the type or level of program activities
conducted (process), the directproducts and services delivered bya program
(outputs), or the resultsof those products and services(outcomes).

A "program" may be any activity, project, function, or policy that hasan
identifiable purpose or set ofobjectives.

Program evaluations are individualsystematic studies conductedperiodically
or on an ad hoc basisto assess how well a program isworking. They are
often conductedby experts external to the program,either inside or outside
the agency, as well as by program managers.

A program evaluation typicallyexamines achievement of program objectives
in the context of otheraspects of program performance orin the context in
which it occurs. Four main types can be identified, all of which use
measures of program performance, along with otherinformation, to learn the
benefits of a program or how to improve it.

       Relationship betweenPerformance Measurement and Program Evaluation

Different Focus

Performance measurement focuses on whether a program has achievedits
objectives, expressed as measurable performance standards. Program
evaluations typically examinea broader range of information onprogram
performance and its context than is feasible to monitor on an ongoing
basis.

Depending on their focus, evaluations may examine aspects ofprogram
operations (such as in aprocess evaluation) or factors inthe program
environment that mayimpede or contribute to its success,to help explain
the linkages betweenprogram inputs, activities, outputs,and outcomes.
Alternatively, evaluations may assess the program's effects beyond its
intended objectives, or estimate what would have occurred in the absence
of the program, in order to assess theprogram's net impact. Additionally,
program evaluations may systematically compare the effectiveness of
alternative programs aimed at thesame objective.

                                 Different Use

Both forms of assessment aim to support resource allocation andother
policy decisions to improveservice delivery and programeffectiveness. But
performancemeasurement, because of its ongoing nature, can serve as an
earlywarning system to managementand as a vehicle for
improvingaccountability to the public.

A program evaluation's typicallymore in-depth examination of program
performance and contextallows for an overall assessment of whether the
program worksand identification of adjustmentsthat may improve its
results.

                          Types of Program Evaluation

Process (orImplementation)Evaluation

Outcome Evaluation

ImpactEvaluation

Cost-Benefit and Cost-Effectiveness Analyses

This form of evaluation assesses the extent to which a program is
operating as it was intended. It typicallyassesses program activities'
conformance to statutory and regulatoryrequirements, program design,
andprofessional standards or customer expectations.

This form of evaluation assesses the extent to which a program achievesits
outcome-oriented objectives. Itfocuses on outputs and outcomes(including
unintended effects) tojudge program effectiveness but may also assess
program processto understand how outcomes are produced.

Impact evaluation is a form of outcome evaluation that assesses the net
effect of a program by comparingprogram outcomes with an estimateof what
would have happened in theabsence of the program. This form of evaluation
is employed when external factors are known to influence the program's
outcomes, in order to isolate the program's contribution to achievement of
its objectives.

These analyses compare a program's outputs or outcomes with the
costs(resources expended) to producethem. When applied to
existingprograms, they are also considereda form of program
evaluation.Cost-effectiveness analysis assessesthe cost of meeting a
single goal orobjective and can be used to identify the least costly
alternative for meeting that goal. Cost-benefit analysisaims to identify
all relevant costsand benefits, usually expressed indollar terms.
*** End of document. ***