Program Measurement and Evaluation: Definitions and Relationships (Other
Written Prod., 04/01/97, GAO/GGD-98-26).
GAO presented information on the different categories of systematic
program assessments and explained the relationships among common types
of assessments: (1) performance measures; (2) program evaluations; and
(3) performance audits.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: GGD-98-26
TITLE: Program Measurement and Evaluation: Definitions and
Relationships
DATE: 04/01/97
SUBJECT: Evaluation methods
Audits
Cost effectiveness analysis
Congressional/executive relations
Program evaluation
IDENTIFIER: GPRA
Government Performance and Results Act
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO report. Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved. Major **
** divisions and subdivisions of the text, such as Chapters, **
** Sections, and Appendixes, are identified by double and **
** single lines. The numbers on the right end of these lines **
** indicate the position of each of the subsections in the **
** document outline. These numbers do NOT correspond with the **
** page numbers of the printed product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
** A printed copy of this report may be obtained from the GAO **
** Document Distribution Center. For further details, please **
** send an e-mail message to: **
** **
** **
** **
** with the message 'info' in the body. **
******************************************************************
Cover
================================================================ COVER
Glossary
April 1998
PERFORMANCE MEASUREMENT AND
EVALUATION - DEFINITIONS AND
RELATIONSHIPS
GAO/GGD-98-26
(966705)
Abbreviations
=============================================================== ABBREV
PROGRAM PERFORMANCE ASSESSMENT
============================================================ Chapter 0
Both the executive branch and congressional committees need
evaluative information to help them make decisions about the programs
they oversee--information that tells them whether, and in what
important ways, a program is working well or poorly, and why. In
enacting the Government Performance and Results Act of 1993 (the
Results Act), Congress expressed frustration that congressional and
executive branch decisionmaking was often hampered by the lack of
good information on the results of federal program efforts. Seeking
to promote improved federal management and the increased efficiency
and effectiveness of federal programs, the Results Act instituted a
governmentwide requirement for agencies to set goals and report
annually on program performance.
Many analytic approaches have been employed over the years by the
agencies and others to assess the operations and results of federal
programs, policies, activities, and organizations. Periodically,
individual audit and evaluation studies are designed to answer
specific questions about how well a program is working, and thus such
studies may take several forms. The Results Act explicitly
recognizes and encourages a complementary role for these types of
program assessment: annual performance reports are to include both
performance measurement results and evaluation findings. This
document describes and explains the relationships among common types
of systematic program assessments: performance measures, program
evaluations, and performance audits.
This glossary was prepared at the request of the Senate Committee on
Governmental Affairs. Major contributors were Stephanie Shipman and
Joseph Wholey. Please address any questions to me or Stephanie
Shipman on (202) 512-7997.
Susan S. Westin
Associate Director, Advanced Studies and
Evaluation Methodology
TYPES OF PROGRAM PERFORMANCE
ASSESSMENT
============================================================ Chapter 1
PERFORMANCE MEASUREMENT
---------------------------------------------------------- Chapter 1:1
Performance measurement is the ongoing monitoring and reporting of
program accomplishments, particularly progress towards preestablished
goals. It is typically conducted by program or agency management.
Performance measures may address the type or level of program
activities conducted (process), the direct products and services
delivered by a program (outputs), and/or the results of those
products and services (outcomes).
A "program" may be any activity, project, function, or policy that
has an identifiable purpose or set of objectives.
PROGRAM EVALUATION
---------------------------------------------------------- Chapter 1:2
Program evaluations are individual systematic studies conducted
periodically or on an ad hoc basis to assess how well a program is
working. They are often conducted by experts external to the
program, either inside or outside the agency, as well as by program
managers.
A program evaluation typically examines achievement of program
objectives in the context of other aspects of program performance or
in the context in which it occurs. Four main types can be
identified, all of which use measures of program performance, along
with other information, to learn the benefits of a program or how to
improve it. Evaluations are one type of study done by GAO under the
broad category of performance audits.
RELATIONSHIP BETWEEN PERFORMANCE
MEASUREMENT AND PROGRAM EVALUATION
============================================================ Chapter 2
DIFFERENT FOCUS
---------------------------------------------------------- Chapter 2:1
Performance measurement focuses on whether a program has achieved its
objectives, expressed as measurable performance standards. Program
evaluations typically examine a broader range of information on
program performance and its context than is feasible to monitor on an
ongoing basis.
Depending on their focus, evaluations may examine aspects of program
operations (such as in a process evaluation), or factors in the
program environment that may impede or contribute to its success, to
help explain the linkages between program inputs, activities,
outputs, and outcomes. Alternatively, evaluations may assess the
program's effects beyond its intended objectives, or estimate what
would have occurred in the absence of the program, in order to assess
the program's net impact. Additionally, program evaluations may
systematically compare the effectiveness of alternative programs
aimed at the same objective.
DIFFERENT USE
---------------------------------------------------------- Chapter 2:2
Both forms of assessment aim to support resource allocation and other
policy decisions to improve service delivery and program
effectiveness. But performance measurement, because of its ongoing
nature, can serve as an early warning system to management and as a
vehicle for improving accountability to the public.
A program evaluation's typically more in-depth examination of program
performance and context allows for an overall assessment of whether
the program works and identification of adjustments that may improve
its results.
TYPES OF PROGRAM EVALUATION
============================================================ Chapter 3
PROCESS (OR IMPLEMENTATION)
EVALUATION
---------------------------------------------------------- Chapter 3:1
This form of evaluation assesses the extent to which a program is
operating as it was intended. It typically assesses program
activities' conformance to statutory and regulatory requirements,
program design, and professional standards or customer expectations.
OUTCOME EVALUATION
---------------------------------------------------------- Chapter 3:2
This form of evaluation assesses the extent to which a program
achieves its outcome-oriented objectives. It focuses on outputs and
outcomes (including unintended effects) to judge program
effectiveness but may also assess program process to understand how
outcomes are produced.
IMPACT EVALUATION
---------------------------------------------------------- Chapter 3:3
Impact evaluation is a form of outcome evaluation that assesses the
net effect of a program by comparing program outcomes with an
estimate of what would have happened in the absence of the program.
This form of evaluation is employed when external factors are known
to influence the program's outcomes, in order to isolate the
program's contribution to achievement of its objectives.
COST-BENEFIT AND
COST-EFFECTIVENESS ANALYSES
---------------------------------------------------------- Chapter 3:4
These analyses compare a program's outputs or outcomes with the costs
(resources expended) to produce them. When applied to existing
programs, they are also considered a form of program evaluation.
Cost-effectiveness analysis assesses the cost of meeting a single
goal or objective, and can be used to identify the least costly
alternative to meet that goal. Cost-benefit analysis aims to
identify all relevant costs and benefits, usually expressed in dollar
terms.
*** End of document. ***