Managing for Results: Experiences of Selected Credit Programs (Letter
Report, 02/19/98, GAO/GGD-98-41).

The Government Performance and Results Act of 1993 requires federal
agencies to set goals, measure performance, and use performance
information to improve results. Since the passage of this legislation,
many agencies have been preparing for its governmentwide implementation
starting in the fall of 1997. In June 1997, GAO concluded that
governmentwide implementation would be uneven. (See GAO/GGD-97-109.)
That report also briefly discussed the experiences of federal credit
programs in developing goals and measures under the Results Act. Federal
credit, or loan, programs are generally intended to provide financial
assistance to borrowers that the private sector will not serve. This
report provides additional information on major credit agencies' efforts
to implement the Results Act. It also describes the efforts of the
Federal Credit Policy Working Group--an interagency advisory group on
credit management policy--to develop common performance measures for
credit programs.

--------------------------- Indexing Terms -----------------------------

 REPORTNUM:  GGD-98-41
     TITLE:  Managing for Results: Experiences of Selected Credit 
             Programs
      DATE:  02/19/98
   SUBJECT:  Interagency relations
             Financial management
             Strategic planning
             Agency missions
             Credit
             Program evaluation
             Data collection
             Loans
             Reporting requirements
             Loan accounting systems
IDENTIFIER:  VA Home Loan Guaranty Program
             William D. Ford Federal Direct Loan Program
             Federal Family Education Loan Program
             Government Performance and Results Act
             GPRA
             USDA Single Family Housing Direct Loan Program
             USDA Single Family Housing Guaranteed Loan Program
             
******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO report.  Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved.  Major          **
** divisions and subdivisions of the text, such as Chapters,    **
** Sections, and Appendixes, are identified by double and       **
** single lines.  The numbers on the right end of these lines   **
** indicate the position of each of the subsections in the      **
** document outline.  These numbers do NOT correspond with the  **
** page numbers of the printed product.                         **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
** A printed copy of this report may be obtained from the GAO   **
** Document Distribution Center.  For further details, please   **
** send an e-mail message to:                                   **
**                                                              **
**                                            **
**                                                              **
** with the message 'info' in the body.                         **
******************************************************************


Cover
================================================================ COVER


Report to the Director, Office of Management and Budget

February 1998

MANAGING FOR RESULTS - EXPERIENCES
OF SELECTED CREDIT PROGRAMS

GAO/GGD-98-41

Experiences of Selected Credit Programs

(410058)


Abbreviations
=============================================================== ABBREV

  ABC - activity-based costing
  CFO - Chief Financial Officers Act
  DCA - Debt Collection Act
  DCIA - Debt Collection Improvement Act
  FASAB - Federal Accounting Standards Advisory Board
  FATS - Foreclosure Avoidance Through Servicing
  FCRA - Federal Credit Reform Act
  GMRA - Government Management and Reform Act
  GPRA - General Performance and Results Act
  HUD - Department of Housing and Urban Development
  OIG - Office of Inspector General
  OMB - Office of Management and Budget
  SFH - Single-Family Housing
  USDA - U.S.  Department of Agriculture
  VA - Department of Veterans Affairs

Letter
=============================================================== LETTER


B-275125

February 19, 1998

The Honorable Franklin D.  Raines
Director, Office of Management
 and Budget

Dear Mr.  Raines: 

The Government Performance and Results Act of 1993, also referred to
as "GPRA" or "the Results Act," seeks to improve the efficiency,
effectiveness, and accountability of federal programs by establishing
a system to set goals for programs based on their intended purposes,
measure the performance of those programs, and use performance
information to improve results.  As you know, if successfully
implemented, the Results Act will help agencies focus on how to
improve their programs' performance in achieving desired results. 

The Act requires agencies to establish goals, performance measures,
and target levels of performance for fiscal year 1999.  However,
since the passage of the Act, many agencies have been preparing for
its governmentwide implementation, beginning in the fall of 1997.  In
June 1997, under the Results Act requirement that we report on
agencies' implementation of the Act, we concluded that governmentwide
implementation would be uneven.\1 That report included a brief
discussion of federal credit programs' experiences in developing
goals and measures under the Results Act. 

This report provides additional information and context on major
credit agencies' efforts to implement the Act.  It also describes the
efforts of the Federal Credit Policy Working Group--an interagency
advisory group on credit management policy--to develop common
performance measures for credit programs.\2 In July 1995, the Working
Group established a task force to develop and enhance credit program
performance measures.  The report is addressed to you because of the
Office of Management and Budget's (OMB) responsibility for overseeing
agencies' implementation of the Act and because the Deputy Director
of Management at OMB chairs the Working Group. 

Federal credit, or loan, programs are generally intended to provide
financial assistance to borrowers that the private sector will not
serve.  In doing so, these programs are typically expected to help
achieve certain social and economic results, such as reaching
underserved populations and neighborhoods and supporting investment
that is important to the economy.  As reported in the fiscal year
1998 budget, at the end of fiscal year 1996, the outstanding face
value of credit programs had reached almost $1 trillion in direct
loans and loan guarantees.\3

For this report, our first objective was to identify goals and
measures established by the selected credit programs that related to
the programs' intended purposes and to determine whether the programs
had set target levels of performance for assessing their progress in
achieving their desired results.  Under the Results Act, target
levels of performance are to enable decisionmakers to compare planned
versus actual results achieved for a given year.  Our second
objective was to identify the challenges agency officials cited in
developing performance information, including goals and measures, for
the selected programs and any approaches those programs were taking
to address those challenges.  Our third objective was to describe the
status of the Working Group's effort to develop common performance
measures for federal credit programs. 

We selected for our review credit programs that varied in terms of
type of program (e.g., housing and education loans); mode of credit
delivery (e.g., direct and guaranteed loans); and program size as
measured by the amount of outstanding loans.  These programs were

  -- the Department of Veterans Affairs' (VA) Loan Guaranty Program,
     which provides home loans for veterans and active duty
     personnel;

  -- the Department of Education's William D.  Ford Direct Loan
     Program and its Federal Family Education Loan Program (referred
     to in this report as Education's direct and guaranteed student
     loan programs, respectively), which issue or guarantee loans,
     respectively, for students enrolled in postsecondary education;
     and

  -- the Department of Agriculture's (USDA) Single-Family Housing
     (SFH) direct loan program and guaranteed loan program, which
     make or guarantee loans to low- and moderate-income rural
     families, respectively, who are unable to obtain credit from
     other sources. 

According to data reported by OMB in the fiscal year 1998 budget and
agency data, of the total amount of federal credit outstanding in
fiscal year 1996 for guaranteed loans ($805 billion) and direct loans
($165 billion), these five programs represented about 32 and 18
percent, respectively. 


--------------------
\1 The Government Performance and Results Act:  1997 Governmentwide
Implementation Will Be Uneven (GAO/GGD-97-109, June 2, 1997). 

\2 The Federal Credit Policy Working Group is a forum of OMB and
federal credit agency officials that was established in 1986 to
provide advice in the formulation and implementation of credit
management policy. 

\3 Under direct loan programs, a federal agency generally makes a
direct disbursement to an approved borrower and then services and
collects on the loan.  Under guaranteed loan programs, federal
agencies rely on private sector lenders to originate and service
loans within federal guidelines.  In the event of a default on a
guaranteed loan, the government is liable for all or part of the
interest and loan principal. 


   RESULTS IN BRIEF
------------------------------------------------------------ Letter :1

In their efforts to implement the Results Act, the five credit
programs established goals and performance measures that appeared to
be generally related to the programs' intended purposes.  For
example, to monitor its performance in achieving one of its intended
purposes, namely helping veterans retain their homes, VA's Loan
Guaranty Program established the "Foreclosure Avoidance Through
Servicing (FATS) Ratio." According to VA, the FATS ratio is to
measure the extent to which foreclosures would have been greater had
VA not pursued alternatives to foreclosure, such as intervening with
the holder of the loan on behalf of the borrower to set up a
repayment plan.  The selected programs also set fiscal year 1998
targets for most of their respective measures.  Thus, if the selected
programs collect accurate corresponding data on their actual
performance, they should be able to monitor their progress in
achieving desired results on those measures and have fiscal year 1998
baseline data to use in setting future targets for those measures. 

Although the selected programs have established goals, measures, and
targets in their efforts to implement the Results Act, on the basis
of our discussions with agency officials, we identified three general
challenges the programs have been facing in developing performance
information:  (1) a struggle in reaching consensus among stakeholders
on the programs' intended purposes, performance measures, and target
levels of performance; (2) difficulty in separating the effects of
external forces from program influences on results; and (3) a lack of
relevant program performance and financial baseline data.  Agency
officials also described some approaches they were taking to address
the challenges they have been facing.  For example, VA officials told
us that they had difficulty linking field office activities to the
intended results of the program as established by headquarters staff. 
To address this difficulty, VA headquarters and field office staff
worked together to develop the program's performance measures, and VA
managers reached agreement on target levels of performance. 

According to OMB and the Working Group, comparing results using
common measures across credit programs allows program managers and
other decisionmakers to identify best practices among those programs
that have the potential for improving other credit programs'
performance.  By September 1996, a Working Group task force had
proposed a set of common performance measures for federal credit
programs.  Examples of common financial and programmatic measures
proposed by the task force included total delinquent debt and the
percentage of borrowers who were pleased with the timeliness and
quality of credit program service, respectively.  In June 1997, OMB
reported that major credit agencies have adopted some of the proposed
measures.\4 Specifically, those agencies have adopted some of the
proposed common measures, in particular, financial measures for
meeting the annual budgeting and reporting requirements of the Debt
Collection Improvement Act and Federal Credit Reform Act--budget and
credit management reform legislation enacted to address long-standing
concerns about the program and financial management of credit
programs. 

However, two general problems have limited the Working Group's
progress in developing common performance measures for credit
programs.  First, several credit programs lack relevant program
performance and financial baseline data.  Second, the Working Group
has been unable to reach consensus on the appropriateness of
decisionmakers using some of the task force's proposed measures to
assess the performance of individual credit programs or to compare
that performance against the performance of other credit programs. 
For example, some programs give credit only to persons who are unable
to obtain credit from other sources; other programs give credit to
anyone who is entitled to the programs' benefits, regardless of his
or her access to credit.  Thus, officials in some credit agencies
questioned whether it would be appropriate to make comparisons among
credit programs' default rates, because the financial characteristics
of borrowers for each program may be different. 

The Working Group anticipated that agencies that administer credit
programs could include common financial and programmatic measures in
their annual performance plans and reports under the Results Act. 
However, OMB does not intend to require credit agencies to adopt
common performance measures when consensus about the appropriateness
of such measures has not been achieved.  We agree that OMB should not
force the use of common measures when concerns about their
appropriateness exist; but at the time of our review, the Working
Group had not resolved those concerns and had not decided how and
when those concerns would be addressed.  Thus, it is unclear whether
OMB and the credit agencies will maintain their current level of
attention to developing common measures.  Also unclear is the extent
to which agencies that administer credit programs will include common
measures for those programs in their annual performance plans that
could provide useful information to decisionmakers interested in
making performance and cost comparisons. 

We recognize the difficulty the Working Group is facing in reaching
consensus on common performance measures and that significant
differences in program characteristics may limit the usefulness of
some measures for broad, cross-program comparisons.  However, we
believe the potential benefits that could be realized from developing
common performance measures, where appropriate, underscore the
importance of OMB and the credit agencies continuing their efforts to
develop and reach consensus on such measures.  We recommend ways that
the Director of OMB can work with those agencies to achieve progress
in this area.  Our recommendations are intended to serve a twofold
purpose.  First, they would help ensure that the search for common
measures continues.  And second, they would document the results of
those efforts in a way that would (1) permit further analysis
directed at the identification of additional common measures and (2)
facilitate an understanding of any limitations of using existing
common measures to compare results across credit programs. 


--------------------
\4 1997 Federal Financial Management Status Report and Five-Year
Plan, OMB, June 25, 1997. 


   BACKGROUND
------------------------------------------------------------ Letter :2

In the 1990s, Congress put in place a statutory framework to address
the long-standing weaknesses in federal operations, improve federal
management practices, and provide greater accountability for
achieving results.  This framework included as its essential elements
the Results Act and key financial management and information
technology reform legislation:  the Chief Financial Officers Act of
1990 (CFO)--as expanded by the Government Management Reform Act of
1994 (GMRA)--and the Paperwork Reduction Act of 1995 and the
Clinger-Cohen Act of 1996, respectively.  Taken together, these
legislative initiatives seek to respond to a need for accurate,
reliable, and integrated budget, financial, and program information
for congressional and executive branch decisionmaking, information
that much of our prior work has shown to be badly lacking.\5

The goal-setting and performance measurement and improvement system
envisioned by the Results Act is the centerpiece of this framework
and starts with the requirement that each executive agency develop
and periodically update a strategic plan to lay out its mission,
long-term goals and objectives, and strategies for achieving those
goals and objectives.  Under the Results Act, the first of these
plans were due by September 30, 1997.  Next, each agency is to
develop an annual performance plan, beginning with the agency's plan
for fiscal year 1999, which covers each program activity set forth in
the agency's budget.  Agencies were to submit their fiscal year 1999
annual performance plans to OMB in the fall of 1997 with their fiscal
year 1999 budget requests and are to submit those plans to Congress
after the President's fiscal year 1999 budget is provided to Congress
in February 1998.  Among other things, agencies' annual performance
plans are to contain their programs' goals and measures for fiscal
year 1999.  Finally, each agency is to report publicly on its
programs' performance, specifically on the degree to which the goals
that are laid out in the agency's annual performance plan are being
met and on actions it plans to take to achieve unmet goals.  The
first of these reports, on programs' performance for fiscal year
1999, is due by March 31, 2000; subsequent reports are due by March
31 for the years that follow.  (For a detailed description of the
Results Act's requirements, see app.  I.)

The CFO Act and GMRA are intended to strengthen the reliability of
agencies' financial and programmatic performance information and the
reporting of such information by, among other things, having agencies
develop better performance measures and cost information and design
results-oriented reports that integrate budget, accounting, and
program information.\6 Finally, the Paperwork Reduction Act and
Clinger-Cohen Act seek to help agencies address long-standing
weaknesses in their use of information technology.  Under this
legislation, each agency is to better link its technology plan and
information technology use to achieving the agency's desired
results.\7

In addition, long-standing concerns about the program and financial
management of credit programs have prompted Congress to enact
important budget and credit management reform initiatives over the
last 15 years.  These initiatives include the Debt Collection Act of
1982 (DCA) and amendments, the Debt Collection Improvement Act of
1996 (DCIA), and the Federal Credit Reform Act of 1990 (FCRA).  DCA
and DCIA are significant pieces of credit management legislation
designed to, among other things, facilitate federal efforts to
decrease delinquencies and increase collections.  Under DCA, agencies
were to annually report to the Director of OMB and the Secretary of
the Treasury on the status of their debt collection activities. 
Under DCIA, agencies are now to annually report on those activities
only to the Secretary of the Treasury, who in turn is to annually
report to Congress, beginning no later than 1999, on such activities
governmentwide.\8 FCRA changed the budget treatment of direct loans
and loan guarantees made on or after October 1, 1991, to (1)
facilitate more accurate reporting by credit agencies on the full
cost to the government in the budget for the year in which the
programs made or guaranteed the loans so that executive branch and
congressional decisionmakers might consider such costs when making
budget decisions and (2) permit appropriate cost comparisons between
direct and guaranteed credit programs and between credit and
noncredit programs intended to achieve similar purposes.\9

Guided by these legislative initiatives, since 1992, OMB has
encouraged federal credit agencies to improve their credit programs'
financial and programmatic performance measures and to adopt a set of
common performance measures for those programs.  According to OMB and
the Federal Credit Policy Working Group, common measures should help
credit program managers and other decisionmakers assess how similar
functions are performed and promote an atmosphere of cooperation and
a sharing of ideas among agency officials on how to improve the
performance of credit programs. 

In response to the increasing importance being placed on agencies'
integration of performance measurement with budgeting, management
improvement, and overall agency accountability, the Working Group has
focused since July 1995 on developing common performance measures for
credit programs.  To do so, the Working Group established a task
force to develop measures for credit programs consistent with the
Results Act and credit and financial management reform initiatives. 
In August 1997 we reported that agencies' annual planning under the
Results Act could be used as a vehicle for developing, where
appropriate, common performance measures for permitting future
comparisons of similar programs' results and the methods those
programs used to achieve those results.\10


--------------------
\5 GAO/GGD-97-109, June 2, 1997. 

\6 For more information on the CFO Act, see Financial Management: 
Continued Momentum Essential to Achieve CFO Act Goals
(GAO/T-AIMD-96-10, Dec.  14, 1995). 

\7 For more information on the best practices that leading public and
private organizations use to effectively manage information
technology, which served as a basis for the Paperwork Reduction Act
and Clinger-Cohen Act, see Executive Guide:  Improving Mission
Performance Through Strategic Information Management and
Technology--Learning From Leading Organizations (GAO/AIMD-94-115, May
1994). 

\8 For more information on DCA (P.L.  97-365) and DCIA (P.L. 
104-134), see Debt Collection:  Improved Reporting Needed on Billions
of Dollars in Delinquent Debt and Agency Collection Performance
(GAO/AIMD-97-48, June 2, 1997). 

\9 Guidance for reporting of direct and guaranteed loans is provided
in the Statement of Federal Financial Accounting Standards No.  2,
Accounting for Direct Loans and Loan Guarantees, which essentially
mirrors budgetary reporting under FCRA.  For more information on the
FCRA (P.L.  101-508), see Credit Reform:  Case-by-Case Assessment
Advisable in Evaluating Coverage and Compliance (GAO/AIMD-94-57, July
28, 1994). 

\10 Managing For Results:  Using the Results Act to Address Mission
Fragmentation and Program Overlap (GAO/AIMD-97-146, Aug.  29, 1997). 


   SCOPE AND METHODOLOGY
------------------------------------------------------------ Letter :3

To meet our first objective--to identify goals and measures
established by the selected credit programs that related to the
programs' intended purposes and determine whether the programs had
set target levels of performance for assessing their progress in
achieving their desired results--we compared the programs' goals and
measures to their respective intended purposes, as identified by the
programs or their respective agencies.  We interviewed agency
officials about the programs' intended purposes and asked those
officials to comment on the relationships we identified between the
programs' goals and measures and intended purposes.  To determine
whether the programs had set target levels of performance for
assessing their progress in achieving their desired results, we
identified those measures for which the programs had either (1)
identified fiscal year 1998 targets or (2) reported prior year
baseline data for those measures and indicated how performance on
those measures was to change (i.e., increase or decrease) in fiscal
year 1998 relative to the baseline. 

The programs' goals and performance measures that we used for our
assessment were those established by the programs as of May 1, 1997,
which, according to agency officials, were generally the same ones
the programs submitted with their respective agencies' fiscal year
1998 budget presentations to OMB and Congress.  The programs also
were proposing to include these goals and measures for their
agencies' fiscal year 1999 annual performance plans under the Results
Act.  Although we make observations about differences in goals and
measures among the programs, our review did not address the
reasonableness of the processes or methods the programs used to
determine how to assess progress or establish target levels toward
achieving the programs' intended purposes; determine whether other,
more appropriate measures existed; or evaluate the feasibility of the
targets the programs established. 

To meet our second objective--to identify the challenges agency
officials cited in developing performance information, including
goals and measures, for the selected programs and any approaches
those programs were taking to address those challenges--we asked
agency officials responsible for and involved in the development of
goals and measures for those programs to rate how great a challenge
it was to perform each of 49 activities we identified as associated
with developing performance information.  Examples of these
activities included "determining a realistic target level of
performance for annual performance goals" and "developing measures
for assessing the net effect of the program compared with what would
have occurred in the absence of the program." To identify these
activities, we referred to key steps and practices identified in our
Executive Guide:  Effectively Implementing the Government Performance
and Results Act and other work that we had under way assessing the
challenges agencies were facing in implementing performance
measurement.\11

We asked officials to use a five-point scale to rate the 49
activities, ranging from "little to no" challenge to a "very great"
challenge for the selected credit programs in their agencies.  For
purposes of this report, we refer to activities that any of the
agency officials rated as a "great" or "very great" challenge as
significantly challenging activities.  We then interviewed agency
officials about why they rated certain activities associated with
developing performance information as significantly challenging.  We
analyzed their responses and related documentation to identify
general challenges that led agency officials to report those
activities as significantly challenging for the selected programs. 
To identify any approaches those programs were taking to address the
challenges they have been facing, we talked with agency officials
about such approaches and analyzed agency documentation.  We also
considered prior and ongoing work we have done on the efforts of VA,
Education, USDA, and other credit agencies to implement various
credit and financial management reforms. 

To meet our third objective--to describe the status of the Working
Group's effort to develop common performance measures for federal
credit programs--we reviewed various documentation from OMB and
members of the Working Group that described this effort and agency
officials' views about those measures.  We also talked to OMB
officials and members of the Working Group, including agency
officials at those agencies administering the programs we selected
for our review, to obtain their views of the common performance
measures proposed.  Our review did not address the reasonableness of
the processes or methods the Working Group's task force used in
determining how to assess progress for federal credit programs or
determine whether other measures existed that may be more
appropriate.  (For a more detailed discussion on our objectives,
scope, and methodology, see app.  II.)

We did our work from September 1996 to December 1997, in Washington,
D.C., in accordance with generally accepted government auditing
standards.  We requested comments on a draft of this report from the
Director of OMB, the Secretaries of Education and Agriculture, and
the Acting Secretary of VA or their designees.  Comments were
provided orally by designees of the four agencies, and those comments
are discussed at the end of this report. 


--------------------
\11 Executive Guide:  Effectively Implementing the Government
Performance and Results Act (GAO/GGD-96-118, June 1996); and Managing
For Results:  Analytic Challenges in Measuring Performance
(GAO/HEHS/GGD-97-138, May 30, 1997). 


   SELECTED CREDIT PROGRAMS
   ESTABLISHED GOALS, MEASURES,
   AND TARGETS TO MONITOR THEIR
   PROGRESS
------------------------------------------------------------ Letter :4

In their efforts to implement the Results Act, the five credit
programs established goals and performance measures that appeared to
be generally related to the programs' intended purposes.  In some
cases, the programs established goals or measures that addressed
intermediate results that the programs expected to lead to their
intended purposes.  Also, some of the selected programs established a
range of measures that should provide a more complete picture of
particular aspects of their performance related to the programs'
intended purposes.  Finally, the selected programs also set fiscal
year 1998 targets for most of their measures. 


      GOALS AND PERFORMANCE
      MEASURES GENERALLY RELATED
      TO THE PROGRAMS' INTENDED
      PURPOSES
---------------------------------------------------------- Letter :4.1

In their fiscal year 1998 budget presentations, each of the five
credit programs established goals and performance measures that
appeared to be generally related to the programs' intended purposes. 
For example, to monitor its performance in achieving one of its
intended purposes, namely helping veterans retain their homes, VA's
Loan Guaranty Program established the Foreclosure Avoidance Through
Servicing (FATS) Ratio measure.  According to VA, the FATS ratio is
to provide data on the extent to which foreclosures would have been
greater had VA not pursued alternatives to foreclosure, such as
intervening with the holder of the loan on behalf of the borrower to
set up a repayment plan.  For each of the selected credit programs,
appendix III shows the intended purposes identified by those programs
and their respective agencies and examples of related goals and
performance measures the programs established in their fiscal year
1998 budget presentations. 

In some cases, the programs' goals or performance measures addressed
intermediate results that the programs expected to lead to an
intended purpose.  For example, the performance measure that VA's
Loan Guaranty Program established to monitor its performance in
achieving its intended purpose of helping veterans purchase homes was
the percentage of respondents to VA's Lender Customer Satisfaction
Survey who say they are satisfied with their overall interaction with
VA.  According to VA, "maximizing lender satisfaction with their
dealings with VA employees .  .  .  will encourage lenders to
participate in the program, expanding financing opportunities for
veterans." In this way, the intermediate result of increasing lender
satisfaction could be expected to contribute to helping veterans and
active duty personnel purchase homes. 

Similarly, Education's direct student loan program established as
performance measures the rate of (1) institutional (i.e., school)
participation; (2) overall satisfaction of schools with the direct
student loan program; and (3) institutional retention in the program. 
The program uses schools as the vehicles for providing loans to
students and their families.  By providing a streamlined loan
delivery system, the program expects to attract schools to
participate in the program.  Further, by satisfying participating
schools, the program expects to encourage those schools to stay as
participants in the program.  Thus, increases in schools'
participation, satisfaction, and retention in the program are
intermediate results that the program expects will lead to broader
student access to capital for postsecondary education. 

Some of the selected programs established a range of measures to
provide a more complete picture of particular aspects of their
performance related to their intended purposes.  For example, VA's
Loan Guaranty Program established two performance measures for
monitoring the timeliness of issuing a certificate of eligibility,
which is related to its intended purpose of treating all veterans in
a timely manner.  These measures were (1) the percentage of veterans
responding to a VA veteran survey who say they are satisfied with the
time it takes VA to certify veterans' eligibility for a home loan;
and (2) the average time VA calculated it took to issue a certificate
of eligibility, which is to supplement the survey data.  Similarly,
Education established several performance measures for assessing its
guaranteed and direct student loan programs' performance toward
successfully managing the programs in a cost-effective manner.  These
measures included the programs' lifetime gross dollar default rates;
lifetime net default rates (i.e., loss rate); annual delinquency
rates; per unit administrative costs; and the annual collection
rates. 


      THE CREDIT PROGRAMS SET
      FISCAL YEAR 1998 TARGETS FOR
      MOST OF THEIR MEASURES
---------------------------------------------------------- Letter :4.2

The Results Act defines a performance goal as the target level of
performance expressed as a tangible, measurable objective against
which achievement is to be compared.  Thus, annual performance goals
should consist of two parts:  (1) the performance measure that
represents the specific characteristic of the program that the
program uses to gauge its performance and (2) the annual target level
of performance to be achieved during a given fiscal year for the
measure.  The Results Act also requires each agency to report to the
President and Congress annually, beginning for fiscal year 1999, on
the degree to which the agency is meeting its annual performance
goals.  Thus, under the Act, an agency is to monitor and report on
its actual performance during the year compared to the targets it had
established for its performance measures for that year. 

As shown in table 1, we found that the selected programs set fiscal
year 1998 targets for most of their respective measures.  These
included measures for which the programs either (1) set fiscal year
1998 target levels of performance; or (2) reported prior year
baseline data for those measures and indicated how performance on
those measures was to change (i.e., increase or decrease) in fiscal
year 1998 relative to their baselines.  Thus, if the selected
programs collect accurate corresponding data on their actual
performance, they should be able to monitor their progress in
achieving desired results on those measures and have fiscal year 1998
baseline data to use in setting future targets for those measures. 
For example, one of the fiscal year 1998 performance goals set by
Education's guaranteed student loan program is that the "Level of
[overall school] satisfaction will meet or exceed the level of school
satisfaction measured last year, [where] 67 percent of the schools
reported satisfaction." However, for other measures, the program did
not set targets that could be used to monitor their progress on those
measures.  For example, Education's guaranteed student loan program
did not establish a fiscal year 1998 target for its annual
delinquency rate measure.  The program reported that this measure
will provide information on the dollar amount of loans "past due" as
a percentage of dollars in repayment and that baseline data for the
measure will be developed as the definition of "past due" is
finalized. 



                                Table 1
                
                  Number of Performance Measures With
                        Fiscal Year 1998 Targets

                                              Number of performance
                                                     measures
                                            --------------------------
                                                          With FY 1998
Program                                            Total     targets\a
------------------------------------------  ------------  ------------
VA's Loan Guaranty Program (guaranteed                17            13
 home loans)
USDA's SFH direct and guaranteed loan                 10             9
 programs\b
Education's direct student loan program               13             9
Education's guaranteed student                        14             8
 loan program
----------------------------------------------------------------------
\a The number of measures with fiscal year 1998 targets include those
measures where the programs either (1) set fiscal year 1998 target
levels of performance; or (2) reported prior year baseline data for
those measures and indicated how performance on those measures was to
change (i.e., increase or decrease) in fiscal year 1998 relative to
their baselines. 

\b According to a USDA official, USDA established one set of
performance measures and targets to monitor the combined performance
of its direct and guaranteed SFH loan programs as well as its Housing
Repair Loan Program and its Very-Low-Income Repair Grant Program. 

Source:  Selected programs' measures and targets as of May 1, 1997,
which, according to agency officials, were generally the same ones
the programs submitted with their respective agencies' fiscal year
1998 budget presentations to OMB and Congress. 


   THE CREDIT PROGRAMS HAVE BEEN
   FACING THREE GENERAL CHALLENGES
   TO DEVELOPING PERFORMANCE
   INFORMATION
------------------------------------------------------------ Letter :5

On the basis of agency officials' responses to questions on
developing performance information for the selected programs, we
identified three general challenges those programs have been facing: 
(1) a struggle in reaching consensus among stakeholders on the
programs' intended purposes, performance measures, and target levels
of performance; (2) difficulty in separating the effects of external
forces from program influences on results; and (3) a lack of relevant
program performance and financial baseline data.  Agency officials
also described some approaches they were taking to address the
challenges they have been facing. 


      A STRUGGLE IN REACHING
      CONSENSUS AMONG STAKEHOLDERS
      ON THE PROGRAMS' INTENDED
      PURPOSES, PERFORMANCE
      MEASURES, AND TARGET LEVELS
      OF PERFORMANCE
---------------------------------------------------------- Letter :5.1

As we have noted in a previous report, because the interests of a
program's stakeholders can and often do differ significantly, full
agreement among those stakeholders on all aspects of the program's
performance is relatively uncommon.\12 However, our past and current
work also has shown that although it is difficult to get stakeholders
to reach agreement, stakeholder involvement can help an agency
identify results-oriented performance measures and set realistic
target levels.\13 For example, VA officials said that they had
difficulty reaching consensus with internal program stakeholders at
the agency's field offices.  This difficulty concerned how the
activities of the field offices would be linked to achieving the
intended results of VA's Loan Guaranty Program as established by
headquarters staff.  To address this difficulty, VA headquarters and
field office staff worked together to develop the program's
performance measures.  They said the program then brought together
key VA headquarters and field managers to reach agreement on target
performance levels for linking field office activities to the
intended results of the program. 

VA's Loan Guaranty Program also struggled with trying to reach
consensus with OMB.  According to a VA official, OMB suggested that
the program establish "outcome-oriented" performance measures, where
feasible, which could provide data on the extent to which the program
is helping veterans achieve a higher rate of homeownership.  However,
VA officials said that the program is an entitlement program (i.e.,
veterans receive the benefit regardless of need as a reward for their
service); and it is not clear whether increasing homeownership among
veterans is a primary intended purpose of the program.  According to
the officials, the more appropriate performance measures for
assessing the program's performance and holding it accountable are
those for monitoring how well VA is delivering the benefit (e.g.,
satisfying veterans and keeping program costs down). 

To address OMB's suggestion, the program established a homeownership
assistance measure in its fiscal year 1998 budget presentation, which
is to provide data on the percentage of veterans surveyed who said
they would not have been able to purchase any home or would have had
to purchase a less expensive home without a VA guaranteed loan. 
However, the program did not identify the homeownership assistance
measure as a performance measure.  Instead, the program identified
this measure as a measure of workload and other program data, which a
VA official said was to provide "contextual program information,"
rather than information for gauging the program's progress in
achieving its intended purposes and holding the program accountable. 
The VA official added that the homeownership assistance measure is an
"imperfect measure" because of its reliance on self-reporting by
veterans. 

Similarly, at the time of our interviews with USDA officials, they
said internal stakeholders were grappling with what the appropriate
results for their SFH loan programs were.  According to those
officials, the struggle of trying to reach consensus among those
stakeholders contributed to why they rated as a significantly
challenging activity "developing measures for assessing the net
effect of the program." One agency official said that the programs'
intended purposes were "putting people [who are unable to get credit
from other sources] in homes"; thus, to the extent that they put such
borrowers in homes, the programs are having a net impact.  However,
another agency official suggested that the programs' intended
purposes also should include "improving the quality of life among
rural residents" and "improving housing conditions and the economy in
a given community or state." He said that measures for gauging the
programs' progress toward such purposes would attempt to collect data
on, for example, the extent to which putting people in homes is
improving the quality of life among rural residents. 

USDA officials also said it is difficult to balance stakeholders'
interests.  They said that they are expected to increase program
service while also reducing program costs and minimizing default
rates.  However, they said this is difficult because the SFH programs
were designed to offer credit to a population that the private sector
would consider high risk.  Specifically, to be eligible for a SFH
direct loan, a borrower must have a family income that is "very low"
to "low" (i.e., a family income under 80 percent of the median income
in the area); and the borrower must be unable to get credit from any
other source.  Therefore, the program's target population may be more
likely to default.  At the time of our review, USDA, VA, and
Education were working with stakeholders in and outside of the
selected programs, including OMB and Congress, to reach consensus
with those stakeholders on the most appropriate goals and measures
for the programs. 


--------------------
\12 GAO/GGD-96-118, June 1996. 

\13 GAO/GGD-97-109, June 2, 1997. 


      DIFFICULTY IN SEPARATING THE
      EFFECTS OF EXTERNAL FORCES
      FROM PROGRAM INFLUENCES ON
      RESULTS
---------------------------------------------------------- Letter :5.2

The efforts of federal agencies often are but one factor among many
that may influence whether, and the degree to which, their programs
achieve their intended results.  Our past and current work has found
that many agencies have been challenged to separate out the influence
that their program activities have had on the achievement of program
results when those results also could have been influenced by
external forces.\14

Agency officials from all five of the selected credit programs
reported as a significantly challenging activity "separating the
impact of the credit programs' activities from the impact of other
factors external to those programs but contributing to the results
achieved." They generally cited economic trends; the role of third
parties in helping the programs provide loans; and the existence of
other federal financial aid programs (e.g., grants) as examples of
forces external to their programs that can affect program results. 

For example, the foreclosure rate could be viewed as a measure
related to the VA Loan Guaranty Program's intended purpose of helping
veterans and active duty personnel retain homes.  However, VA
reported that external forces, such as interest rates, unemployment,
and the general state of the economy, can influence the foreclosure
rate.  A VA official said that because of such external forces, it
has been difficult to confidently attribute a change in the
foreclosure rate to the program's activities and thus view it as a
valid measure of the program's performance. 

VA officials said that the program attempted to address this problem
in 1993 but was not successful.  Specifically, a VA program official
said that at the encouragement of OMB, the program attempted to
develop a model to help the program estimate its foreclosure rate and
monitor its performance.  However, he said when the program
implemented the model, it significantly overestimated the number of
VA foreclosures and thus was not an adequate model for determining
the external forces that could affect the rate.  The VA official said
that because of the many external forces that could affect the number
of foreclosures, it was unclear if the model could be adjusted to
help it adequately predict foreclosures and whether--given the cost
of potentially making many adjustments to the model--the value of
doing this was worth the additional cost.  Therefore, the program
took another approach.  The program established a surrogate measure,
the FATS ratio (which, as mentioned earlier, is to provide data on
the extent to which foreclosures would have been greater had VA not
pursued alternatives to foreclosure), for monitoring its performance
in assisting veterans to avoid foreclosures.  The program views the
FATS ratio as a more valid measure than the foreclosure rate for
assessing the program's performance in helping veterans and active
duty personnel retain homes. 

USDA officials also said that separating the impact of their SFH
guaranteed and direct loan programs' activities from the impact of
other external forces on, for example, the quality of life for rural
residents is exceedingly difficult.  These officials said that the
quality of rural residents' lives could be affected negatively by
other, unrelated events, such as borrowers' incurring health problems
or financial hardship, or the closing of a military base eliminating
jobs in the area.  They said that although providing single-family
housing loans may help to improve the quality of life among those
rural residents, such improvement could also be due to other external
forces, such as home loans being provided by other federal housing
loan programs (e.g., programs administered by VA or the Department of
Housing and Urban Development). 

Similarly, Education officials told us that loans that are issued or
guaranteed through their agency's direct or guaranteed student loan
programs are among several types of financial aid that Education
offers to help ensure access to postsecondary education.  They said
that students' college participation and completion rates can be
affected by borrowers' eligibility for loans through the direct or
guaranteed loan programs as well as by such external forces as the
eligibility of borrowers for the other types of financial aid
assistance provided by Education (e.g., grants); the extent of
parental support for the borrowers attending school; and the
borrowers having to financially support a family.  Although Education
established student participation and completion rates as performance
measures, it did so to monitor the combined performance of its
financial aid programs instead of the specific performance of either
the direct or guaranteed student loan programs on those results. 

Education officials said that an approach the agency was taking to
better understand the determinants of college enrollment--including
financial aid obtained through direct loans, guaranteed loans,
grants, or other financial aid programs--was contracting for a study
of the effects of financial aid, including aid provided by these
programs, and various external forces on this result.  According to
an Education official, this study is expected to be completed in
early 1998.  We believe the findings from this study may help to
inform any future program evaluations assessing the impact of
Education's financial aid programs on results compared to the impact
of external forces.  In a prior report, we discussed how impact
evaluations can help an agency confidently attribute the achievement
of intended results to one or more of its programs by providing
information on the extent to which those programs contributed to the
results achieved relative to the impact of external forces.\15


--------------------
\14 GAO/GGD-97-109, June 2, 1997. 

\15 GAO/GGD-97-109, June 2, 1997. 


      LACK OF RELEVANT BASELINE
      DATA
---------------------------------------------------------- Letter :5.3

As we reported in June 1997, our prior work has shown that baseline
and trend data on past performance can help agencies set realistic
target levels of performance for their programs given the past
performance of those programs.\16 However, we also noted that because
agencies often did not focus on having results-oriented performance
information in the past, they generally have not collected such data. 
Thus, they have not had all of the baseline and trend data they
believed they needed to set goals.  Further, credit agencies,
including VA, USDA, and Education, generally have had difficulty
producing reliable performance data, particularly financial data,
which executive and legislative branch decisionmakers need to make
well-informed decisions. 

Agency officials from all five of the credit programs said that a
lack of baseline data was why they rated one or more of the following
activities as significantly challenging:  "developing objective,
quantifiable, and measurable annual program performance goals";
"determining a realistic target level of performance"; "developing
unit cost information for the programs' outputs"; and "developing
unit cost information for the programs' outcomes." For example, VA
officials said that some of their performance measures for their Loan
Guaranty Program were new, and baseline data were thus not available
on those measures.  Consequently, VA did not have data on past
performance to use in setting some of the program's fiscal year 1998
target levels of performance and reported that those targets were "to
be determined."

Education officials also attributed the challenges they had in
determining realistic target levels of performance for Education's
direct student loan program to a lack of baseline data.  According to
those officials, the program had not been in existence long enough to
have historical data on many of the program's measures to use in
setting fiscal year 1998 targets.  They said that a lack of
historical data for the direct loan program was a particular problem
in terms of predicting borrower repayment behavior, since few
borrowers had yet entered the repayment phase.  Thus, to set the
target for the program's default rate measure, Education used
historical data for the same measure established for Education's
guaranteed student loan program. 

Similarly, USDA officials said the agency's SFH programs did not have
information systems to collect data on some performance measures,
such as the number of loans made in targeted geographic areas.  Thus,
according to those officials, the programs did not have an informed
basis on which to set fiscal year 1998 target levels of performance
for those measures and did not include them in their fiscal year 1998
budget submission to OMB.  Rather, the program included substitute
measures for which the programs had information systems in place to
collect the data (e.g., the number of rural families with improved or
more suitable housing conditions). 


--------------------
\16 GAO/GGD-97-109, June 2, 1997. 


         LACK OF RELIABLE
         FINANCIAL DATA
-------------------------------------------------------- Letter :5.3.1

Our prior and ongoing work and that of agencies' internal or
independent auditors have found that some credit agencies still have
difficulty, despite numerous years of experience, in producing
reliable financial data, such as credit programs' subsidy rates--the
estimated cost to the government from direct loans and loan
guarantees.\17 For example, USDA and Education received a disclaimer
of an opinion from internal and independent auditors, respectively,
on their fiscal year 1996 financial statements.  In part, this was
due to those agencies not being able to provide the data needed to
(1) accurately reflect the cost to the government and (2) permit
appropriate cost comparisons between credit and noncredit programs. 

VA received an unqualified opinion on its fiscal year 1996 financial
statements from VA's Office of Inspector General (OIG).  However, the
OIG audit, which included a review of the Loan Guaranty Program,
found that the program did not reliably accumulate the financial
information needed to comply with federal financial accounting
standards, identified significant errors that required financial
statement adjustments, and identified other errors where data
compiled manually did not always reconcile with original source
amounts. 

If successfully implemented, the CFO Act will help credit agencies
resolve long-standing problems with data reliability.  Further, in
passing the Results Act, Congress emphasized that the usefulness of
agencies' performance data depends, to a large degree, on the
reliability of those data.  Therefore, the Results Act requires
agencies to describe in their annual performance plans the means to
be used to verify and validate performance data.  We have suggested
in prior reports that such information, including information about
the reliability of credit agencies' performance data, could be
equally important for those agencies to disclose in their reports to
ensure report users of the quality of that data.\18

One area in particular need of attention is the development of
reliable financial information on the full cost and unit cost of a
program, which is an integral part of measuring that program's
efficiency and cost effectiveness.\19 An essential step in developing
such information is the identification of individual program costs,
such as direct labor.  In that regard, unit cost information can be
particularly useful in identifying trends and determining key cost
drivers of the program.  Agency officials from all five of the credit
programs we reviewed said they have been challenged to develop unit
cost information for the programs' outputs and outcomes.  They
generally cited difficulties in allocating basic cost data to
specific programs as a reason for this challenge.  For example, USDA
officials explained that its field offices are involved in
administering the SFH direct and guaranteed loan programs as well as
other USDA programs.  They said difficulties in separating data on
labor costs for the various programs, for example, have contributed
to the challenge they have faced in developing unit costs for the SFH
programs. 

Similarly, VA officials said that developing meaningful unit cost
information for the Loan Guaranty Program has been a significantly
challenging activity due to the lack of an adequate methodology,
including VA's inability to separate the data on the actual costs for
the program, such as labor costs, from the costs for the several
other programs that VA field offices administer.  A VA official said
that because of the difficulty it has had in isolating program cost
data, the program uses a "very loose" process to calculate unit
costs, which involves dividing the various resource levels authorized
for the program, such as staffing levels (i.e., authorized full-time
equivalents)\20 by the activity level during the year (e.g., number
of loans guaranteed). 

New accounting standards developed by the Federal Accounting
Standards Advisory Board (FASAB) require federal agencies subject to
the CFO Act and Results Act to collect relevant and reliable data on
the full costs of carrying out a mission or producing products or
services.  Although these standards were scheduled to be effective
for all federal programs beginning with fiscal year 1997, because of
serious shortfalls in agencies' cost accounting systems, FASAB
extended the date by 1 year to fiscal year 1998.  The standards took
effect on October 1, 1997.  Credit agency officials said their
respective agencies were establishing information systems to collect
needed cost data.  Specifically, VA officials said their agency was
looking at the use of activity-based costing (ABC) to develop more
meaningful unit cost information.\21 According to an Education
official, his agency also is developing a system that should be able
to provide at least some data on unit costs by 1999.  USDA reported
in its fiscal year 1998 budget presentation that it was working to
develop the data for its SFH direct and guaranteed loan programs'
measure on the "cost of housing a family per recipient household."


--------------------
\17 Federal Credit Programs:  Agencies Had Serious Problems Meeting
Credit Reform Requirements (GAO/AFMD-93-17, Jan.  6, 1993);
GAO/T-AIMD-96-10, Dec.  14, 1995; and GAO/AIMD-97-48, June 2, 1997. 

\18 GPRA Performance Reports (GAO/GGD-96-66R, Feb.  14, 1996); and
GAO/AIMD-97-48, June 2, 1997. 

\19 The full cost of a program is calculated as the sum of all
resources used by the program that indirectly or directly contributed
to producing the program's products and services, including resources
provided by entities outside of the program.  A credit program's unit
costs are the total costs per unit of loan issued or guaranteed, or
of service provided by that program. 

\20 A full-time equivalent consists of one or more employed
individuals who collectively complete 2,080 work hours in a given
year.  Therefore, both one full-time employee and two half-time
employees equal one full-time equivalent. 

\21 ABC is a methodology that assigns costs to products or services
on the basis of the resources they consume.  ABC gives visibility to
how effectively resources are being used and how all relevant
activities contribute to the cost of a product or service. 


   PROGRESS IN DEVELOPING COMMON
   PERFORMANCE MEASURES HAS BEEN
   LIMITED
------------------------------------------------------------ Letter :6

According to OMB and the Working Group, comparing results using
common measures across credit programs allows program managers and
other decisionmakers to identify best practices among those programs
that have the potential for improving other credit programs'
performance.  In addition, the Working Group anticipated that
agencies that administer credit programs could include such measures
in their annual performance plans and reports under the Results Act. 
By September 1996, a Working Group task force had proposed a set of
common financial and programmatic performance measures for federal
credit programs.  Some of the proposed measures--namely, financial
measures for meeting the annual budgeting and reporting requirements
of DCIA and FCRA--have been adopted by major credit agencies. 

However, two general problems have limited the Working Group's
progress in developing common performance measures for credit
programs.  First, as previously noted, several credit programs lack
relevant program performance and financial baseline data.  Second,
the Working Group has been unable to reach consensus on the
appropriateness of decisionmakers using some of the task force's
proposed measures to assess the performance of individual credit
programs or to compare that performance against the performance of
other credit programs.  In addition, OMB does not intend to require
credit programs to adopt common performance measures when consensus
about the appropriateness of such measures has not been achieved. 


      COMMON PERFORMANCE MEASURES
      FOR CREDIT PROGRAMS HAVE
      BEEN PROPOSED, AND SOME HAVE
      BEEN ADOPTED
---------------------------------------------------------- Letter :6.1

In July 1995, the Working Group established a task force to develop
common performance measures that could help agencies and other
decisionmakers make relevant comparisons of the results of credit
programs.  By September 1996, the task force had proposed a set of
common financial and programmatic measures in the following four
areas: 

(1) Financial performance.  Measures in this area include total
receivables; total delinquent debt; default rates; actual versus
projected subsidy rates; and administrative costs, such as the costs
of extending credit and servicing loans. 

(2) Program performance in achieving desired loan characteristics. 
Measures in this area include the percentage of loans going to
borrowers who would otherwise not have access to private credit and
the percentage of borrowers pleased with the timeliness and quality
of credit program service. 

(3) Program effects on society.  Measures in this area include (1)
intended effects, such as "supporting investment important to the
economy" as monitored by, for example, the amount and quality of
low-income housing financed (home loan programs) and business
investment financed (business loan programs); and (2) unintended
effects, such as borrowers accumulating excessive debt burden. 

(4) Program "additionality." Measures in this area indicate the
results achieved by the program by providing financial assistance to
borrowers that private markets will not serve.  An example of a
measure in this area is the net increase in homeownership as a result
of the program supplementing versus substituting for private
financing. 

OMB and the Working Group's task force also have encouraged credit
programs to establish, where appropriate, program-specific measures
and explanatory data for helping to ensure a complete assessment of
their programs' performance and to explain the results to users of
performance information. 

In its June 1997 1997 Federal Financial Management Status Report and
Five-Year Plan, OMB and the CFO Council reported that major credit
agencies have adopted the proposed common financial measures for
assessing their debt collection activities.\22 For example, credit
agencies are to collect data on total receivables, total delinquent
debt, and total collections for their credit programs.  According to
an OMB official, these and other common measures that the task force
proposed in the area of financial performance are measures that major
credit agencies adopted for meeting the annual budgeting and
reporting requirements of DCIA and FCRA.  However, he said that those
agencies have not yet adopted measures that the task force proposed
under the other three areas--measures for monitoring and improving a
credit program's (1) performance in achieving desired loan
characteristics, (2) effects on society, and (3) additionality. 


--------------------
\22 Authorized by the CFO Act, the CFO Council is an interagency,
governmentwide body that comprises the CFOs and Deputy CFOs of the 24
largest federal agencies and senior OMB and Department of Treasury
officials.  The Council was established to address critical
crosscutting financial issues. 


      TWO GENERAL PROBLEMS HAVE
      LIMITED PROGRESS IN
      DEVELOPING COMMON
      PERFORMANCE MEASURES
---------------------------------------------------------- Letter :6.2

Although the common financial measures proposed by the task force
have been adopted, two general problems have limited the Working
Group's progress in developing common performance measures.  First,
as previously noted, several credit programs lack relevant program
performance and financial baseline data.  Second, the Working Group
has been unable to reach consensus on the appropriateness of
decisionmakers using some of the task force's proposed measures to
assess the performance of individual credit programs or to compare
that performance against the performance of other credit programs. 
For example, some programs give credit only to persons who are unable
to obtain credit from other sources; other programs give credit to
anyone who is entitled to the programs' benefits, regardless of his
or her access to credit.  Thus, officials in some credit agencies
questioned whether it would be appropriate to make comparisons among
the default rates of credit programs, because the financial
characteristics of borrowers for each program may be different. 


         RELEVANT DATA ON SEVERAL
         OF THE COMMON MEASURES
         PROPOSED ARE LACKING
-------------------------------------------------------- Letter :6.2.1

Agency officials told the task force that for several of the proposed
measures, their agencies' credit programs generally did not collect
data relevant to those measures, collected incomplete data, or did
not routinely collect such data.  Such measures included the
percentage of borrowers who were pleased with the timeliness and
quality of credit program service and measures for monitoring the
effects of credit programs on society and program additionality.  For
example, common measures proposed by the task force for monitoring
the performance of home loan programs and business loan programs in
"supporting investment important to the economy"--one of four
intended effects on society the task force proposed--included the
amount and quality of low-income housing financed and business
investment financed, respectively.  Officials from VA, the Department
of Housing and Urban Development (HUD), and the Small Business
Administration told the task force that the credit programs at their
respective agencies generally did not collect such data.  Moreover,
even when data existed, our prior and ongoing work and audits by
credit agencies' inspectors general and others have consistently
disclosed serious weaknesses in agencies' systems, which has affected
the reliability of data that are used to account for and manage
credit programs.\23


--------------------
\23 GAO/AFMD-93-17, Jan.  6, 1993; GAO/T-AIMD-96-10, Dec.  14, 1995;
and GAO/AIMD-97-48, June 2, 1997. 


         THE WORKING GROUP HAS
         BEEN UNABLE TO REACH
         CONSENSUS ON THE
         APPROPRIATENESS OF SOME
         OF THE PROPOSED MEASURES
-------------------------------------------------------- Letter :6.2.2

Consistent with the views of OMB and the Working Group, our work over
the last few years has recognized that common performance measures
for similar programs can provide important information for permitting
comparisons of the results of those programs and the methods used to
achieve those results.\24 Such information could help program
managers identify credit program performance gaps; set improvement
goals; improve credit program processes; and inform other
decisionmakers, such as OMB and Congress.  However, we found that
members of the Working Group have been unable to reach consensus on
the appropriateness of some of the common performance measures
proposed by the task force. 

Specifically, officials in some agencies questioned whether data
collected from some of the proposed common measures would be
meaningful for assessing the performance of their agencies' credit
programs.  For example, VA and Education officials said that their
agencies' Loan Guaranty Program and direct and guaranteed student
loan programs, respectively, are entitlement programs in which the
government is obligated to give credit to anyone who qualifies for
the programs' benefits, regardless of his or her access to credit.\25
Thus, these officials questioned whether a common programmatic
measure proposed by the task force, "percent of loans or guarantees
originated going to borrowers who would otherwise not have access to
private credit," was meaningful for their programs.  As mentioned
earlier, to be eligible for a USDA SFH direct loan, a borrower must
have a family income that is "very low" to "low," and the borrower
must be unable to get credit from any other source.  A USDA official
said that the target and result for this proposed measure would
always be 100 percent; thus, he also questioned whether the measure
was meaningful for that program. 

In addition, for certain measures, officials in some credit agencies
questioned whether the data collected would be appropriate to use in
making comparisons among different credit programs' performance
results, such as the programs' default rate, because the financial
characteristics of borrowers for each program may be different.  For
example, USDA officials said that because the borrower must be unable
to obtain private credit, the program's target population may be more
likely to default.  Thus, according to USDA officials, it may be
inappropriate to compare, for example, the actual default rate of the
SFH direct loan program with the actual default rate of the VA Loan
Guaranty Program, which is an entitlement program. 

We agree that common measures need to be carefully explained to help
ensure that significant program differences are properly interpreted. 
However, it is not clear that such differences outweigh the potential
usefulness of common measures.  Credit agencies have generally agreed
with our suggestion that they provide explanatory information where
necessary in agency reports to portray program differences more
appropriately and help users of performance information understand
the reported performance.\26

Also related to the problem of comparable data, we recently reported
that agencies' use of inconsistent definitions for their programs'
measures could hamper decisionmakers' use of data collected from
those measures in planning, comparing performance, and reporting on
performance achieved.\27 Our June 1997 report noted that some credit
agencies differ in how they classify previously delinquent debt on
which borrowers are currently making payments.\28 Some reclassify
such debt as "current," but others keep it in a delinquent category
regardless of the current payment status.  Although such
classification practices may be suitable within an agency, they make
it difficult to compare agency performance or aggregate data for
similar programs.  For example, VA loans maintain their delinquent
status until the delinquency is repaid or written off.  Conversely, a
home loan program within HUD reclassified single-family delinquent
loans to a current repayment status when borrowers complied with
forbearance terms, which typically included making partial mortgage
payments for up to 3 years.  In our June report, we recommended that
the Department of the Treasury's Financial Management Service work
with major credit agencies and OMB to help those agencies
consistently report on delinquent debt or disclose their
inconsistencies.  Treasury, OMB, and the major credit agencies
generally agreed with that recommendation, and the agencies commented
that consistent application of governmentwide debt collection
reporting criteria is essential. 

Similarly, we observed that when consistent definitions do not exist
among credit programs, it is difficult to know whether results
reported would be readily comparable.  For example, one of the common
programmatic measures proposed by the task force for monitoring loan
characteristics was the "percent of borrowers who are pleased with
the timeliness and quality of credit program service." However, for
this proposed task force measure, it is unclear what is meant by
"timeliness" and "quality" of service, because each credit program
may have a different interpretation.  Education's guaranteed and
direct student loan programs, for example, established a broad
measure of "overall borrower satisfaction"; VA's Loan Guaranty
Program established more specific borrower satisfaction measures--the
percent of respondents surveyed who will say (1) they are satisfied
with their contact with VA; (2) they are satisfied with the time it
took to obtain a Certificate of Eligibility; and (3) their loan did
not take longer to process than expected as a result of a delay
blamed on VA (which is to help the program monitor when program staff
may need to work with the lender in identifying reasons for the delay
in loan processing time, toward improving performance on this
measure).  Thus, it is unclear whether the data collected would be
comparable if this proposed measure of the task force were used by
all credit programs. 

The Working Group anticipated that agencies that administer credit
programs could include common financial and programmatic measures in
their annual performance plans and reports under the Results Act. 
According to OMB's Senior Advisor for Cash and Credit Management, at
an upcoming meeting of the Working Group OMB intends to ask members
whether they want to continue to address their concerns about
adopting additional common performance measures for credit programs. 
However, he said that OMB is not taking a prescriptive role in
directing agencies' performance measurement activities.  Rather, he
said the administration wants agencies to take the initiative on such
activities.  According to this official, if members choose not to
address those concerns, OMB does not intend to require credit
programs to adopt common performance measures when consensus about
the appropriateness of such measures has not been achieved.  Thus, it
is unclear when, or if, agencies that administer credit programs will
include common measures for those programs in their annual
performance plans under the Results Act. 


--------------------
\24 GAO/AIMD-97-146, Aug.  29, 1997; GAO/AIMD-97-48, June 2, 1997;
and GAO/GGD-96-118, June 1996. 

\25 Under entitlement or mandatory programs, the government is
obligated by provisions of law to make a loan directly or guarantee a
loan made by a private lender to an eligible borrower. 

\26 GAO/AIMD-97-48, June 2, 1997. 

\27 GAO/GGD-97-109, June 2, 1997; and GAO/AIMD-97-48, June 2, 1997. 

\28 GAO/AIMD-97-48, June 2, 1997. 


   CONCLUSIONS
------------------------------------------------------------ Letter :7

The credit programs we reviewed have established goals and
performance measures that appeared to be generally related to the
programs' intended purposes and set target levels of performance for
most of their respective measures.  In addition, our prior work
suggests that the development of common measures, where appropriate,
can provide important information for permitting comparisons of
similar programs' results and the methods they used to achieve those
results.  We have suggested that agencies' annual performance plans
and reports under the Results Act can serve as suitable vehicles for
developing such measures and providing such information.  The Working
Group has focused on developing common financial and programmatic
performance measures for credit programs and anticipated that
agencies that administer credit programs could include such measures
in their annual performance plans and reports under the Results Act. 

However, two general problems have limited the Working Group's
progress in developing common performance measures for credit
programs:  (1) the lack of relevant program performance and financial
baseline data for several credit programs and (2) the inability among
members of the Working Group to reach consensus on the
appropriateness of some of the proposed measures. 

OMB does not intend to require credit agencies to adopt common
performance measures when consensus about the appropriateness of such
measures has not been achieved.  We agree that OMB should not force
the use of common measures when concerns about their appropriateness
exist.  However, at the time of our review, the Working Group had not
resolved those concerns and had yet to decide how and when those
concerns would be addressed; thus, it is unclear whether OMB and the
credit agencies will maintain their current level of attention to
developing common measures.  Also unclear is the extent to which
agencies that administer credit programs will include common measures
for those programs in their annual performance plans that could
provide useful information to decisionmakers interested in making
performance and cost comparisons. 

We recognize the difficulty the Working Group is facing in reaching
consensus on common performance measures and that significant
differences in program characteristics may limit the usefulness of
some measures for broad, cross-program comparisons.  However, we
believe the potential benefits that could be realized from developing
common performance measures, where appropriate, underscore the
importance of OMB and the credit agencies continuing their efforts to
develop and reach consensus on such measures.  In addition, because
the development and use of performance measures, especially common
measures that can be used across programs and agencies, are in the
early stages of implementation, it will be especially useful for
decisionmakers to evaluate early experiences to identify successful,
as well as unsuccessful, approaches.  To that end, documented
information on the measures considered, how they are used and should
be interpreted, and how they can be improved will be helpful to
agencies in further achieving the purposes of the Results Act. 


   RECOMMENDATIONS TO THE DIRECTOR
   OF OMB
------------------------------------------------------------ Letter :8

Building on recommendations and suggestions we have made in prior
reports, we recommend that the Director of OMB sustain OMB's efforts
to work with major credit agencies to use annual performance planning
under the Results Act as a vehicle for developing common performance
measures across credit programs, where appropriate.  In doing so, we
recommend that beginning with those agencies' fiscal year 2000 annual
performance plans, the Director of OMB require each agency that
administers credit programs to identify in their plans

(1) performance measures the agency is using for its credit
program(s) that are the same as those used by other credit programs
and the strengths and limitations of using those measures to make
performance and cost comparisons among those programs; and

(2) what actions, if any, are being taken or could be taken to refine
the agency's performance measurement efforts to address the
identified limitations to using existing measures to make performance
and cost comparisons across credit programs. 

These or some comparable requirements would serve a twofold purpose. 
First, they would help ensure that the search for common measures
continues.  And second, they would document the results of those
efforts in a way that would (1) permit further analysis directed at
the identification of additional common measures and (2) facilitate
an understanding of any limitations of using existing common measures
to compare results across credit programs. 


   AGENCY COMMENTS AND OUR
   EVALUATION
------------------------------------------------------------ Letter :9

On December 11, 1997, we requested comments on a draft of this report
from the Secretaries of Education and Agriculture, the Acting
Secretary of VA, and the Director of OMB or their designees.  On
December 17, 1997, the liaison to GAO from USDA's Rural Development
mission area, which administers the SFH direct and guaranteed loan
programs, said the Department generally agreed with the draft
report's factual material, conclusions, and recommendations.  A rural
development official later provided minor technical suggestions,
which we included in the report as appropriate.  Similarly, on
December 19, 1997, the liaison to GAO from the Department of
Education also said the Department concurred with the draft report's
general findings and later provided minor technical suggestions,
which we included in the report as appropriate. 

On January 8, 1998, the VA liaison to GAO said the Department also
generally agreed with the draft report's factual material,
conclusions, and recommendations, except for that part of the
recommendation requiring credit agencies to include a discussion
about common measures in their fiscal year 1999 annual performance
plans.  This comment was consistent with the comment provided to us
on January 6, 1998, by a representative of VA's Performance Analysis
Service within VA's Office of Budget.  Specifically, that
representative told us that VA would not likely have time to provide
a meaningful discussion about common measures in its fiscal year 1999
plan because VA (1) had yet to achieve consensus among major
stakeholders on the common measures, (2) had not done the analysis
necessary to provide the discussion about common performance measures
that our draft report recommended, and (3) is to submit the agency's
fiscal year 1999 performance plan to Congress within the next few
weeks.  OMB officials had a similar comment on the feasibility of
requiring a discussion about common measures in credit agencies'
fiscal year 1999 performance plans, which we discuss later in this
section. 

In response to this comment from both VA and OMB officials, we
removed from the recommendation our suggestion that such a discussion
be included in credit agencies' fiscal year 1999 annual performance
plans.  However, we retained the recommendation that the Director of
OMB require credit agencies to include a discussion about common
measures in those agencies' fiscal year 2000 annual performance plans
and subsequent performance plans.  We continue to believe that using
annual performance planning under the Results Act as a vehicle for
developing and discussing common performance measures across credit
programs, where appropriate, could provide decisionmakers with
important information and help agencies further achieve the purposes
of the Results Act.  Such information could be useful to
decisionmakers in comparing similar programs' results and the methods
they used to achieve those results and for understanding how such
measures are used, should be interpreted, and could be improved. 

Further, in a conversation with us on December 17, 1997, a
representative of VA's Loan Guaranty Service commented that
developing information systems to collect data from common
performance measures may be costly and that before VA developed
systems to collect such data, the costs of doing so should be weighed
against the benefits.  We agree that collecting needed data from
common measures is a problem that credit agencies face, as discussed
in the draft report.  We also have suggested that annual performance
plans provide agencies with the opportunity to alert Congress to the
problems they have had or anticipate having in collecting needed
data, including the cost and data quality trade-offs associated with
various collection strategies.\29 The representative of VA's Loan
Guaranty Service also provided minor technical suggestions, which we
included in this report where appropriate. 

On January 7, 1998, we met with the OMB Senior Advisor for Cash and
Credit Management and the Senior Advisor to the Deputy for
Management, who said the report would serve as a valuable tool and
resource in OMB's continuing efforts to encourage and work with major
credit agencies to effectively implement the Results Act.  However,
those officials cautioned that based on OMB's experiences in working
with major credit agencies to draft their fiscal year 1999
performance plans and to identify appropriate, results-oriented
common performance measures for credit programs, developing and
reaching consensus on credit program performance measures will
continue to be difficult, time-consuming, and iterative.  They said
that those experiences and OMB's review of major credit agencies'
fiscal year 1999 performance plans suggested to them that the
priority of those agencies at this point in the implementation of the
Results Act needs to be on ensuring the quality of the performance
goals and measures for their individual credit programs.  According
to these OMB officials, in developing program-specific as well as
common performance measures for credit programs, major credit
agencies need to continue working to reach consensus among the key
stakeholders of the agencies' credit programs and to develop
information systems for collecting needed performance data, which
will be challenging.  Moreover, the OMB officials said that agencies
are concluding the preparation of their fiscal year 1999 annual
performance plans that will accompany the President's budget
submission to Congress in February 1998.  The officials said that
those plans reflect budget, policy, and programmatic decisions
already made in the course of preparing the budget.  Thus, the
officials believed it would not be feasible for the Director of OMB
to direct credit agencies to include in their fiscal year 1999 plans
the discussion about common measures that our draft report
recommended. 

OMB officials said that given the challenges of developing
appropriate, results-oriented common performance measures for credit
programs, it is unclear when or if such measures could be adopted by
major credit agencies.  OMB's Senior Advisor for Cash and Credit
Management suggested that as the Working Group's efforts advance, two
common results-oriented performance measures that may be considered
appropriate for agencies that administer credit programs are (1) the
number of loans a program made that are repaid successfully and (2)
the percentage of customers satisfied with the program.  However, he
said that developing measures that would help isolate a credit
program's contribution to achieving a particular common result from
the contribution of external factors may not be possible for all
credit programs.  The OMB officials also noted that performance
information is just one key factor among many that will go into
decisionmaking on management and budget policy issues. 

The OMB officials told us that OMB will ensure in calendar year 1998
that developing common results-oriented performance measures across
credit programs is a priority agenda item for discussions among the
Working Group members.  They said that as part of these discussions,
major credit agencies could share with one another their experiences
in developing their individual fiscal year 1999 performance plans and
their congressional committees' reactions to those plans.  On the
basis of agencies' experiences and congressional reactions, the OMB
officials said they believe that OMB and the major credit agencies
would have a better foundation from which to discuss common
performance measures for the agencies' credit programs.  According to
the OMB officials, this experience will provide OMB and the agencies
with the basis for determining the feasibility of incorporating
common measures and a discussion about such measures into future
annual performance plans, where appropriate. 

We believe that OMB's planned approach to use major credit agencies'
fiscal year 1999 performance planning efforts as the foundation for
discussions among these agencies on common performance measures is
responsive to the intent of our recommendation.  In this regard, in
response to OMB officials' comments about the feasibility of the
Director of OMB requiring credit agencies to include in their fiscal
year 1999 annual performance plans the discussion about common
measures that our draft report recommended, as mentioned earlier, we
removed the reference to those agencies including such a discussion
in their fiscal year 1999 plans.  However, as also mentioned earlier,
we retained the recommendation for those agencies to include such a
discussion in their fiscal year 2000 annual performance plans and
subsequent performance plans.  Such discussions can serve as vehicles
for highlighting many of the other cautionary notes that the OMB
officials raised, such as the difficulties in developing measures
that seek to isolate a credit program's unique contributions to a
particular result. 


--------------------
\29 Managing for Results:  Agencies' Annual Performance Plans Can
Help Address Strategic Planning Challenges (GAO/GGD-98-44, Jan.  30,
1998). 


---------------------------------------------------------- Letter :9.1

We are sending copies of this report to the Majority Leader, House of
Representatives; the Chairman and Ranking Minority Member, Committee
on Government Reform and Oversight, House of Representatives; the
Chairman and Ranking Minority Member, Committee on Governmental
Affairs, United States Senate; the Secretaries of Education and
Agriculture; the Acting Secretary of VA; and the heads of agencies
that administer credit programs and are represented on the Federal
Credit Policy Working Group.  We also will make copies available to
others on request. 

The major contributors to this report are listed in appendix IV. 
Please contact me on (202) 512-8676 if you have any questions. 

Sincerely yours,

J.  Christopher Mihm
Associate Director, Federal Management
 and Workforce Issues


OVERVIEW OF THE RESULTS ACT
=========================================================== Appendix I

The Results Act is the primary legislative framework through which
agencies will be required to set strategic goals, measure
performance, and report on the degree to which goals were met.  It
starts by requiring each federal agency to develop a strategic plan
that covers a period of at least 5 years and includes the agency's
mission statement; identifies the agency's long-term strategic goals;
and describes how the agency intends to achieve those goals through
its activities and through its human, capital, information, and other
resources.  The first strategic plans that the Act required agencies
to develop were to be completed by September 30, 1997. 

Also, the act requires each agency to submit to the Office of
Management and Budget (OMB), beginning for fiscal year 1999, an
annual performance plan.  The first annual performance plans were to
be submitted to OMB in the fall of 1997.  The annual performance plan
is to provide the direct linkage between the strategic goals outlined
in the agency's strategic plan and what managers and employees do day
to day.  In essence, this plan is to contain the annual performance
goals the agency will use to gauge its progress toward accomplishing
its strategic goals and identify the performance measures the agency
will use to assess its progress.  Also, OMB will use individual
agencies' performance plans to develop an overall federal government
performance plan that OMB is to submit annually to Congress with the
president's budget, beginning for fiscal year 1999. 

The Results Act also requires that each agency submit an annual
report to the president and to the appropriate authorization and
appropriations committees of Congress on program performance for the
previous fiscal year (copies are to be provided to other
congressional committees and to the public upon request).  The first
of these reports, on program performance for fiscal year 1999, is due
by March 31, 2000; and subsequent reports are due by March 31 for the
years that follow.  However, for fiscal years 2000 and 2001,
agencies' reports are to include performance data beginning with
fiscal year 1999.  For each subsequent year, agencies are to include
performance data for the year covered by the report and 3 prior
years. 

Finally, in crafting the Results Act, Congress recognized that
managerial accountability for results is linked to managers having
sufficient flexibility, discretion, and authority to accomplish
desired results.  The Act authorizes agencies to apply for managerial
flexibility waivers in their annual performance plans beginning with
fiscal year 1999.  The authority of agencies to request waivers of
administrative procedural requirements and controls is intended to
provide federal managers with more flexibility to structure agency
systems to better support program goals.  The nonstatutory
requirements that OMB can waive under the Results Act generally
involve the allocation and use of resources, such as restrictions on
shifting funds among items within a budget account.  Agencies must
report in their annual performance reports on the use and
effectiveness of any managerial flexibility waivers that they
receive. 


OBJECTIVES, SCOPE, AND METHODOLOGY
========================================================== Appendix II

Our first objective was to identify goals and measures established by
the selected credit programs that related to the programs' intended
purposes and determine whether the programs had set target levels of
performance for assessing their progress in achieving their desired
results.  Under the Results Act, target levels of performance are to
enable a comparison of planned versus actual results achieved for a
given year.  Our second objective was to identify the challenges
agency officials cited in developing performance information,
including goals and measures, for the selected programs and any
approaches those programs were taking to address those challenges. 
Our third objective was to describe the status of the Working Group's
effort to develop common performance measures for federal credit
programs. 

For our review, we selected a nonrandom, purposive sample of five
federal credit programs at three agencies.  These programs were the
Department of Veterans Affairs' Loan Guaranty Program; the Department
of Education's William D.  Ford Direct Loan Program and its Federal
Family Education Loan Program (referred to in this report as
Education's direct and guaranteed student loan programs,
respectively); and the Department of Agriculture's Single-Family
Housing (SFH) direct loan program and guaranteed loan program.  We
selected for our review credit programs that varied in terms of type
of program (e.g., housing and education loans); mode of credit
delivery (e.g., direct and guaranteed loans); and program size as
measured by the amount of outstanding loans.  According to data
reported by OMB in the fiscal year 1998 budget and agency data, of
the total amount of federal credit outstanding in fiscal year 1996
for guaranteed loans ($805 billion) and direct loans ($165 billion),
these five programs represented about 32 and 18 percent,
respectively.  The smallest and largest of the guaranteed loan
programs accounted for slightly less than 1/2 percent and 19 percent,
respectively, of the fiscal year 1996 credit outstanding in loan
guarantees governmentwide.  Similarly, the smallest and largest of
the direct loan programs held 7 percent and 11 percent, respectively,
of the fiscal year 1996 credit outstanding in direct loans
governmentwide.  Because of the small and nonrandom nature of our
sample, our observations and analyses are not generalizable to other
federal credit programs. 

To address the first part of our first objective (i.e., to identify
goals and measures established by the selected credit programs that
related to the programs' intended purposes), we compared goals and
measures established by the programs as of May 1, 1997, to the
programs' intended purposes as identified by those programs or their
respective agencies.  According to agency officials, these goals and
measures were generally the same ones the programs submitted to OMB
and Congress with their respective agencies' fiscal year 1998 budget
presentations.  The programs also were proposing to include these
goals and measures for their agencies' fiscal year 1999 annual
performance plans under the Results Act. 

To determine whether a goal or measure was related to a program's
intended purposes, we reviewed available agency and credit program
documentation for (1) a description of the program's intended
purposes and (2) a discussion that reasonably related that particular
goal or measure to the program's intended purposes.  When agency
documentation did not contain such a discussion, we examined the
wording and considered the meaning of each program's goals and
measures and compared them to the program's intended purposes to
identify relationships between them.  We also interviewed agency
officials about the programs' intended purposes and asked those
officials to comment on the relationships we identified between the
programs goals and measures and intended purposes. 

To address the second part of our first objective (i.e., to determine
whether the selected programs set fiscal year 1998 target levels of
performance for assessing their progress in achieving desired
results), we identified those measures for which the programs had
either (1) identified fiscal year 1998 targets or (2) reported prior
year baseline data on those measures and indicated how performance on
those measures was to change (i.e., increase or decrease) in fiscal
year 1998 relative to the baseline.  Our review did not address the
reasonableness of the processes or methods the programs used to
determine how to assess progress or establish target levels toward
achieving the programs' intended purposes; determine whether other,
more appropriate, measures existed; or evaluate the feasibility of
the targets the programs established. 

To address the first part of our second objective (i.e., to identify
the challenges agency officials cited facing in developing
performance information for the selected programs), we developed a
data collection instrument that listed 49 activities that we
identified as being associated with developing performance
information.  Examples of these activities included "determining a
realistic target level of performance for annual performance goals"
and "developing measures for assessing the net effect of the program
compared with what would have occurred in the absence of the
program." To identify these activities, we referred to key steps and
practices identified in our Executive Guide:  Effectively
Implementing the Government Performance and Results Act and other
work that we had under way assessing the challenges agencies were
facing in implementing performance measurement.\1

We sent the instrument to agency officials responsible for and
involved in the development of goals and measures for the selected
programs and asked those officials to rate, using a five-point scale
from "little or no" challenge to a "very great" challenge, how great
a challenge each of the 49 activities was to perform for those
programs.  Officials also could indicate that they had not engaged in
a particular activity.  For purposes of this report, we refer to
activities that any of the agency officials rated as a "great" or
"very great" challenge as significantly challenging activities.  We
then interviewed those officials to discuss why they rated certain
activities as significantly challenging.  We analyzed their responses
and related documentation to identify general challenges that led
officials to report those activities as significantly challenging for
the selected programs. 

To address the second part of our second objective (i.e., to identify
any approaches that the selected programs were taking to address the
challenges they identified to developing performance information), we
interviewed the agency officials who responded to our data collection
instrument to obtain information on the approaches they were taking
to address those activities they had identified as significantly
challenging and analyzed agency documentation.  We also asked these
officials to comment on the way in which we described the approaches
the programs were taking to address the challenges in developing
performance information that they had identified.  We also considered
prior and ongoing work we have done on the efforts of VA, Education,
USDA, and other credit agencies to implement various credit and
financial management reforms. 

To address our third objective (i.e., to describe the status of the
Working Group's effort to develop common performance measures for
federal credit programs), we reviewed various documentation from OMB
and members of the Working Group that described its effort to develop
common performance measures for credit programs and the views of
agency officials on those measures.  We also talked to OMB officials
and members of the Working Group, including agency officials at those
agencies administering the programs we selected for our review, to
obtain their views on the common performance measures proposed.  Our
review did not address the reasonableness of the processes or methods
the Working Group's task force used in determining how to assess
progress for federal credit programs or determine whether other
measures existed that may be more appropriate. 


--------------------
\1 GAO/GGD-96-118, June 1996; and GAO/HEHS/GGD-97-138, May 30, 1997. 


SELECTED CREDIT PROGRAMS' INTENDED
PURPOSES AND EXAMPLES OF GOALS AND
MEASURES THEY PRESENTED IN THEIR
FISCAL YEAR 1998 BUDGETS
========================================================= Appendix III

                                                                                Examples related to the program's
                                                                                         intended purpose
                                                                                ----------------------------------
                                                            Link between the
                                Intended purpose(s) as      intended
                                identified by the selected  purpose(s) and
                                program and its respective  goal and measure                      Performance
                Type of loans   agency                      in example          General goal(s)   measure(s)        Explanation of relationship
Program         provided        --------------------------  ------------------  ----------------  ----------------  ---------------------------------
VA's Loan       Guaranteed      Help veterans and active    Help veterans       Assist veterans   Percent of        The general goal defines, in
Guaranty        home loans      duty personnel purchase     obtain loans        in obtaining      respondents to    part, the intended purpose of
Program                         and retain homes in         to purchase homes.  home mortgage     the Lender        helping veterans purchase homes.
                                recognition of their                            loans.            Customer          VA identified this goal and
                                service to the nation.                                            Satisfaction      measure as related on the basis
                                                                                                  Survey who say    of the following rationale:
                                                                                                  they are          Lender satisfaction addresses an
                                                                                                  satisfied with    intermediate result expected to
                                                                                                  their overall     lead to achieving this goal and
                                                                                                  interaction with  intended purpose. Specifically,
                                                                                                  VA.               maximizing lenders' satisfaction
                                                                                                                    with their dealings with VA
                                                                                                                    employees is expected to
                                                                                                                    encourage lenders to participate
                                                                                                                    in the program, which is expected
                                                                                                                    to expand financing opportunities
                                                                                                                    (i.e., available mortgage loans)
                                                                                                                    for helping veterans purchase
                                                                                                                    homes.


                                                            Help veterans       Assist veterans   Foreclosure       The general goal defines this
                                                            retain their        in avoiding       Avoidance         intended purpose, and VA
                                                            homes.              foreclosures.     Through           identified this goal and measure
                                                                                                  Servicing (FATS)  as related. The FATS Ratio is to
                                                                                                  Ratio.            provide data on the extent to
                                                                                                                    which foreclosures would have
                                                                                                                    been greater had VA not pursued
                                                                                                                    alternatives to foreclosure, such
                                                                                                                    as intervening with the holder of
                                                                                                                    the loan on behalf of the
                                                                                                                    borrower to set up a repayment
                                                                                                                    plan.


                                Treat all veterans and      Treat all veterans  Assist veterans   Percent of        VA identified this goal and
                                other participants in the   in a courteous and  in obtaining      respondents to    measure as related. This measure
                                program in a courteous,     responsive manner.  home mortgage     the Veteran       defines, in part, "assisting"
                                responsive, and timely                          loans.            Customer          veterans in obtaining homes
                                manner.                                                           Satisfaction      because it addresses, in summary
                                                                                                  Survey who say    form, treating veterans in a
                                                                                                  they are          courteous and responsive manner.
                                                                                                  satisfied with
                                                                                                  their overall
                                                                                                  contact with
                                                                                                  VA.


                                                            Treat all veterans  Assist veterans   Percent of        VA identified this goal and
                                                            in a timely         in obtaining      respondents to    measure as related. This measure
                                                            manner.             home mortgage     the Veteran       defines, in part, "assisting"
                                                                                loans.            Customer          veterans in obtaining homes
                                                                                                  Satisfaction      because it addresses treating
                                                                                                  Survey who say    veterans in a timely manner.
                                                                                                  they are          Before a lender issues a VA
                                                                                                  satisfied with    guaranteed mortgage loan, VA must
                                                                                                  the time it took  certify to the lender that the
                                                                                                  to obtain a       borrower is a veteran who is
                                                                                                  Certificate of    eligible for a loan guaranteed by
                                                                                                  Eligibility.      VA.

VA's Loan       Guaranteed      Operate in the most         Operate in the      Efficient credit  Percent of early  The goal describes the strategy
Guaranty        home loans      efficient manner possible   most efficient      and program       defaults of all   and addresses intermediate
Program                         to minimize costs and       manner possible to  management.       loans             results expected to lead to
(continued)                     ensure the best use of the  minimize costs and                    originated.       minimizing costs and ensuring the
                                taxpayer's dollar.          ensure the best                                         best use of the the taxpayer's
                                                            use of the                                              dollar. On the basis of the
                                                            taxpayer's dollar.                                      following rationale, the measure,
                                                                                                                    which VA identified as related to
                                                                                                                    this goal, addresses this
                                                                                                                    intended result: Once a loan is
                                                                                                                    put into default status,
                                                                                                                    collection activities are
                                                                                                                    initiated, which can ultimately
                                                                                                                    include the foreclosure of the
                                                                                                                    borrower's home. Such activities
                                                                                                                    are costly to the government and,
                                                                                                                    thus, the taxpayer. Early
                                                                                                                    defaults (i.e., defaults within 6
                                                                                                                    months of origination) are more
                                                                                                                    likely than a later default to be
                                                                                                                    due to a deficiency in the
                                                                                                                    underwriting of the loans.\a
                                                                                                                    Thus, efficient credit and
                                                                                                                    program management is expected to
                                                                                                                    reduce early defaults and,
                                                                                                                    therefore, minimize costs and
                                                                                                                    ensure the best use of the
                                                                                                                    taxpayer's dollar.


USDA's SFH      Direct home     Provide homeownership       Provide             No general goal   Number of rural   This measure includes the number
direct          loans           loans to:                   homeownership       for this          families with     of direct loans provided, or
loan program                    --very-low-income and low-  loans to eligible   intended purpose  improved or more  issued, to eligible borrowers.
                                income families (i.e.,      rural borrowers.    was identified    suitable housing  Thus, the rationale for the
                                families who have incomes                       by the program    conditions.       relationship is that the program
                                under 80% of median) who                        or agency.\b                        provides loans, or "housing
                                do not own adequate                                                                 opportunities," for improved or
                                housing and cannot obtain                                                           more suitable housing to very-
                                credit from other                                                                   low-to low-income families and
                                sources;                                                                            farm owners who do not own
                                --eligible farm owners for                                                          adequate housing and who cannot
                                housing for themselves or                                                           obtain credit from other
                                for farm laborers.                                                                  sources.


                                Provide "supervised         Provide             No general goal   Percentage of     The rationale for the
                                credit" to many rural       "supervised         for this          borrowers         relationship between this measure
                                borrowers to help them      credit"             intended purpose  current.          and helping rural borrowers
                                maintain their homes in     to help them        was identified                      maintain their homes follows:
                                times of financial crises   (i.e., rural        by the program                      Rural borrowers encountering
                                through credit counseling,  borrowers)          or agency.\b                        financial crises are likely to
                                workout agreements, and     maintain their                                          miss scheduled payments and thus
                                moratoriums.                homes.                                                  are not likely to be counted as
                                                                                                                    "current" in making scheduled
                                                                                                                    loan payments. When a borrower is
                                                                                                                    viewed as not current, the
                                                                                                                    borrower's loan is delinquent and
                                                                                                                    put into default status. Once a
                                                                                                                    loan is put into default status,
                                                                                                                    collection activities are
                                                                                                                    initiated, which can ultimately
                                                                                                                    include the foreclosure of the
                                                                                                                    borrower's home.


USDA's SFH      Guaranteed      Provide homeownership       Utilize private     No general goal   Number of rural   This measure includes the number
guaranteed      home loans      opportunities to moderate-  lenders to provide  for this          families with     of guaranteed loans provided to
loan program                    income rural residents      mortgages to        intended purpose  improved or more  eligible borrowers. Thus, the
                                (i.e., between 80% and      eligible, rural     was identified    suitable housing  rationale for this relationship
                                115% of median).            resident borrowers  by the program    conditions.       is that the program utilizes
                                                            and thus provide    or agency.\b                        private lenders to provide
                                Utilize private lenders to  homeownership                                           mortgages, or homeownership
                                provide mortgages to        opportunities to                                        opportunities, for improved or
                                borrowers who would be      eligible rural                                          more suitable housing to
                                unable to obtain credit     residents.                                              moderate-income rural residents
                                without the guarantee.                                                              who would be unable to obtain
                                                                                                                    credit without the guarantee.


Education's     Direct          Provide students and their  Provide students    Maintain a high   Rate of           The rationale for the
direct student  postsecondary   families with federally     and their families  level of          borrowers'        relationship between this goal
loan program    education       sponsored loans--using a    with federally      borrower          overall           and measure and helping borrowers
                loans           streamlined student loan    sponsored loans to  satisfaction.     satisfaction      meet increasing postsecondary
                                system that simplifies      help borrowers                        with the program  education costs is that:
                                loan access and allows for  meet increasing                       during the first
                                flexible repayment--to      postsecondary                         year.\c           --providing loans using a
                                help borrowers meet         education costs.                                        streamlined system will simplify
                                increasing postsecondary                                                            borrowers' access to
                                education costs and to                                                              postsecondary education loans,
                                reduce taxpayer costs.                                                              thereby helping to satisfy those
                                                                                                                    borrowers in simplifying their
                                                                                                                    ability to meet increasing
                                                                                                                    postsecondary education costs;
                                                                                                                    and

                                                                                                                    --using a streamlined system that
                                                                                                                    allows for flexible repayment
                                                                                                                    will help ease borrowers' debt
                                                                                                                    burden, thereby also helping to
                                                                                                                    satisfy those borrowers in
                                                                                                                    helping them meet increasing
                                                                                                                    postsecondary education costs.


                                                            Help eligible       Provide flexible  Cohort default    The general goal defines the
                                                            borrowers meet      repayment         rate.             program's strategy and intended
                                                            increasing          options so that                     purpose, and Education identified
                                                            postsecondary       debt burden is                      this goal and measure as related.
                                                            education costs     eased and                           The rationale for the
                                                            and reduce          defaults are                        relationship between the measure
                                                            taxpayer costs.     minimized.                          and intended purpose is that
                                                                                                                    borrowers whose loans are placed
                                                                                                                    into a default status are likely
                                                                                                                    encountering debt burden and thus
                                                                                                                    having difficulty meeting
                                                                                                                    postsecondary costs. Once a loan
                                                                                                                    is put into default status, debt
                                                                                                                    collection activities are
                                                                                                                    initiated, which are costly to
                                                                                                                    the government and thus the
                                                                                                                    taxpayer. In addition, by using a
                                                                                                                    streamlined system that allows
                                                                                                                    for flexible repayment, the
                                                                                                                    program will help ease borrowers'
                                                                                                                    debt burden, thereby helping
                                                                                                                    those borrowers meet increasing
                                                                                                                    postsecondary education costs.


Education's     Direct          Ensure access to capital    Ensure access to    Continue to       Institutional     This goal describes the strategy
direct student  postsecondary   for postsecondary           capital for         provide a         direct loan       for achieving this intended
loan program    education       education.                  postsecondary       streamlined loan  program           purpose. In addition, this goal
(continued)     loans                                       education.          delivery system   participation     and measure describe an
                                                                                to attract        rate.             intermediate result the program
                                                                                schools to                          expects to lead to achieving this
                                                                                participate.                        intended purpose. Specifically,
                                                                                                                    the program uses schools as the
                                                                                                                    vehicles for providing loans to
                                                                                                                    students and their families. By
                                                                                                                    providing a streamlined loan
                                                                                                                    delivery system, the program
                                                                                                                    expects to increase schools'
                                                                                                                    participation, which is an
                                                                                                                    intermediate result the program
                                                                                                  Rate of overall   expects will lead to broader
                                                                                Maintain a high   [school]          student access to capital for
                                                                                level of school   satisfaction      postsecondary education.
                                                                                satisfaction.     with the direct
                                                                                                  student loan      This goal and these measures
                                                                                                  program.          describe intermediate results the
                                                                                                                    program also expects to lead it
                                                                                                  Institutional     to achieving this intended
                                                                                                  retention         purpose. As previously noted, the
                                                                                                  rate.             program uses schools as the
                                                                                                                    vehicles for providing loans to
                                                                                                                    students and their families.
                                                                                                                    Increasing participating schools'
                                                                                                                    satisfaction is an intermediate
                                                                                                                    result that the program expects
                                                                                                                    will lead to encouraging those
                                                                                                                    schools to stay as participants
                                                                                                                    in the program; and increasing
                                                                                                                    schools' retention in the program
                                                                                                                    is an intermedate result that the
                                                                                                                    program expects will lead to
                                                                                                                    broader student access to capital
                                                                                                                    for postsecondary education.

                                Successfully implement and  Successfully        Continue to       Number of         The goal defines the intended
                                manage the direct student   implement and       provide strong    internal control  purpose, and Education identified
                                loan program.               manage the direct   fiscal            program           this goal and measure as
                                                            student loan        management of     weaknesses        related.
                                                            program.            the program.      identified in
                                                                                                  Education's
                                                                                                  financial
                                                                                                  statement audit.

Education's     Guaranteed      Provide students and their  Provide students    Maintain a high   Overall rate of   The rationale for the
guaranteed      postsecondary   families with federally     and their families  level of          borrower          relationship between this goal
student loan    education       sponsored loans to help     with federally      borrower          satisfaction      and measure and the intended
program         loans           meet increasing             sponsored loans to  satisfaction      with the          purpose is that by providing
                                postsecondary education     help                from the time of  guaranteed        students and their families with
                                costs.                      them meet           loan origination  student loan      loans (through the guaranteeing
                                                            increasing          through the end   program.          of loans made by private
                                                            postsecondary       of the repayment                    lenders), the program will be
                                                            education costs.    period.                             helping to satisfy those
                                                                                                                    borrowers in their ability to
                                                                                                                    meet increasing postsecondary
                                                                                                                    education costs.


                                Ensure access to capital    Ensure access to    Ensure access to  Number of         The goal defines the intended
                                for postsecondary           capital for         guaranteed loans  borrower          purpose, and Education identified
                                education.                  postsecondary       in a changing     complaints.       the goal and measure as related.
                                                            education.          marketplace.                        The number of borrower complaints
                                                                                                                    is to include data on the number
                                                                                                                    of borrowers who complain to
                                                                                                                    Education about being denied
                                                                                                                    access to a guaranteed
                                                                                                                    postsecondary education loan.


Education's     Guaranteed      Successfully deliver and    Successfully        Provide a         Annual            The goal defines the intended
guaranteed      postsecondary   manage the guaranteed       deliver and manage  program that      delinquency       purpose, and Education identified
student loan    education       student loan program in an  the guaranteed      is cost-          rate.             this goal and these measures as
program         loans           efficient and cost-         student loan        effective for                       related. Monitoring the program's
(continued)                     effective manner to help    program in an       the taxpayer.     Annual            performance in, for example,
                                students and their parents  efficient and                         collection        managing the program's loan
                                meet postsecondary          cost-effective                        rate.             portfolio debt and the program's
                                education costs.            manner.                                                 administrative costs addresses
                                                                                                  Per unit          providing a program that is cost-
                                                                                                  administrative    effective for the taxpayer, or
                                                                                                  costs.            successfully delivering and
                                                                                                                    managing the program in an
                                                                                                                    efficient and cost-effective
                                                                                                                    manner.

-----------------------------------------------------------------------------------------------------------------------------------------------------
\a Underwriting involves a detailed credit analysis, based on credit
information furnished by the borrower, such as employment history,
salary, and financial statements, before a loan is granted. 

\b In commenting on a draft of this report, a USDA Rural Development
official said that in the agency's September 30, 1997, strategic
plan, Rural Development established a general goal for the SFH direct
and guaranteed loan programs that related to the programs' intended
purposes and examples of performance measures.  Specifically,
according to this official, the general goal established was "Rural
Development will improve the quality of life of rural residents by
providing access to technical assistance, capital and credit for
quality housing, and modern, essential community facilities." He
explained that the number of rural families with improved or more
suitable housing conditions performance measure is related to that
part of the goal of "improving the quality of life of rural residents
by providing access to capital and credit for quality housing"; and
the percentage of borrowers current performance measure is related to
that part of the goal of "improving the quality of life of rural
residents by providing access to technical assistance."

\c In commenting on a draft of this report, the Education liaison to
GAO told us that through the agency's Direct Loan evaluation, the
agency is annually measuring borrower satisfaction (i.e., in-school
borrowers, out-of-school borrowers, and parent borrowers) with both
the direct and guaranteed student loan programs.  Therefore,
according to this official, the part of the measure, "rate of
borrowers' overall satisfaction with the [direct student loan]
program during the first year," that refers to monitoring that
satisfaction "during the first year" should be dropped. 

Source:  Selected credit programs and their respective agencies. 


MAJOR CONTRIBUTORS
========================================================== Appendix IV

GENERAL GOVERNMENT DIVISION

Al Stapleton, Assistant Director, Federal Management and Workforce
 Issues, (202) 512-3418
Victoria Miller O'Dea, Evaluator-in-Charge, (202) 512-6397
Thomas M.  Beall, Senior Social Science Analyst
Kiki Theodoropoulos, Senior Evaluator and Communications Analyst

ACKNOWLEDGEMENTS

In addition to those named above, the following individuals made
notable contributions to this report:  From the Accounting and
Information Management Division:  Jeff Steinhoff, Director of
Planning and Reporting; Dan Blair, Mary Ellen Chervenic, Michael J. 
Curro, Julie S.  Tessauro, and McCoy Williams, Assistant Directors;
and Rita A.  Grieco and Carolyn Litsinger, Senior Evaluators.  From
the General Government Division:  Joseph S.  Wholey, Senior Advisor
for Evaluation Methodology; and Stephanie Shipman, Assistant
Director.  From the Health, Education and Human Services Division: 
Joseph J.  Eglin, Jr., Assistant Director; Paula N.  Denman, Senior
Evaluator; and Sara E.  Edmondson, Senior Social Science Analyst. 
From the Office of General Counsel:  Alan Belkin, Assistant General
Counsel; and James M.  Rebbe, Attorney.  From the Resources,
Cover
================================================================ COVER

Community, and

Related GAO Products

Managing for Results:  Agencies' Annual Performance Plans Can Help
Address Strategic Planning Challenges (GAO/GGD-98-44, Jan.  30,
1998). 

Managing For Results:  Using the Results Act to Address Mission
Fragmentation and Program Overlap (GAO/AIMD-97-146, Aug.  29, 1997). 

The Government Performance and Results Act:  1997 Governmentwide
Implementation Will Be Uneven (GAO/GGD-97-109, June 2, 1997). 

Debt Collection:  Improved Reporting Needed on Billions of Dollars in
Delinquent Debt and Agency Collection Performance (GAO/AIMD-97-48,
June 2, 1997). 

Managing For Results:  Analytic Challenges in Measuring Performance
(GAO/HEHS/GGD-97-138, May 30, 1997). 

Executive Guide:  Effectively Implementing the Government Performance
and Results Act (GAO/GGD-96-118, June 1996). 

GPRA Performance Reports (GAO/GGD-96-66R, Feb.  14, 1996). 

Financial Management:  Continued Momentum Essential to Achieve CFO
Act Goals (GAO/T-AIMD-96-10, Dec.  14, 1995). 

Executive Guide:  Improving Mission Performance Through Strategic
Information Management and Technology--Learning From Leading
Organizations (GAO/AIMD-94-115, May 1994). 

Credit Reform:  Case-by-Case Assessment Advisable in Evaluating
Coverage and Compliance (GAO/AIMD-94-57, July 28, 1994). 

Federal Credit Programs:  Agencies Had Serious Problems Meeting
Credit Reform Accounting Requirements (GAO/AFMD-93-17, Jan.  6,
1993). 


*** End of document. ***