Credit Reform: Review of OMB's Credit Subsidy Model (Letter Report,
08/29/97, GAO/AIMD-97-145).
Pursuant to a legislative requirement, GAO reviewed the Office of
Management and Budget's (OMB) Credit Management Subsidy Model (CSM),
focusing on whether the CSM: (1) conforms with relevant provisions of
applicable legislation and accounting standards; (2) provides reliable
results; and (3) is maintained and operated under a system of adequate
controls. An additional objective was to identify supplemental audit
steps that auditors should perform to ensure that federal credit
agencies are using the CSM properly. GAO contracted with the independent
public accounting firm of Ernst & Young LLP to evaluate OMB's written
representations (assertions) about the CSM's capabilities and opine on
whether they are fairly stated in all material respects.
GAO noted that: (1) OMB's assertions on the CSM thoroughly explain the
CSM's capabilities, limitations, and user agency responsibilities; (2)
Ernst & Young concluded that OMB's assertions on the CSM are fairly
stated in all material aspects and recommended several steps OMB should
take to improve the reliability of CSM results and controls surrounding
it; (3) based on GAO's review of Ernst & Young's work, GAO generally
concurs with its conclusion and recommendations; (4) the Federal Credit
Reform Act of 1990 and related federal accounting standards define the
cost (subsidy) of a direct loan or loan guarantee as the estimated
long-term cost to the government on a net present value basis at the
time when a loan is disbursed; (5) the operation of the CSM conforms
with this definition in that the model computes a subsidy cost by
calculating the estimated net present value, at the time of loan
disbursement, of agency-generated cash flows over the life of the loan;
(6) OMB's assertions state that because of several limitations in the
CSM's design, the subsidy cost calculated by the CSM may differ from a
theoretically precise result; (7) for all but one of the limitations,
credit agencies and their auditors can take steps to minimize or
eliminate the impact of the limitations on the subsidy cost calculation;
(8) the impact on the subsidy cost calculation of the limitation
involving the use of nonstandard equations for discounting certain
projected cash flows, however, is more difficult to evaluate and cannot
be minimized by credit agencies and their auditors; (9) several
weaknesses were identified relating to controls surrounding the
development, maintenance, and use of the CSM; (10) GAO believes that if
OMB implements a validation, verification, and testing approach (VV&T)
or similar process, improves documentation, and provides guidance to
credit agencies on controlling access to the CSM, the basic control
weaknesses identified by Ernst & Young will be addressed; (11) OMB's
assertions also state that user agencies are responsible for properly
using the CSM; (12) consequently, when obtaining assurance that CSM
subsidy cost calculations are correct, auditors will need to ensure that
agencies are properly using the CSM; and (13) to help auditors obtain
this assurance, GAO identified, with assistance from OMB staff, a series
of supplemental audit procedures for auditors to follow when auditing
federal credit agencies' financial statements and subsidy cost
calculations.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: AIMD-97-145
TITLE: Credit Reform: Review of OMB's Credit Subsidy Model
DATE: 08/29/97
SUBJECT: Direct loans
Government guaranteed loans
Financial statements
Computer modeling
Computer software verification and validation
Federal agency accounting systems
Accounting procedures
Credit
Auditing procedures
Subsidies
IDENTIFIER: OMB Credit Subsidy Model
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO report. Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved. Major **
** divisions and subdivisions of the text, such as Chapters, **
** Sections, and Appendixes, are identified by double and **
** single lines. The numbers on the right end of these lines **
** indicate the position of each of the subsections in the **
** document outline. These numbers do NOT correspond with the **
** page numbers of the printed product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
** A printed copy of this report may be obtained from the GAO **
** Document Distribution Center. For further details, please **
** send an e-mail message to: **
** **
** **
** **
** with the message 'info' in the body. **
******************************************************************
Cover
================================================================ COVER
Report to the Director, Office of Management and Budget
August 1997
CREDIT REFORM - REVIEW OF OMB'S
CREDIT SUBSIDY MODEL
GAO/AIMD-97-145
OMB's Credit Subsidy Model
(913810)
Abbreviations
=============================================================== ABBREV
AICPA - American Institute of Certified Public Accountants
CSM - Credit Subsidy Model
FASAB - Federal Accounting Standards Advisory Board
FCRA - Federal Credit Reform Act of 1990
GAO - General Accounting Office
OMB - Office of Management and Budget
SBA - Small Business Administration
SFFAS - Statement of Federal Financial Accounting Standards
VV&T - Validation, Verification, and Testing
Letter
=============================================================== LETTER
B-276870
August 29, 1997
The Honorable Franklin D. Raines
Director, Office of Management and Budget
Dear Mr. Raines:
Federal agencies use the Office of Management and Budget's (OMB)
Credit Subsidy Model (CSM) to calculate the subsidy cost of direct
loan and loan guarantee programs for budget and financial reporting
purposes. The Government Management Reform Act of 1994, an expansion
of the Chief Financial Officers Act of 1990, requires that all major
agencies, beginning in fiscal year 1996, prepare annual financial
statements and have them audited and that an audited governmentwide
financial statement be produced every year starting with fiscal year
1997. With outstanding direct loan and guaranteed loan balances for
federal credit programs approaching a reported $1 trillion,
accountants and auditors preparing and auditing these financial
statements for federal credit agencies, as well as CSM users, need to
have assurance that the CSM calculates a reliable subsidy cost in
compliance with applicable legislation and accounting standards.
In order to provide such assurance on a governmentwide basis, we
undertook a review of the CSM that preparers and auditors of
financial statements at all federal credit agencies could rely upon.
Specifically, the objectives of our review were to determine whether
the CSM (1) conforms with relevant provisions of applicable
legislation and accounting standards, (2) provides reliable results,
and (3) is maintained and operated under a system of adequate
controls. An additional objective was to identify supplemental audit
steps that auditors should perform to ensure that federal credit
agencies are using the CSM properly.
To assist us in our review, OMB management prepared written
representations (referred to as assertions) about the CSM's
capabilities, including its compliance with applicable laws and
regulations, its reliability, and the nature of relevant controls.
We contracted with the independent public accounting firm of Ernst &
Young LLP to evaluate OMB's assertions and opine on whether they are
fairly stated in all material respects. (OMB's assertions along with
Ernst & Young's report are included in appendix I.)
This letter discusses the highlights of OMB's assertions and our
findings and recommendations. Appendix II includes supplemental
audit steps that we believe financial statement auditors should
perform to ensure proper use of the CSM by federal credit agencies.
RESULTS IN BRIEF
------------------------------------------------------------ Letter :1
OMB's assertions on the CSM thoroughly explain the CSM's
capabilities, limitations, and user agency responsibilities. Ernst &
Young concluded that OMB's assertions are fairly stated in all
material respects and recommended several steps OMB should take to
improve the reliability of CSM results and controls surrounding it.
Based on our review of Ernst & Young's work, we generally concur with
its conclusion and recommendations.
The key findings and recommendations follow.
-- The Federal Credit Reform Act of 1990 (FCRA) and related federal
accounting standards define the cost (subsidy) of a direct loan
or loan guarantee as the estimated long-term cost to the
government on a net present value basis at the time when a loan
is disbursed. The CSM's calculation of subsidies complies with
this definition in that the model computes a subsidy cost by
calculating the estimated net present value, at the time of loan
disbursement, of agency-generated cash flows over the life of
the loan.
-- OMB's assertions state that because of several limitations in
the CSM's design, the subsidy cost calculated by the CSM may
differ from a "theoretically precise" result. Although its
assertions state that it has not found any instance in which
such differences were significant, OMB also notes that the size
of these differences cannot be precisely determined in general
because the relevant factors, such as the applicable discount
rate and the size and timing of future cash flows, will vary
from case to case. For all but one of the limitations, credit
agencies and their auditors can take steps to minimize or
eliminate the impact of the limitations on the subsidy cost
calculation. The impact on the subsidy cost calculation of the
limitation involving the use of nonstandard equations for
discounting certain projected cash flows, however, is more
difficult to evaluate and cannot be minimized by credit agencies
and their auditors. OMB should correct this limitation by
replacing these nonstandard equations with standard discounting
equations.
-- Several weaknesses were identified relating to controls
surrounding the development, maintenance, and use of the CSM.
The CSM was not designed, and is not maintained, in accordance
with the validation, verification, and testing (VV&T) approach
to computer software development. VV&T is a process of review,
analysis, and testing employed throughout a structured system
development lifecycle\1 to ensure the production of quality,
reliable software. Also, documentation provided to CSM users
instructing them on proper installation and use contains several
errors and omissions. Furthermore, agencies visited by Ernst &
Young did not have logical computer access controls to prevent
unauthorized access to or improper modification of the CSM. As
a result, several recommendations were made to improve the
control environment over the CSM. OMB staff agreed with the
need to improve controls and documentation but expressed some
concerns with aspects of Ernst & Young's report addressing
controls over the CSM. We believe that if OMB implements a VV&T
or similar process, improves documentation, and provides
guidance to credit agencies on controlling access to the CSM,
the basic control weaknesses identified by Ernst & Young will be
addressed.
OMB's assertions also state that user agencies are responsible for
properly using the CSM. This includes using proper data, correctly
installing the appropriate version of the CSM, and making a correct
choice from available CSM options to accurately reflect specific
credit program characteristics. Consequently, when obtaining
assurance that CSM subsidy cost calculations are correct, auditors
will need to ensure that agencies are properly using the CSM. To
help auditors obtain this assurance, we identified, with assistance
from credit agencies' inspectors general, OMB's credit reform staff,
and others, a series of supplemental audit procedures for auditors to
follow when auditing federal credit agencies' financial statements
and subsidy cost calculations. (See appendix II.)
--------------------
\1 Federal Information Processing Standards (FIPS) Publication No.
101, Guidelines for Lifecycle Validation, Verification, and Testing
of Computer Software, defines software lifecycle as the period of
time beginning when the software product is conceived and ending when
the resultant software product is no longer available for use. The
software lifecycle is typically broken into phases, such as
requirements, design, programming and testing, installation, and
operations and maintenance.
BACKGROUND
------------------------------------------------------------ Letter :2
The federal government uses direct loans and loan guarantees as tools
to achieve numerous program objectives, such as assistance for
housing, farming, education, small businesses, and foreign
governments. Before the enactment of FCRA, credit programs--like
most other programs--were recorded in budgetary accounts on a cash
basis. This cash basis distorted the timing of when costs would
actually be incurred and, thus, the comparability of credit program
costs with other programs intended to achieve similar purposes, such
as grants. For example, the cash-basis cost of a direct loan in a
fiscal year was equal to the cash-basis cost of a grant. The
long-term cost of a direct loan, however, may be much less than a
grant because of loan repayments. Cash-basis budgetary recording
also suggested a bias in favor of loan guarantees over direct loans.
Loan guarantees appeared to be free because cash-basis recording did
not recognize that some loan guarantees default. Furthermore, direct
loans appeared to be relatively costly because the cash-basis
recording did not recognize that many direct loans are repaid.
FCRA changed the treatment of credit programs beginning with fiscal
year 1992 so that their costs can be compared more accurately with
each other and with the costs of other federal spending. Two key
principles of credit reform are (1) the definition of cost (subsidy)
in terms of the net present value of cash flows over the life of a
loan and (2) the requirement that budget authority to cover the
subsidy cost be provided in advance before new direct loan
obligations are incurred and new loan guarantee commitments are made.
FCRA defines the subsidy cost of direct loans as the present value
over the loan's life of disbursements by the government (loan
disbursements and other payments) minus estimated payments to the
government (repayment of principal, payments of interest, and other
payments) after adjusting for projected defaults, prepayments, fees,
penalties, and other recoveries. It defines the subsidy cost of loan
guarantees as the present value of cash flows from estimated payments
by the government (for defaults and delinquencies, interest rate
subsidies, and other payments) minus estimated payments to the
government (for loan origination and other fees, penalties, and
recoveries). According to FCRA, the net present value is calculated
by discounting the cash flows at the average interest rate on
marketable Treasury securities of similar maturity to the direct or
guaranteed loan when the loans are disbursed.
FCRA gave OMB oversight responsibility to ensure proper
implementation of credit reform, including agency calculation of
subsidy costs. To provide a consistent, common approach to calculate
the present value of credit program costs, OMB developed the CSM, a
computer software program that calculates a subsidy rate based on
agency-generated estimates of cash flows to and from the government.
The CSM also calculates the portions of the subsidy cost attributable
to defaults, interest subsidies, fees, and other subsidy components.
Thus, the CSM is basically a calculator. Agency-generated cash flows
are entered into the CSM by means of an electronic spreadsheet. The
CSM's basic function is to calculate the net present value of these
cash flows by discounting them to the year monies are disbursed and
dividing the amount of subsidy by the present value of the amount of
disbursement to obtain the subsidy percentage. Agency-generated cash
flows are essential for determining subsidy costs. Changing data on
the cash flows, such as the expected rate of defaults, changes the
subsidy calculation. Therefore, the CSM's subsidy calculation is
only as reliable as the data in agency-generated cash flows the CSM
uses.
Although FCRA requires the use of present value to measure the
subsidy costs of direct loans and loan guarantees for budgetary
accounting and reporting, the law does not address financial
statements and associated reporting. However, the Federal Accounting
Standards Advisory Board (FASAB)\2 concluded that significant
benefits would result from integrating budgetary and financial
accounting for federal credit programs. FASAB recommended that since
budgetary resources for direct loan and loan guarantee subsidies are
required to be reported on a net present value basis, financial
reporting of loan activity should be on the same basis. Statement of
Federal Financial Accounting Standards (SFFAS) No. 2, Accounting for
Direct Loans and Loan Guarantees, was issued in 1993 to provide
accounting standards for federal direct loans and loan guarantees
that incorporate FCRA's subsidy calculation requirements. With the
issuance of SFFAS No. 2, subsidy calculations became important not
only for budgetary accounting and reporting purposes but also for
financial reporting purposes.
--------------------
\2 FASAB was established in October 1990 by the Secretary of the
Treasury, the Director of OMB, and the Comptroller General to
consider and recommend accounting principles for the federal
government. If Treasury, OMB, and GAO decide to adopt the
recommended standards, the standards are published by OMB and GAO and
become effective.
SCOPE AND METHODOLOGY
------------------------------------------------------------ Letter :3
To determine whether the CSM complies with applicable laws and
accounting standards, provides reliable results, and is maintained
and operated under a system of adequate controls, we engaged the
independent public accounting firm of Ernst & Young to perform an
attestation in accordance with American Institute of Certified Public
Accountants (AICPA) attestation standards on OMB management's
assertions regarding the CSM's capabilities and limitations. A
complete discussion of Ernst & Young's scope and methodology is
included in its report in appendix I. To ensure that Ernst & Young
complied with contract requirements and applicable auditing
standards, we
-- defined the scope of work to be completed by Ernst & Young;
-- met periodically with Ernst & Young during the course of its
evaluation and attended key meetings with them, including their
initial meeting with OMB staff;
-- reviewed Ernst & Young's work in accordance with generally
accepted government auditing standards;
-- performed a limited analysis of the CSM, its assumptions, and
mechanics in order to better understand the results of Ernst &
Young's work;
-- analyzed the discounting formulas used by the CSM to discount
the cash flows to the time of disbursement; and
-- developed a limited number of test cash flow spreadsheets for
use with the CSM to compare its results with those calculated
manually and to gain an understanding of the proper use of the
CSM.
To identify supplemental audit steps that auditors should perform, we
reviewed the CSM's User's Guide, OMB's assertions, and Ernst &
Young's report. We also received advice and assistance from the
Federal Audit Executive Council, credit agencies' inspectors general,
representatives of the Governmentwide Credit Reform Subgroup, and
OMB's credit reform staff.
Our analysis of the Ernst & Young report and related work was
conducted in Washington, D.C., from April 1997 through June 1997 in
accordance with generally accepted government auditing standards. We
requested comments on a draft of this report from the Director of OMB
or his designated representative. OMB staff responsible for credit
reform suggested some technical clarifications to our report, which
we have incorporated where appropriate.
CSM CALCULATIONS COMPLY WITH
DEFINITION OF CREDIT SUBSIDY
------------------------------------------------------------ Letter :4
FCRA and SFFAS No. 2 contain several requirements about the
budgetary and financial accounting treatment of direct loans and loan
guarantees. However, the primary requirement pertinent to the
calculation of the subsidy is the definition of cost. FCRA and SFFAS
No. 2 define the cost of a direct loan or loan guarantee as the net
present value of estimated future cash flows at the time when the
loan is disbursed. This calculation incorporates cash flows to and
from the government, excluding administrative costs and any
incidental effects on governmental receipts or outlays. The CSM's
calculation of subsidies complies with this definition in that the
CSM computes a subsidy cost by calculating the net present value of
agency-generated cash flows of expected payments to and from the
government by discounting these cash flows to the fiscal year when
they are disbursed. For loans that disburse in more than 1 year, the
CSM allocates the cash flows to each disbursement year and discounts
the associated cash flows to the appropriate year of disbursement.
FCRA and SFFAS No. 2 require that cash flows contain certain
components, such as loan disbursements; repayments of principal;
payments of interest; and other payments, including fees, penalties,
and other recoveries. Spreadsheets that capture these cash flows are
not part of the CSM and responsibility for creating these
spreadsheets lies with CSM users rather than with OMB. However, OMB
designed the CSM to read spreadsheets that contain these components.
CSM RESULTS MAY DIFFER FROM
THEORETICALLY PRECISE NET
PRESENT VALUE CALCULATIONS
------------------------------------------------------------ Letter :5
OMB's assertions state that limitations exist in the CSM resulting
from (1) the complexity of the FCRA requirement to calculate the net
present value with respect to the time of disbursement, (2) efforts
to simplify the CSM while at the same time making it flexible enough
to fit all federal credit programs, (3) inherent limitations of
discounting methods and financial models such as rounding
definitions, and (4) the use of discounting formulas that differ
slightly from standard methods. Because of these limitations, the
subsidy percentage calculated by the CSM may differ from a
"theoretically precise" result. For example, under some government
loan programs, an agency receives principal and interest payments
from borrowers on a daily basis throughout the year. Therefore, a
theoretically precise subsidy calculation would require the daily
discounting of these cash flows to time of disbursement. OMB
believes that the added precision of such daily discounting would be
burdensome and yield little value. Consequently, OMB provides timing
options that approximate the daily discounting of cash flows.
Although neither OMB nor Ernst & Young have identified any instances
where differences between the CSM subsidy cost calculation and the
theoretically precise calculation were significant, the materiality
of these differences cannot be precisely determined in general
because the relevant factors, such as the applicable discount rate
and the size and timing of future cash flows, will vary from case to
case. Except for one of the limitations, however, our assessment is
that CSM users and their auditors can take steps to minimize or
eliminate the impact of the limitations.
Of the several limitations OMB included in its assertions, three
impact the subsidy cost calculations. These are described in detail
in OMB's assertions and Ernst & Young's report. The first limitation
results from the CSM's use of nonstandard discounting equations to
calculate the net present value of cash flows for partial periods,
such as semiannual and quarterly. The CSM adjusts its discounting
equations for partial periods, when timing options other than "simple
annual" are used,\3 by dividing the discount rate by a factor, which
is determined by the timing of cash flows and the periodicity of
discounting. However, such partial period adjustments should be made
exponentially to conform with standard discounting conventions. For
example, the standard adjustment to the discounting equation for the
semiannual discounting of cash flows occurring at the end of each
6-month period is the square root of (1 + rate) while the CSM uses
(1 + rate/2). This results in CSM present values that are slightly
lower than those calculated using standard geometric formulas.\4
Because these equations are embedded in the CSM's source code, users
and their auditors are unable to mitigate this limitation. To
resolve this problem, OMB should revise the computer source code so
that the net present value calculations reflect standard discounting
equations.
The second limitation arises for programs that disburse loans over
several years. FCRA requires that cash flows be discounted to the
time of disbursement. OMB interprets the FCRA "time of disbursement"
for calculation purposes as the "fiscal year of disbursement."
Consequently, in cases where programs disburse over several years,
precisely calculating subsidies requires that agencies prepare cash
flows clearly associated with each disbursement so that these cash
flows can be discounted to the year of disbursement. Because
disbursement year cash flows cannot always be provided due to limited
agency accounting systems and credit program data, the CSM permits
less detailed, aggregated cohort level data to be used as an
approximation.\5 If cohort level data are used, the CSM uses one of
two methods to disaggregate the cash flows into portions that are
attributable to the amounts that are disbursed in each year.
However, the use of cohort level data can introduce distortions that
result from (1) the disaggregation of the cohort level data and (2)
the CSM's averaging of discount rates for programs where discount
rates differ for each disbursement year.\6 Agencies can eliminate the
impact of this limitation by using disbursement year data, when
available, rather than cohort level data.
The third limitation involves rounding. Because of rounding, and
particularly in programs that have disbursements over several years,
the calculated subsidy will be less precise if an inappropriate scale
is used in the cash flow data. If the data are presented in millions
and the actual values are in thousands, a significant amount of data
may be lost when the CSM rounds to three decimal places. This effect
is most pronounced when a large portion of program cash flow items
are very small, since rounding of smaller dollar values increases the
risk that the rounded values will be materially different than the
actual values. For example, if a series of underlying values in
millions of dollars is 0.0054, 0.0054, 0.0054, the CSM will round
each to 0.005--losing 0.0004, or roughly 8 percent in each case,
which may be significant. If these values were expressed in
thousands of dollars (5.400 instead of 0.0054), none of the
underlying values would be lost due to rounding.
--------------------
\3 The "simple annual" timing option assumes that all outflows occur
at the beginning of the year and all inflows occur at the end of the
year.
\4 The effect on present value calculations of using CSM's
nonstandard discounting formulas for partial periods compared with
using standard formulas is illustrated by the following example.
Cash flow payments of $100 are made at the end of each 6-month period
for 5 years. The discount rate is 5 percent. The CSM formula will
calculate a present value of $875.21 for these cash flows, whereas
the standard formula will calculate a present value of $876.59. The
CSM's net present value is $1.38 or 0.16 percent lower than the
standard formula's present value. While this difference appears to
be insignificant, the impact on individual subsidy rate calculations
of the CSM's nonstandard formulas depends on various factors,
including the discount rate and the timing of cash flows, and cannot
be generalized.
\5 OMB defines a cohort as those direct loans or loan guarantees of a
program that are subsidized by an appropriation for a fiscal year
even if disbursements occur in subsequent years or if the loan is
modified.
\6 OMB's assertions in appendix I discuss the CSM's allocation of
cohort level cash flows to disbursement years and the averaging of
discount rates (disbursement-weighted average discount rate) in
paragraphs A.1 through A.3 and C.5(b).
RELIABLE SUBSIDY
CALCULATIONS ALSO REQUIRE
QUALITY CASH FLOW DATA,
PROPER USE OF THE CSM, AND
MANAGEMENT OVERSIGHT
---------------------------------------------------------- Letter :5.1
When assessing the reliability of the CSM's subsidy rate
calculations, we found it useful to remember the important but
limited role that the CSM has in the credit reform process. Reliable
subsidy calculations also require quality cash flow data, clear
guidance from OMB and proper use of the CSM by credit agencies, and
close management oversight by both the credit agency and OMB.
Because the CSM is essentially a calculator that processes estimated
cash flows provided by the credit agency, its subsidy calculation is
only as reliable as the agency-generated cash flow data. In the
audits of credit agencies' financial statements for fiscal year 1995,
significant weaknesses were identified with the quality of cash flow
estimates and supporting data. For example, the Department of
Agriculture, which has the federal government's largest balance of
loans receivable, received a qualified audit opinion on its Rural
Development component financial statements, in part, because of
inadequately supported cash flows. Fiscal year 1996 financial
statement audit results available as of July 1997 indicate that
generally credit agencies are still having difficulty preparing
quality, well-supported cash flows that comply with FCRA and SFFAS
No. 2 requirements. Staff from GAO, OMB, and credit agencies are
currently working together to develop approaches to improve cash flow
estimates.
Although the basic function of the CSM--to discount cash flows to the
year of disbursement--is conceptually straightforward, use of the CSM
can be complex because of the various options available and types of
data to be entered. Consequently, proper use of the CSM requires
sufficient, clear guidance from OMB on what the CSM options are and
how best to use them to reflect the characteristics of credit agency
loan programs. Also, credit agency officials must recognize that use
of the CSM requires not only adequate knowledge of credit agency loan
programs but familiarity with the concepts contained in FCRA and
SFFAS No. 2. Moreover, given the complexity inherent in developing
cash flow spreadsheets and using them with the CSM in subsidy
calculations, agency management must exercise proper oversight to
ensure that cash flow data is of high quality, the CSM is used
properly, and controls surrounding the preparation of cash flows and
the calculation of subsidies are adequate and operating as intended.
Finally, given the role assigned to it by FCRA, OMB must oversee
agencies' credit reform implementation even though responsibility for
preparing cash flows is with the credit agencies.
We recently had the opportunity to illustrate the need for adequate
oversight by credit agencies and OMB. In our July 16, 1997,
testimony before the House Committee on Small Business,\7 we reported
on the estimates of credit subsidy for the Small Business
Administration's (SBA) guaranteed business loan and certified
development company programs--more commonly called the "7(a)" and
"504" programs, respectively. We reported on an error in SBA's cash
flow spreadsheet that we had uncovered in the calculation of the
fiscal year 1997 subsidy costs for the 7(a) program. A critical cell
in SBA's cash flow spreadsheet was based on the number of dollars
guaranteed instead of the number of dollars disbursed, that is, the
total face amount of the loans. (SBA projected that it would
guarantee on average about 76 percent of the fiscal year 1997 loan
cohort.) As a result of this error, SBA's estimated credit subsidy
rate was higher by about 32 percent (1 divided by 0.76, the average
guaranteed portion of loans disbursed by private lenders).
This error went unnoticed by both SBA and OMB staff responsible for
reviewing the 7(a) credit subsidy rate estimate. If those staff had
compared the component data generated by the CSM for the erroneous
fiscal year 1997 estimate with the components of the fiscal year 1996
estimate, they would have seen an unexplainable increase in the fee
revenue component (there was no increase in the fee rates charged).
According to SFFAS No. 2, subsidy estimate component data should be
used to monitor and make decisions about the federal government's
credit programs.
In 1995, the Governmentwide Credit Reform Subgroup was formed to
resolve issues faced by (1) agencies in implementing credit reform
and preparing quality cash flow data and (2) auditors reviewing
credit subsidy estimates. An issue paper prepared by the Subgroup,
Preparing and Auditing Direct Loan and Loan Guarantee Subsidies Under
the Federal Credit Reform Act, is expected to be issued during fiscal
year 1998.
--------------------
\7 Small Business Administration: Credit Subsidy Estimates for the
Sections 7(a) and 504 Business Loan Programs (GAO/T-RCED-97-197, July
16, 1997).
CONTROLS AND DOCUMENTATION
SHOULD BE IMPROVED
------------------------------------------------------------ Letter :6
Ernst & Young's report includes the following control weaknesses
surrounding the development, maintenance, and use of the CSM. First,
the CSM was not designed, and is not maintained, in accordance with
the validation, verification, and testing (VV&T) approach to software
development. VV&T is a process of review, analysis, and testing
employed throughout a structured system development lifecycle to
ensure the production and maintenance of quality, reliable software.
Second, the CSM program was developed and tested by a single
programmer and was not independently tested to ensure that its
functionality met the initial design request. Ernst & Young noted
that the loss or absence of the original programmer may substantially
hinder significant modification of the current program. Third,
documentation provided to CSM users contains several errors and
omissions, and exists in several pieces. Fourth, OMB's storage of
the program source code is insufficient to protect against loss,
destruction, and corruption. Fifth, agencies visited by Ernst &
Young were using the CSM without logical access controls to prevent
unauthorized access. Finally, because it is difficult to verify
which data the CSM used to calculate the subsidy, the CSM printed
output should be enhanced.
Three recommendations were made to improve controls over the CSM, and
OMB credit reform staff generally agreed with them. Specifically,
OMB staff agreed that (1) future revisions to the CSM will be
accompanied by more detailed and complete documentation of the
validation, verification, and testing of software, (2) documentation
will be improved and expanded to correct for errors and omissions,
and (3) the CSM printed output should be enhanced to provide an audit
trail showing which data the CSM used to calculate the subsidy.
However, OMB staff expressed concerns about some of the findings and
one recommendation relating to controls over the CSM. Although OMB
acknowledged in its assertions that it did not have a structured and
documented VV&T process for developing and testing the CSM, OMB staff
told us that the CSM had been developed through extensive discussions
among OMB and agency staffs and had been tested over several years by
CSM users at credit agencies as well as by OMB credit reform staff.
OMB staff also emphasized that computer access controls are an
agency's responsibility and noted that current versions of desktop
operating systems have password protection and other controls.
Moreover, OMB said that the source code is stored on-site and
off-site, in digital tape, fixed disk, and CD-ROM formats and that
these storage media are adequate to prevent loss, destruction, or
corruption. Finally, OMB's position is that the loss of the original
CSM programmer would not seriously affect future modifications of the
program since (1) there is no immediate or urgent need for
modifications to the CSM, so replacement staff would have ample time
to familiarize themselves with the CSM, (2) other OMB staff or
contract personnel could easily make such modifications by using the
existing source code, knowledge of the programming language, and
familiarity with credit reform concepts, and (3) the CSM is more
likely to be replaced than modified.
We believe that the improvements to the control environment
surrounding the CSM agreed to by OMB, especially the use of VV&T or a
similar process, will resolve the major control issues raised by
Ernst & Young. Although we recognize that user agencies have
ultimate responsibility for computer access controls, agencies
clearly need guidance on properly controlling access to the
CSM--Ernst & Young's visits to seven user agencies found that none of
them had logical access controls over the personal computers
containing the CSM. We believe that OMB guidance on proper controls
over access to the official agency copy of the CSM can be easily and
quickly communicated to agency staff. In addition, since the
completion of Ernst & Young's work, we have confirmed that OMB has
adequate storage of the CSM source code to prevent loss, destruction,
or corruption.
REVISED CSM TO BE RELEASED
AFTER JUNE 1998
------------------------------------------------------------ Letter :7
OMB staff told us that they are considering improvements to the CSM,
including a refinement of methods, more detailed output, improved
documentation, and other improvements identified in the management
assertions and, where appropriate, recommendations from the Ernst &
Young report. Also, before releasing this improved version, OMB
staff are considering whether to have an audit of the CSM
calculations. OMB staff told us that the release of the new version
of the CSM will be no earlier than June 1998.
OMB staff also told us that they would recommend an interim release
of the CSM, prior to the major release described above, if there were
a change in law or other requirements or if a significant defect in
the calculations was identified. However, in the OMB staff's
judgment, the relatively minor improvements that they believe could
be accomplished in an interim update must be weighed against what
they believe will be a substantial effort, mainly by agencies, to
reinstall the model on hundreds of computers and train staff in the
changes from the previous release. As of July 1997, OMB staff told
us that they have found no evidence that an interim update is
required. Further, OMB staff noted that OMB's management assertions,
which Ernst & Young concluded are "fairly stated in all material
respects," state that the effect of limitations in the current
release of the CSM, based on cases reviewed to date, "have not
revealed any instance in which such differences were significant."
PROCEDURES AUDITORS SHOULD
PERFORM TO ENSURE PROPER USE OF
THE CREDIT SUBSIDY MODEL
------------------------------------------------------------ Letter :8
OMB's assertions and Ernst & Young's report pointed out that proper
use of the CSM is the responsibility of the user agencies. This
responsibility includes using proper cash flow data, correctly
installing the appropriate CSM version, and making correct choices
from available CSM options to accurately reflect specific credit
program characteristics. In contracting with Ernst & Young, we did
not ask the firm to determine whether agencies are properly using the
CSM. Therefore, to ensure that CSM subsidy calculations are correct,
auditors will need to, among other things, obtain assurance that
agencies are using the CSM properly. With assistance from the
Federal Audit Executive Council, credit agencies' inspectors general,
representatives of the Governmentwide Credit Reform Subgroup, and
OMB's credit reform staff, we identified supplemental audit
procedures to be performed in audits of federal credit agencies and
subsidy calculations. These procedures are listed in appendix II.
CONCLUSIONS
------------------------------------------------------------ Letter :9
Taken together, OMB's assertions on the CSM's capabilities, Ernst &
Young's report, and the audit procedures included in this report
should provide federal credit agencies and their auditors with a
better understanding of how the CSM functions and additional guidance
on proper use of the CSM. Although generally agreeing with Ernst &
Young's recommended steps for improving the CSM, OMB staff believe
that an immediate release of a revised, improved CSM would not be
worth the costs involved. OMB staff further note that they have
found no evidence that the limitations in the current release of the
CSM have had a material impact on subsidy calculations. Thus, they
propose waiting until they have decided upon various policy matters
and other changes to the CSM before they issue a revised version of
the CSM. While this may be reasonable, we believe that the lack of
adequate access controls at user agencies should be corrected
immediately.
RECOMMENDATIONS
----------------------------------------------------------- Letter :10
Based on our review of OMB's assertions and Ernst & Young's report,
we recommend that the Director of OMB ensure that guidance is
provided to user agencies to establish logical access controls
surrounding use of the CSM. In addition, we recommend that the
Director of OMB ensure that the following steps are taken in
developing the next revision to the CSM:
-- revise the discounting equations in the CSM to follow standard
finance theory,
-- strengthen controls over the CSM by implementing a VV&T or
similar process,
-- improve the CSM documentation to correct for the mistakes and
omissions noted in OMB's assertions and Ernst & Young's report,
and
-- enhance the CSM printout with additional data so that users and
auditors are able to specifically identify which data were used
by the CSM in the subsidy calculations.
--------------------------------------------------------- Letter :10.1
Within 60 days of the date of this letter, we would appreciate
receiving a written statement on actions taken to address our
recommendations.
We are sending copies of this report to the Senate and House
Appropriations and Budget Committees, the Senate Committee on
Governmental Affairs, and the House Committee on Government Reform
and Oversight. We are also sending copies to the chief financial
officers and budget officials at federal credit agencies; the
inspectors general with audit responsibilities for these agencies;
and other interested parties. Copies will also be made available to
others upon request.
If you have any questions about this report, please call McCoy
Williams, Assistant Director, at (202) 512-6906. Major contributors
to this report are listed in appendix III.
Sincerely yours,
Linda M. Calbom
Director, Civil Audits
(See figure in printed edition.)Appendix I
ERNST & YOUNG'S REPORT INCLUDING
OMB'S ASSERTIONS
============================================================== Letter
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
AUDIT PROCEDURES TO VERIFY PROPER
USE OF THE CREDIT SUBSIDY MODEL
========================================================== Appendix II
Proper use of OMB's Credit Subsidy Model (CSM) requires that user
agencies correctly install the appropriate CSM version, make correct
choices from available CSM options and commands to accurately reflect
specific credit program characteristics, control access to the CSM,
and understand the CSM's capabilities and limitations.
With assistance from the Federal Audit Executive Council, credit
agencies' inspectors general, representatives of the Governmentwide
Credit Reform Subgroup, and OMB's credit reform staff, we identified
the following audit procedures that should be performed to ensure
proper use of the CSM. Comprehensive guidance on auditing credit
reform subsidy estimates is included in Preparing and Auditing Direct
Loan and Loan Guarantee Subsidies Under the Federal Credit Reform
Act, a draft issue paper prepared by the Governmentwide Credit Reform
Subgroup, which is expected to be issued during fiscal year 1998.
The audit procedures discussed in the following sections should be
used in conjunction with those presented in the issue paper.
Additionally, these procedures are intended to provide audit guidance
that may or may not be applicable in all situations. The auditors
should use professional judgment in determining which are applicable
to the agency they are auditing.
ENSURE USE OF AN APPROPRIATE
AND UNMODIFIED VERSION OF
THE CSM
------------------------------------------------------ Appendix II:0.1
Since 1990, OMB has periodically revised the CSM to add enhancements,
make methodology changes, and otherwise improve its operation.
Different versions of the CSM may produce slightly different subsidy
rates. As of July 1997, the current version of the CSM was Version
r.9, dated August 1, 1994. We expect that OMB will, on occasion,
release new versions of the CSM. In addition, although it may be
unlikely, the agency's computer file of the CSM may become modified
intentionally or accidentally. Therefore, the auditor should obtain
the appropriate version of the CSM for the fiscal year under audit by
contacting the agency's OMB budget examiner.\1 This version should be
compared with the version used by the agency in its subsidy
calculations. To verify that the agency's version of the CSM is
unmodified, the auditor should use the "file compare" feature of
desktop operating software to compare the agency's version with the
OMB official, approved version. If the two versions are the same,
the auditor can conclude that the agency's version is unmodified. If
they differ, the auditor should bring this to the attention of agency
management and the OMB budget examiner and obtain an explanation for
the differences. Finally, as the ultimate check, the auditor can
calculate the subsidy rate using the agency's cash flows, the
agency's version of the CSM, and OMB's version of the CSM and compare
the results.
--------------------
\1 The auditor may also wish to obtain from the OMB budget examiner
information on past errors in agency use of the CSM and a copy of the
most recent or appropriate version of the CSM User's Guide.
VERIFY THAT APPROVED CASH
FLOW DATA IS THE SAME DATA
USED BY THE CSM TO CALCULATE
THE SUBSIDY RATE
------------------------------------------------------ Appendix II:0.2
The user agency should provide the auditor with the approved cash
flow data that support its credit program subsidy rate for each of
the credit programs selected for internal control and substantive
testing. (Cash flow data will be available from electronic
spreadsheet files in a format prescribed by the CSM User's Guide.)
The auditor should verify that these data were, in fact, the same
data used by the CSM to calculate the applicable subsidy rate. The
spreadsheet file name, the range name, and the date and time the
spreadsheet was last changed are included in the printed CSM output.
The auditor can check this information against the named spreadsheet
file provided by the agency to verify the cash flow data used in the
CSM's subsidy calculation. However, if the spreadsheet file provided
by the agency was changed after the subsidy calculation, the date and
time stamp on the spreadsheet file will not match what is on the CSM
output. In this case, the CSM output will not provide sufficient
information to verify the cash flow data used by the CSM. Therefore,
the auditor will need to use other methods. One method is to
recalculate the subsidy rate using the cash flow data provided by the
agency and the auditor's copy of the appropriate version of the CSM
obtained from the applicable agency's OMB budget examiner. If the
recalculated subsidy rate is the same as the subsidy rate under
audit, the auditor should be able to conclude that the cash flow data
provided by the user agency was the same data used by the CSM. If
the recalculated subsidy rate is different, the auditor should bring
this to the attention of agency management and the OMB budget
examiner and obtain an explanation for the difference.
FOLLOW UP ON ERROR MESSAGES
------------------------------------------------------ Appendix II:0.3
Prior to calculating a subsidy rate, the CSM performs several edits
on agency-generated cash flows to help ensure that cash flow data do
not contain obvious errors. If the CSM edit process identifies a
serious error, the CSM will issue an error message and terminate its
operation without calculating a subsidy. However, if the CSM edit
process determines an error to be less serious, it will issue a
"warning" but will not terminate the program. Warnings will be
listed with the subsidy rate calculation on CSM output sent to a
printer. The auditor should review CSM output to identify whether
any warning messages are listed and follow up with agency management
to determine why the situation causing the warning message was not
resolved and whether not eliminating the error could have any impact
on the subsidy rate calculation.
In addition, the CSM provides options for the user to suppress
certain warning messages. For example, when cumulative scheduled
principal payments do not equal disbursements, a warning message is
normally issued. If the agency has suppressed this warning, auditors
should determine whether this suppression is appropriate. This
concern applies to other warning messages as well. Specifically, the
auditor should check the agency's cash flow spreadsheet to determine
whether the "suppress warnings" command was used. If so, the auditor
should request that the agency explain why warning messages were
suppressed and, if certain warning messages are suppressed, whether
conditions exist that would cause those messages to be generated, and
whether the warning indicates a material problem in the cash flows.
ENSURE THAT OPTIONS CHOSEN
PROPERLY REFLECT SPECIFIC
CHARACTERISTICS OF EACH
CREDIT PROGRAM
------------------------------------------------------ Appendix II:0.4
Proper use of the CSM requires that the agencies select the
appropriate options from those available (see Chapter III of the CSM
User's Guide, Version r.9) and use the appropriate Treasury rate to
discount cash flows to net present value. Particular care should be
used in reviewing the choice of timing options for the principal and
interest payments in direct loan programs. When a row of cash flows
for scheduled principal or interest payments is prepared using
standard financial formulas (which assume disbursements at the
beginning of the period and payments at the end of the period), the
"simple annual" option should be used. In contrast, when estimates
of interest and principal payments are based on the assumption that
these payments occur continuously throughout the year, the timing
option row of cash flows should be "continuous." When the wrong
timing option is used for scheduled principal or interest payments,
the financing subsidy may be materially distorted. The auditor may
also want to review the choice of timing options for payments and
receipts other than principal and interest, although the effects of
these distortions are generally smaller.
Care should also be exercised when reviewing cash flows for loan
guarantee programs that guarantee less than 100 percent of the face
value. As indicated in the User's Guide, the amount in the cash flow
row for "disbursement of loans by private lenders" is the total
amount disbursed by the lenders, regardless of how much is guaranteed
by the credit agency. The amount of disbursed loans guaranteed by
the government is included in the row of the cash flow representing
the estimate of claims made against the government. For example, if
an agency has a program that guarantees 75 percent of loans
disbursed, and the lenders disburse $100,000 in loans that
immediately default, the agency should put $100,000 in the
disbursement by private lenders cash flow row and $75,000 in the cash
flow row for defaults.
ENSURE PROPER SCALE HAS BEEN
USED IN CASH FLOW
SPREADSHEETS
------------------------------------------------------ Appendix II:0.5
OMB's assertions state, "The model rounds cash flows to three decimal
places when read from spreadsheet files. Because of the rounding,
and particularly in programs that have disbursements over several
years, the calculated subsidy can change slightly with the scale of
the program. This effect is most pronounced when many of the cash
flow items are very small after rounding (.005 or .011, for
instance). Small values are especially sensitive to the hazards of
rounding." Therefore, agency controls should be in place to ensure
that rounding to three decimal places has no significant effect on
the spreadsheet values and, in turn, the calculated subsidy. For
example, if a series of underlying values, in millions of dollars,
are 0.0054, 0.0054, 0.0054, the CSM will round each to 0.005--losing
0.0004 in each case, which could be significant. In this situation,
the agency should express values in thousands of dollars so that the
underlying values are 5.400, 5.400, 5.400--losing nothing in the
rounding--in order to obtain a more precise subsidy rate calculation.
The auditor should confirm that management controls are adequate to
ensure that the cash flows contain the proper scale and that rounding
has no significant effect on the subsidy calculation. If these
controls are not adequate, the auditors should review the cash flow
spreadsheet to ensure that the scale used is appropriate. The
auditor should also bring the situation to the attention of agency
management.
DETERMINE WHETHER CASH FLOWS
ARE PREPARED AT APPROPRIATE
LEVEL OF DETAIL
------------------------------------------------------ Appendix II:0.6
The CSM permits spreadsheet cash flow data to be prepared on a
disbursement year basis or a cohort basis. (A disbursement year
consists of all loans from a given cohort that are disbursed in a
given fiscal year.) For the special case in which all disbursements
occur during a single fiscal year, the disbursement year includes the
entire cohort and these bases do not differ. However, for loan
programs with cohorts that disburse over more than one year, the
disbursement year includes just part of the cohort. For such
programs, the cash flows for each disbursement year of a given cohort
are necessary to precisely calculate subsidies at the time of
disbursement. Because agencies cannot always provide such detail,
the CSM permits less detailed cohort level data--combinations of 2 or
more disbursement years--to be used as an approximation. But the use
of cohort level data can introduce distortions. For example, a loan
program can be expected to have a zero financing (interest rate)
subsidy if the borrower rate is the same as the discount rate.
However, if a program disburses loans over 2 or more years, cohort
rather than disbursement year cash flows are used, and the discount
rates are not held constant in all disbursement years, the CSM will
calculate a non-zero subsidy.
Therefore, whenever a loan program has substantial disbursements in 2
or more years and the agency has prepared cash flows using cohort
level rather than disbursement year data, the auditor should
determine why disbursement year cash flows were not used.
Specifically, if there are reasons why disbursement year cash flows
cannot be prepared, these reasons should be documented. On the other
hand, if disbursement year cash flows are available, the auditor
should determine whether the use of cohort level cash flows has had a
material effect on the subsidy calculation. A determination that an
effect is material should take into account the size of the
difference in absolute terms and relative to the subsidy, the effect
on the level of loans supported by the subsidy, and other factors the
auditor may consider important. If the auditor determines that the
effect is material, the auditor should recommend that the agency
prepare cash flows on a disbursement year basis to eliminate the
problem. If the agency is unable to do this, the auditor should
exercise professional judgment to determine whether there is a
potential for material misstatement and whether this situation would
affect the ability to conclude on the fairness of the amounts in
related accounts.
COMPARE CASH FLOW
SPREADSHEET AND RELATED
SUBSIDY RATE WITH PRIOR
YEARS
------------------------------------------------------ Appendix II:0.7
Credit reform and the CSM require credit agencies to develop
spreadsheets of projected cash flows, which must be presented in a
prescribed format and require the spreadsheet preparer to choose
among various commands and options that properly characterize each
credit program. Once an auditor has determined that a spreadsheet
contains the proper format, commands, options, etc. for the credit
program, then the auditor can have some assurance about future years'
cash flows with the same formats, commands, options, etc. If changes
in formats or commands on the cash flow spreadsheets have been made,
auditors should discuss with agency officials why such changes were
made, including what the changes are intended to accomplish. An
auditor may wish to use analytical procedures each year to confirm
that any changes to the credit program are properly reflected in the
spreadsheet and that changes to the spreadsheet and associated
subsidy rate, including components, are reasonable. For example, if
an agency's fee structure has not changed, the auditor should expect
the subsidy rate component attributable to fees to remain the same.
EVALUATE AGENCY SECURITY
CONTROLS OVER CSM ACCESS
------------------------------------------------------ Appendix II:0.8
OMB's assertions state that agencies are responsible for ensuring
that the CSM has not been corrupted or otherwise inappropriately
changed. Such assurance requires that agencies have procedures in
place to limit access to the CSM to authorized personnel only.\2 For
example, the auditor might expect to find procedures to ensure
confirm password protection on the desktop workstation where the CSM
resides. The auditor should review these procedures and determine if
they are in place to verify that they adequately protect the CSM from
unauthorized use and corruption.
--------------------
\2 Agency access controls should be implemented for the CSM copy that
is used to produce the agency's official subsidy rate calculations.
MAJOR CONTRIBUTORS TO THIS REPORT
========================================================= Appendix III
ACCOUNTING AND INFORMATION
MANAGEMENT DIVISION, WASHINGTON,
D.C.
McCoy Williams, Assistant Director
Louis Schuster, Evaluator-in-Charge
Amy Lowenstein, Evaluator
Meg Mills, Communications Analyst
OFFICE OF THE CHIEF ECONOMIST
Harold J. Brumm, Jr., Economist
*** End of document. ***