Public Housing: HUD Needs Better Information on Housing Agencies'
Management Performance (Letter Report, 11/09/2000, GAO/GAO-01-94).
In an effort to improve its oversight of housing agencies, the
Department of Housing and Urban Development (HUD) implemented its Public
Housing Assessment System (PHAS). PHAS provides HUD with a system to
measure the performance of its housing agencies. GAO found that the
effectiveness of HUD's ability to measure housing agencies' performance
under PHAS has not been greatly improved for several reasons. First, HUD
does not verify the accuracy of self-certified data submitted by the
agencies in response to PHAS requirements. According to GAO's survey, in
several instances, agencies that certified themselves as standard or
high performers were later found to be troubled. Second, HUD's filed
offices indicated that they are adequately prepared in certain areas to
assist troubled housing agencies but that in other areas, such as
adequate staffing, they are not. Further, HUD's experience indicated
that the use of sanctions against poor performing agencies can be an
effective tool for improving performance, but that most of its field
offices fail to use them.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: GAO-01-94
TITLE: Public Housing: HUD Needs Better Information on Housing
Agencies' Management Performance
DATE: 11/09/2000
SUBJECT: Public housing
Performance measures
Housing programs
Public administration
Federal property management
Quality assurance
Technical assistance
Sanctions
IDENTIFIER: HUD Working Capital Fund
HUD Public Housing Assessment System
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Testimony. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-01-94
Report to the Chairman, Subcommittee on Housing and Community Opportunity,
Committee on Banking and Financial
Services, House of Representatives
November 2000 PUBLIC HOUSING HUD Needs Better Information on Housing
Agencies' Management Performance
GAO- 01- 94
Letter 3 Appendixes Appendix I: Housing Agencies' Management Performance
Survey 28
Appendix II: GAO Contacts and Staff Acknowledgments 52 Figures Figure 1: Six
Components of PHAS Management Operations 8
Figure 2: Effect of Housing Agencies' PHAS Scores 10 Figure 3: PHAS
Management Operations Components Rated as
Moderately, Very, or Extremely Important 12 Figure 4: Factors Field Offices
Judge Most Important to Providing
Effective Assistance to Housing Agencies 16 Figure 5: Limited Field Office
Use of Most Effective Sanctions 20
Abbreviations
HUD Department of Housing and Urban Development PHAS Public Housing
Assessment System
Lett er
November 9, 2000 The Honorable Rick Lazio Chairman, Subcommittee on Housing
and Community Opportunity Committee on Banking and Financial Services House
of Representatives
Dear Mr. Chairman: The Department of Housing and Urban Development (HUD)
spends nearly $7 billion annually on the public housing program, providing
affordable homes to low- income households. HUD depends on and funds more
than
3,000 local public housing agencies to manage the day- to- day operations of
public housing, providing assistance to and oversight of these agencies
through 43 of its field offices. For HUD to be accountable for the
performance of the public housing program, it must first rely on holding the
housing agencies accountable for providing safe and decent housing and
protecting the federal investment in their properties. For HUD to do so, it
has to be able to effectively measure housing agencies' performance and-
using its measurements of the agencies' performance- assist those that need
help and hold accountable those that do not improve. Also, by statute, HUD
must assess the performance of housing agencies to evaluate their actions in
all major areas of management operations and to designate as
“troubled” any agency that fails on a widespread basis to
provide acceptable housing conditions.
HUD primarily measures housing agencies' management performance through the
management operations indicator of its Public Housing Assessment System
(PHAS), which it implemented in June 2000. The PHAS management operations
indicator is largely unchanged from HUD's previous performance measurement
system, the Public Housing Management Assessment Program, which rated
housing agency performance in areas such as completing routine maintenance
requests. Because we reported to you in the past that HUD needed to make its
old
assessment program more accurate and useful, 1 you asked us to report on the
effectiveness of HUD's oversight of the management performance of 1 Public
Housing: HUD Should Improve the Usefulness and Accuracy of Its Management
Assessment Program (GAO/ RCED- 97- 27, Jan. 29, 1997).
housing agencies through PHAS. As agreed with your office, this report
discusses factors that HUD field and headquarters staff indicate most
influence HUD's ability to (1) effectively measure the management
performance of housing agencies, (2) provide assistance to those agencies
that need to improve their performance, and, (3) hold accountable those
agencies that fail to improve their performance. To address these questions,
we surveyed all 43 of HUD's field office directors of public housing because
they have the most direct day- to- day experience in HUD's oversight of and
assistance to public housing agencies. We received responses from 41 of the
43 field offices, which is a 95- percent response rate. Our survey included
questions on the full range of these officials' oversight responsibilities,
such as their experiences with HUD's previous
management assessment program and PHAS (the current system), the adequacy of
current levels of resources and other factors that affect their oversight of
and assistance to housing agencies, and their experiences
using sanctions to hold accountable poorly performing agencies. We did not
independently verify the field offices' responses. Our detailed scope and
methodology are discussed at the end of this report, and the results of our
survey are presented in appendix I. Results in Brief According to our
survey, the management operations indicator of HUD's
Public Housing Assessment System includes the components HUD's field offices
judge most important to gauging the quality of a housing agency's management
(for example, how many of a housing agencies' major
systems- such as heating and air conditioning- it inspects each year).
However, HUD lacks adequate assurance that the data that housing agencies
self- certify on these components are accurate. Over 40 percent of HUD's
field offices reported that, in the past, housing agencies certified to
inaccurate or incomplete data to a moderate or greater extent because, for
example, the agencies had misinterpreted HUD program requirements,
definitions, or rules. Under the new Public Housing Assessment System, HUD
headquarters and its field offices perform initial quality assurance reviews
of the data the agencies certify, but neither involves first- hand
verification of the data. To identify when housing agencies submit
inaccurate management operations data, HUD primarily relies on independent
audits conducted at most housing agencies each year to
substantiate that data. However, HUD headquarters officials told us that
their initial quality control reviews of these audits have found that some
have not included the work HUD expects of them to substantiate management
operations data. According to our survey, in 21 instances,
agencies that self- certified to being standard or high performers were
later
found to be troubled; field offices reported that 15 of the agencies' audits
did not identify problems with the agencies' self- certified data. Under
HUD's Public Housing Assessment System, failing to identify inaccurate
housing agency data could be costly to other housing agencies because HUD
plans to award performance bonuses to high- performing housing agencies- on
the basis, in part, of their self- certified data- and will pay for the
bonuses with funds it had set aside for other agencies. As a result, we are
making recommendations to HUD to improve its assurances that management
operations data are accurate and ensure that it uses
adequately verified information on housing agencies' performance for
purposes such as awarding performance bonuses. HUD's field offices
identified nine factors in our survey, such as training funds for the field
office staff, that they judge to be most important to providing effective
assistance to housing agencies needing to improve their
performance. Of these nine factors, the field offices reported that for
four- their expertise in HUD programs, expertise in rental property
management, working knowledge of housing agencies, and travel funds- they
are adequately prepared to assist housing agencies. However, they also
reported that for five of these factors- the number of field office staff,
their workload relative to the number of staff, their workload relative to
the field office staff qualifications and training, the amount of field
office training funds, and, the amount of technical assistance funds
available to them- they believe they are currently not adequately prepared
to assist housing agencies. The HUD headquarters official in charge of field
office operations acknowledged that its field offices need additional
training, in part because of substantial turnover in recent years in the
field offices and
because of numerous new and revised program requirements resulting from
recent public housing reforms. HUD has since begun providing additional
training to its field office staff. Moreover, this official agreed that
HUD's field offices face resource constraints, particularly in terms of the
number of staff relative to workload. HUD attributes field office staffing
constraints to attrition and staff reductions in recent years but doubts
that it will be able to increase its field office staffing. As a result, HUD
is currently evaluating the field offices' workload, focusing on reducing or
eliminating responsibilities that may no longer be necessary.
HUD's field offices may use sanctions if they find it necessary to do so in
order to force a housing agency to correct a performance problem, such as an
agency's noncompliance with rules for verifying tenants' incomes.
Sanctions- such as restricting a housing agency's use of HUD funds- can
apply whether or not HUD designates the housing agency as troubled. However,
according to our survey, HUD's field offices use sanctions infrequently when
housing agencies violate one or more of HUD's program rules and
requirements. For example, over three- fourths of HUD's field offices
reported that, in their experience, some of the housing agencies they
oversee have needed to improve their performance because they made poor use
of the funding HUD gave them to operate, maintain, or modernize their
buildings. Nonetheless, over the last 5 years, almost 40 percent of the
field offices responding to our survey indicated that they have never
applied a sanction to improve the performance of a housing agency, and over
two- thirds indicated that they have rarely threatened or initiated
sanctions. According to a HUD headquarters official, this may be because
field offices believe some sanctions, such as withholding HUD funds, could
have negative short- term effects on public housing residents and because
field offices instead prefer to work cooperatively with housing agencies to
address problems. The 1998 public housing reform legislation 2 mandates that
HUD use a specific sanction against troubled housing agencies that fail to
substantially improve their performance as HUD measures it through its
Public Housing Assessment System- giving these agencies at most 2 years to
improve before HUD takes possession of the agency or seeks a courtordered
receivership. As a result, HUD now has quantifiable, transparent criteria
for deciding when and how it will hold accountable troubled housing
agencies. Later this year, HUD expects to seek the first receiverships based
on the 1998 public housing legislation.
In commenting on a draft of this report, HUD provided us with additional
information for our consideration. This information primarily dealt with the
importance of the other PHAS indicators for accurately assessing housing
agencies' management, additional quality assurance steps HUD is taking to
ensure accurate management operations data, and new field
office training to improve oversight of and assistance to housing agencies.
However, HUD did not directly address our report's recommendations. Our
detailed discussion of HUD's comments are included at the end of this
letter.
2 Public Law 105- 276, the Quality Housing and Work Responsibility Act of
1998, Oct. 21, 1998.
Background Through the United States Housing Act of 1937, the Congress
created the federal public housing program to help communities provide
decent, safe,
and sanitary rental housing for low- income families, the elderly, and
persons with disabilities. Today, approximately 1.3 million low- income
households live in public housing. Because tenants' rents typically do not
cover the cost of operating public housing, HUD subsidizes more than 3,000
local public housing agencies to help pay for their operating costs, such as
routine maintenance, staff salaries, and administrative expenses, and
provides the housing agencies with technical assistance in planning,
developing, and managing public housing. HUD also provides funds to housing
agencies for major modernization projects through its Capital Fund program.
3 For fiscal year 2000, HUD provided public housing
agencies nearly $7 billion in such subsidies and technical assistance. HUD
oversees public housing agencies' performance by annually measuring their
performance through PHAS and by conducting day- to- day activities to
monitor compliance with, and provide assistance in meeting, the various HUD
program requirements applicable to each housing agency. Annually, HUD
measures the essential housing operations of public housing agencies through
the Public Housing Assessment System (PHAS). 4 In its day- to- day oversight
of housing agencies, HUD- through 43 of its
field offices- monitors housing agencies' compliance with its programs'
rules and requirements, many of which fall outside of the scope of PHAS. 5
Figure 1 shows the six components of housing agencies' management operations
that PHAS measures.
3 HUD recently consolidated various modernization programs into a single
program, the Capital Fund. HUD allocates money from the Capital Fund to
housing agencies on the basis of a formula that includes factors such as the
age of each agency's buildings.
4 On Sept. 30, 1999, the PHAS management operations indicator replaced the
Public Housing Management Assessment Program, which was the framework
through which HUD met statutory requirements to assess management
performance by using certain measures, such as vacancy rates and the average
time a housing agency takes to repair and reoccupy vacant units. When HUD
replaced the previous management assessment program, it also created new
indicators that measure the physical condition of each agency's buildings,
residents' satisfaction with living conditions, and the financial condition
of the housing agency. Together, these four indicators make up PHAS.
5 For example, HUD oversees housing agencies' compliance with fair housing
laws through an annual contract with each agency because PHAS does not
measure housing agencies' performance in this area.
Figure 1: Six Components of PHAS Management Operations
Vacant Unit Turnaround Time: The housing agency's efforts to reduce the
amount of time it takes to re- rent a unit after a tenant moves out and the
adequacy of its system for tracking the time it takes to make units ready
and re- rent them.
Capital Fund: The percentage of its Capital Fund allocation from HUD that a
housing agency obligates within 3 years, the timeliness of these
obligations, the quality of the work for which the agency used the funds,
and the adequacy of the agency's budget controls.
Work Orders: The time the housing agency takes to complete emergency and
nonemergency work orders and its progress in reducing the time it takes to
complete nonemergency maintenance work orders.
1 2 3
Annual Inspections of Units and Systems: The percentage of a housing 4 5 6 7
8 9 10
agency's units and major systems, such as heating and air conditioning, that
11 12 13 14 15 16 17
18 19 20 21 22 23 24 the agency inspects annually to determine short- term
maintenance and longterm 25 26 27
28 29 30 31 Capital Fund needs.
Security: A housing agency's performance in tracking crime- related problems
in its developments, reporting crimes to local law enforcement agencies,
adopting and enforcing resident screening and eviction policies and
procedures and other anticrime strategies, coordinating with residents and
local governments on implementing anticrime strategies, and effectively
using any HUD drug prevention or crime reduction grants the agency may
receive.
Economic Self- Sufficiency: The agency's efforts to coordinate, promote, or
provide effective programs and activities to promote residents' economic
self- sufficiency.
Source: HUD.
PHAS scores determine (1) whether HUD designates a housing agency as a high
performer, standard performer, or troubled housing agency; (2) which HUD
office will be responsible for overseeing the housing agency; and (3)
whether the housing agency will be eligible for certain bonus funding.
Specifically: Using the total PHAS score, high performers are those scoring
90
percent or greater and at least 60 percent of the points available under
each of the four PHAS indicators; standard performers are those with a total
score of 60 to 89 percent and at least 60 percent of the points available in
the physical condition, financial condition, and management
operations indicators; and, troubled agencies are those with total scores of
less than 60 percent. 6 HUD designates as “substandard” those
housing agencies that score less than 18 of the 30 possible points on each
of the management operations, physical condition, or financial condition
indicators. Also, HUD designates agencies that score less than 60 percent of
the maximum calculation for the Capital Fund component as substandard with
respect to the Capital Fund program. Standard- and high- performing housing
agencies remain under the
jurisdiction of a HUD field office, but troubled and substandard agencies
must report to one of HUD's two troubled agency recovery centers, which
provide specialized oversight and recovery assistance to poorly performing
agencies; and, High- performing housing agencies may be eligible for bonus
points when they apply for certain funding HUD awards on a competitive
basis;
also, HUD plans later this year to begin using its Capital Fund program to
award performance bonuses to high performers. Figure 2 illustrates how this
scoring system and process works.
6 The total PHAS score, from 0 to 100 percent, is the summation of an
agency's scores on the management operations, physical condition, and
financial condition components of PHAS (with each worth 30 points), plus the
measure of the residents' satisfaction, which is worth 10 points.
Figure 2: Effect of Housing Agencies' PHAS Scores
PHAS scores 100
Capital Fund Eligible for Under Under
awards bonus points 90 jurisdiction jurisdiction
of of 43 43 HUD HUD
field field offices offices
Performance 60
improves Inadequate improvement Two
in 2 years troubled
agency Receivership recovery
or centers
HUD takeover Inadequate improvement
in 1 year 0
High performer Standard performer Troubled performer
Source: HUD.
To verify that the management operations data housing agencies certify are
accurate, HUD relies on the independent public accountant auditors who
review housing agencies' financial statements each year to verify the
agencies' certifications and the supporting data behind them. 7 HUD expects
the independent auditors to review the documentation and other 7 The Single
Audit Act (31 U. S. C. Chapter 7501- 7507) requires an annual audit of
states, local
governments (including housing agencies), and nonprofit organizations that
annually expend $300, 000 or more of federal funds. The purpose of the
annual audit is to tell HUD and housing agency officials whether the housing
agency has fairly presented its financial statements and to provide
reasonable assurance that the housing agency is managing its programs in
accordance with applicable laws and regulations.
information to substantiate the management operations certifications
submitted by housing agencies. Any major discrepancy between what a housing
agency certifies and what its auditor finds should result in findings in the
auditor's annual audit report to that agency, and these results are also
sent to HUD.
In the course of their day- to- day oversight of housing agencies'
compliance with HUD program requirements, field offices may use sanctions to
bring about improvements in a housing agency's performance, but in most
cases they are not required to do so if they do not believe sanctions are
the best way to improve poor performance. Sanctions available to the field
offices
could include restricting the agency's use of certain HUD program funds or
temporarily withholding funds altogether. Under PHAS, troubled and
substandard agencies face statutory deadlines for improving their
performance and a mandatory sanction if they fail to do so. Specifically, if
a troubled agency does not substantially improve its performance in each of
the 2 years after HUD first designates it as troubled, HUD must either take
over all or part of the agency's operations itself or seek from a court the
appointment of a receiver to take over all or part of the troubled agency's
operations.
HUD Assesses HUD field office officials reported that the components that
make up the
Important Measures of PHAS management operations indicator are important
measures for
assessing the quality of a housing agency's management. However, many of
Management
these officials' responses to our survey also indicated that housing
Performance, but It
agencies' self- certified data about their performance on the components of
Needs Better Data on
management operations can be inaccurate. HUD relies on the independent
audits that many housing agencies must have done each year to alert it
Housing Agencies' when management operations data might be inaccurate.
However, HUD Performance
has found during a limited number of quality control reviews of these audits
that some auditors have not done the work HUD expected of them and others
are unclear about HUD's expectations. According to our survey, in 15
instances involving troubled housing agencies, the audits did not identify
for HUD problems with the agencies' self- certified data.
HUD's Public Housing To have an effective oversight system, HUD must ensure
that it is Assessment System
measuring the right things- that is, it is using valid measures of Includes
Components Key management performance and has not overlooked additional
components to Measuring Management
identified by those most familiar with housing agency operations. Our survey
results indicate the field office officials believe HUD is measuring
Performance and the right things in order to be sure housing agencies are
providing safe and Identifying Poor Performers
decent housing and protecting the federal investment in their properties. As
figure 3 shows, field office officials concluded that nearly all of the PHAS
management operations components are important measures for gauging how well
a housing agency is managing its public housing program.
Figure 3: PHAS Management Operations Components Rated as Moderately, Very,
or Extremely Important
PHAS Management Operations Components
Capital Fund Work orders Annual inspections of units and systems
Security Economic self- sufficiency
0 20 40 60 80 100 Percent of respondents
Note: The “vacant unit turnaround” component was not part of the
PHAS management operations indicator at the time we developed our survey.
The management operations components we included in our survey were drawn
from the January 11, 2000 final PHAS rule, which listed the five components
we show in figure 3. In June 2000, HUD issued a technical correction to the
PHAS rule, noting that it (1) had inadvertently removed the vacant unit
turnaround component when it also removed a separate
vacancy- related component (vacancy rate) and (2) was reinstating the vacant
unit turnaround component as part of the PHAS management operations
indicator. Source: GAO's survey of HUD's field office directors of public
housing.
Field offices rated only the economic self- sufficiency component as
noticeably less important than the others, possibly because economic
selfsufficiency focuses on the nonhousing needs of public housing residents.
Additionally, few offered suggestions on other areas of performance HUD does
not already measure.
HUD Lacks Adequate HUD lacks adequate assurance that housing agencies'
management Assurance That Housing
operations data are accurate because it does not have in place a proven
Agencies' Management
way to systematically verify the agencies' self- certified performance data.
Operations Data Are
HUD first depends on the housing agencies to collect performance data
throughout the year in order to measure their management operations Accurate
performance. At the end of their fiscal year( s), housing agencies analyze
their performance data, report the information to HUD, and certify that the
information is accurate. However, over 40 percent of the field offices
responding to our survey reported that in the past, housing agencies
certified to inaccurate or incomplete management operations data to a
moderate or greater extent. Most often, they said, they believe housing
agencies did so because the agencies (1) failed to collect all the data HUD
requires; (2) misinterpreted HUD program requirements; or (3) lacked the
expertise necessary to collect, analyze, and present detailed management
operations performance data.
To substantiate housing agencies' management operations data, HUD relies
heavily on the agencies' independent audits to alert it to whether any of
the data may be inaccurate. HUD officials stated that it is necessary to use
the independent audit to substantiate the management operations data to
which the agencies self- certify because its field offices usually do not
have sufficient resources to visit significant numbers of housing agencies
to do
so themselves. Prior to PHAS, HUD required its field offices to visit some
housing agencies to verify the agencies' self- certified data. Under PHAS,
HUD no longer requires these visits. Instead, as one HUD official
characterized it, the independent audit is the Department's first line of
quality control in ensuring that housing agencies' management operations
data are accurate.
According to HUD headquarters officials, because of the importance of the
audit to its oversight of housing agencies, HUD has begun performing quality
control reviews of the independent auditors' work for public housing
agencies. These reviews are to assess the audits' compliance with
professional standards and to ensure that the auditors comply with the
associated statutory requirements. HUD envisions these reviews as a
deterrent against substandard work by housing agencies' auditors. To date,
HUD headquarters has performed only a limited number of reviews of housing
agencies' audits, but it has found instances in which the auditors have not
performed the work HUD requires of them, including testing the housing
agencies' management operations data. HUD has also found that some auditors
are unclear about what HUD expects they should and should not report when
they find discrepancies between these data and the agencies' certifications.
HUD's preliminary findings through these quality control reviews may explain
some of the reasons why, according to our survey, HUD has not always learned
through the agencies' audits when agencies have certified to inaccurate
data. Specifically, according to the field offices responding to our survey,
prior to PHAS, 21 housing agencies certified to data indicating they were
standard- or high- performing agencies. The field offices reported that HUD
later determined these 21 agencies were, in fact, troubled during the time
they had made these certifications. For six of these agencies, the
field offices reported that the housing agencies' audits had identified
problems with the agencies' certifications. However, for 15 of these
agencies, the field offices reported that the audits had not identified such
problems. Under PHAS, HUD is using additional quality assurance activities
to analyze the housing agencies' self- certified data before it either
issues
PHAS scores or, for most agencies, will have received their independent
audits. HUD's Real Estate Assessment Center, which receives and processes
all housing agencies' certifications, performs an initial review to identify
if any submission is incomplete or contains any obvious errors. The field
offices then perform a second quality assurance review to ensure housing
agencies' certifications are consistent with their knowledge of the
agencies. Field offices can ask that a housing agency's certification be put
on hold if they have questions about it. If the field office has
documentation that a housing agency's certification is incorrect, the Real
Estate Assessment Center can reject the agency's PHAS score. Neither of
these
quality assurance reviews involves first- hand verification of the housing
agencies' management operations data. Furthermore, HUD's field offices may
not be in a position to provide the level of quality assurance HUD is
expecting. Specifically, slightly over 80 percent of the field offices
responding to our survey said that, under PHAS, HUD probably or definitely
does not have the tools and resources necessary to ensure that the
management operations data that housing agencies certify are accurate and
complete.
Under PHAS, if HUD does not identify instances of inaccurate self- certified
data, such as those the field offices reported to us, it could potentially
be costly to other housing agencies. A housing agency's PHAS management
operations score, which HUD determines on the basis of the data the agency
self- certifies, can be as much as 30 of the 90 points that it takes to be
designated a PHAS high performer. Later this year, HUD plans to begin
awarding the Capital Fund performance bonuses to PHAS high performers, with
the funds for these bonuses coming out of the funds HUD otherwise
allocates to standard- performing and troubled housing agencies. 8
Additionally, HUD has said that PHAS high performers may, in the future, get
bonus points when HUD considers their applications for funding that is
awarded on a competitive basis. Field Offices Report
Field office officials identified a number of factors- primarily related to
They Are Adequately
their staffing and resources- that they judge as most important to providing
effective assistance to housing agencies that need to improve Prepared for
Certain
their performance (see fig. 4). They also further described the areas
Important Assistance
(factors) in which they believe they currently are adequately or to Housing
Agencies, inadequately prepared to assist housing agencies. but Other Key
Factors May Be Inadequate
8 The 1998 public housing reform legislation requires that HUD include in
the Capital Fund a mechanism to reward high performance (42 U. S. C. 1437g
(d) (2)). While HUD is authorized to use PHAS scores as the mechanism, the
legislation does not mandate that it use these scores.
Figure 4: Factors Field Offices Judge Most Important to Providing Effective
Assistance to Housing Agencies
Important factors Field office expertise
(HUD programs) Workload relative to qualifications and training
Working knowledge of housing agencies
Workload relative to number of staff
Number of field office staff Training funds
Travel funds Technical assistance funds
Field office expertise (rental property management)
0 20 40 60 80 100 Percent of respondents
Source: GAO's survey of HUD's field office directors of public housing. Of
the nine factors they judge most important to providing effective assistance
to housing agencies, field office officials reported that, currently,
conditions are adequate or better for four of these factors. 9 Specifically:
9 “Adequate or better” refers to the sum of those field offices
answering “adequate,” “more than adequate,” or
“much more than adequate.”
Over 70 percent of the field offices had adequate or better expertise in HUD
programs, over 80 percent had an adequate or better working knowledge of
housing agencies,
just over half of the field offices had an adequate or better amount of
travel funds available to them to visit housing agencies, and just over half
had an adequate or better amount of expertise in rental property management.
With respect to five of the nine factors they judge most important to
providing effective assistance to housing agencies, a majority of the field
offices rated their current conditions as less or much less than adequate
for providing effective assistance to housing agencies. Specifically:
Nearly 60 percent said their workload relative to the qualifications and
training of the field office staff was less or much less than adequate, 70
percent said their workload relative to the number of field office staff was
less or much less than adequate, over 80 percent had a less or much less
than adequate number of field
office staff, over 85 percent said that the amount of field office training
funds is less
or much less than adequate, and over 75 percent said that the amount of
technical assistance funds
available to the field office is less or much less than adequate. A number
of reasons may explain why the field offices rated these five factors as
currently inadequate for assisting housing agencies. According to the HUD
headquarters official responsible for field office operations, there has
been substantial turnover in recent years among HUD's field office directors
of public housing, with at least half new to their positions within the last
2 years. This turnover, as well as the need for field office staff to be
up to date with over 60 new or revised program requirements associated with
the 1998 public housing reform legislation, has increased the need for and
the importance of ongoing training for field office staff. However, this
same official acknowledged that HUD has not had a great deal of funds
available for field office training in recent years, which probably explains
why 85 percent of the field offices reported that the amount of training
funds available to them is less or much less than adequate. In commenting on
a draft of this report, HUD noted that in response to the relative
inexperience of some of its field office directors of public housing, it has
developed managerial and program monitoring training courses that
focus on the full range of HUD programs and activities for which they are
responsible. HUD also commented that it had recently convened a 2- week
training session to provide field office staff with the knowledge and skills
they need to (1) effectively monitor housing agencies' compliance with the
new and/ or revised requirements associated with the 1998 public housing
legislation and (2) provide the housing agencies technical assistance in the
implementation and maintenance of HUD programs. The HUD headquarters
official responsible for field office operations also noted that the number
of staff in field offices has decreased in recent years because of attrition
and departmental downsizing, adding that it has been
and remains unlikely that HUD will be able to increase staffing in its field
offices. As a result, HUD is currently considering alternative ways to
address its field offices' workload. HUD has under way a study of its field
offices' workload with a focus on reducing or eliminating responsibilities
where it might make sense to do so. Also, instead of on- site visits to
housing agencies, HUD has been emphasizing to field office staff the
importance of remotely monitoring housing agencies as much as possible by
means such as a new Internet- based system HUD is implementing to share
information between housing agencies and HUD staff. HUD Sanctions
Regardless of whether or not HUD designates a housing agency as Housing
Agencies troubled, some housing agency performance problems can result in
HUD's applying a sanction to the agency in order to compel it to address and
Infrequently, but
correct the problem( s). For example, HUD can restrict a housing agency's
Statutory Reforms use of the operating funds that it provides if the agency
is not complying Have Improved How
with rules for verifying tenants' incomes. However, we found that HUD's
field offices infrequently sanction housing agencies for poor performance
HUD Holds Troubled when they may have reason to do so. The 1998 public
housing reform Agencies Accountable
legislation improved HUD's effectiveness at holding accountable the agencies
HUD has designated as troubled when they fail to improve their performance.
It did so by mandating that HUD seek a specific sanction- a takeover either
by HUD or a court- appointed receiver- for troubled
housing agencies that fail to substantially improve their performance within
1- and/ or 2- year deadlines. HUD uses PHAS to determine whether a housing
agency has substantially improved its performance. Field Offices
Infrequently
HUD field offices infrequently use the sanctions available to them to hold
Sanction Housing Agencies housing agencies accountable when they fail to
substantially improve their performance in accordance with program rules or
statutory requirements.
According to our survey, three- fourths of the field offices indicated that
some of the housing agencies in their jurisdiction have needed to improve
their performance because they made poor use of federal funding for their
operations, maintenance, and modernization and revitalization efforts.
Also, three- fourths of the field offices indicated that some housing
agencies in their jurisdiction have needed to improve because of poor
performance by their executive directors, staff, or governing body. More
than two- thirds of the field offices responding to our survey also reported
that in the last 5 years, they either never or sometimes have threatened to
use or initiated sanctions against housing agencies. Specifically, when we
asked how often the HUD field offices threaten or initiate sanctions, they
reported the following: 29 do so never or sometimes;
9 do so to a moderate extent; and, 3 very often or always threaten or
initiate sanctions.
Additionally, 40 percent of HUD's field offices responding to our survey
said that in the last 5 years they have never applied a sanction or other
punitive action( s) to improve the performance, governance, regulatory
compliance, or other behavior of a housing agency.
Field office staff told us that HUD does not necessarily have to go so far
as to apply a sanction to effect a change for the better in a housing
agency's performance. According to some HUD officials, the threat of a
sanction- such as prohibiting a housing agency official from having anything
to do with HUD programs for a period of time- can have just as positive an
effect as actually applying the sanction because it creates a sense of
urgency for the agency to do what it takes to make the sanction unnecessary.
Those field offices that had applied sanctions, or reported that they had
some basis to judge, are- for the most part- rarely applying the sanctions
they believe are most effective at improving housing agencies' performance.
Figure 5 shows the three sanctions the field offices rated as most effective
at improving housing agency performance and, for those
same offices, how many indicated they rarely use it.
Figure 5: Limited Field Office Use of Most Effective Sanctions
Sanction Restrict uses( s) of HUD funds
Respondents who reported
3
the sanction is "neither Rarely use
effective nor ineffective" or
22
sanction "generally or very ineffective"
33
Respondents who reported they Very or generally effective
11
"moderately" or "often to very often" use the sanction
Withhold HUD funds
Respondents who reported
7
the sanction is "neither effective nor ineffective" or
Rarely use "generally or very ineffective"
19
sanction
25
Respondents who reported they Very or generally effective
6
"moderately" or "often to very often" use the sanction
Issue a limited denial of participation a
Respondents who reported
5
the sanction is "neither effective nor ineffective" or "generally or very
ineffective"
Rarely use
21
sanction Very or generally effective
25
Respondents who
4
reported they "moderately" or "often to very often" use the sanction
a A limited denial of participation excludes for a specific period of time a
specific individual or firm( s) from participating in a specific program or
programs within the jurisdiction of the field office issuing the denial.
Source: GAO's survey of HUD field office directors of public housing.
According to a HUD headquarters official, concern about the possible short-
term effect on residents when HUD does use certain sanctions against a
housing agency may account, in part, for why field offices so rarely make
use of sanctions. For example, if a housing agency is not administering its
Capital Fund allocation properly, withholding those funds until it improves
its performance could be seen as penalizing the residents
who ultimately benefit when the agency uses those funds to modernize or
renovate residents' homes. Also, the HUD headquarters official responsible
for field office operations stated that HUD does not have any official
guidance for its field offices on initiating or using sanctions. As a
result, this official stated, some field staff may feel vulnerable when they
want to use sanctions against poor- performing housing agencies because they
fear
headquarters might overrule them. This official acknowledged that HUD's
cooperative approach to working with poorly performing agencies is a barrier
to its use of sanctions and stated that the Department needs to do a better
job of using sanctions when it is appropriate. In commenting on a draft of
this report, HUD noted that it has subsequently begun training field office
staff on providing targeted technical assistance to noncompliant housing
agencies and assisting these agencies in developing strategies to improve
performance prior to the field offices' using discretionary sanctions.
1998 Public Housing Reform Recent legislative reforms have improved HUD's
ability to hold poorly
Legislation Has Made HUD performing housing agencies accountable for their
management operations More Effective at Holding because they mandate
deadlines for certain poorly performing agencies to Housing Agencies
improve. Specifically, troubled housing agencies must show substantial
Accountable for Improving
improvement in their performance and now have at most 2 years to do so. HUD
defines substantial improvement in terms of PHAS scores and, Their
Performance
pursuant to the reform legislation, requires the following: Within 1 year of
being designated as troubled, the housing agency must improve by at least 50
percent of the difference between its PHAS score
and 60 percent- the minimum threshold for being a standard performer. 10 10
For example, a troubled housing agency with a PHAS score of 50 would have to
improve to a score of 55 within 1 year because that would be 50 percent of
the difference between its score and 60. A substandard management operations
agency scoring 10 out of 30 points would have to improve to 14 within 1 year
(halfway to 18, or 60 percent of the 30 available
points).
Within 2 years, the housing agency must improve its PHAS score to at least
60. Prior to PHAS, troubled housing agencies faced no such deadline or
threshold for improving their performance so that they were no longer
designated as troubled. Instead, each troubled agency was required to enter
into a binding memorandum of agreement with HUD that spelled out
performance improvement goals, deadlines for improvement, and the
possibility of sanctions if the agency failed to meet any of the terms of
the agreement. However, according to our survey, when troubled housing
agencies failed to meet such deadlines, the field offices reported that
their most frequent response was to extend the deadlines. Specifically,
field
offices reported that in 21 instances, troubled housing agencies failed to
meet one or more performance improvement goals to which they had agreed in
writing with HUD. HUD's field offices responded by extending
the agencies' deadlines 19 times. According to HUD officials, the 1998
public housing reform legislation has simplified for them the process of
holding troubled agencies accountable when they fail to improve their
performance in a timely manner. Because HUD implements the statutory
provisions through PHAS, it now has quantifiable and transparent criteria-
time frames and PHAS scores- for determining when and how it will hold these
housing agencies accountable.
Specifically, HUD officials stated that they now consider it automatic that,
1 or 2 years after designating a housing agency as troubled, HUD will, for
any agency that fails to improve as the statute requires, take possession of
that agency or seek from a court the appointment of a receiver to take over
all or part of the agency's operations. 11 Later this year, HUD officials
expect to seek the first receiverships in accordance with the reforms in the
1998
public housing legislation. Conclusions We agree with HUD that it makes
sense to reward high performance by housing agencies. Doing so can encourage
the best agencies to try to continue performing at a high level and provide
strong motivation to other housing agencies to match the performance of the
best- ultimately, it 11 The statute directs that for housing agencies with
fewer than 1, 250 units, HUD must either take over all or part of the
agency's operations itself or seek from a court the appointment of a
receiver to take over all or part of the troubled agency's operations; for
housing agencies with 1, 250 or more units, HUD must seek the judicial
appointment of a receiver.
stands to reason, benefiting residents of public housing. Because HUD is, in
the end, accountable for the performance of the housing agencies, it would
benefit as well. However, when HUD proposes to reward high performance
at the expense of those it judges to be standard- performing or troubled
agencies- as will be the case under its proposed Capital Fund performance
bonuses- it is all the more important that HUD make certain it is accurately
measuring performance in the first place. Only then, we believe,
would it be appropriate for HUD to use PHAS scores as the basis for
performance bonuses or bonus points when it awards competitive grants.
Reassessing the field offices' workload relative to their staffing, as HUD
is doing, is a reasonable approach when it believes it is unlikely to
receive the resources it would need to increase substantially the number of
staff in its field offices. If, at the same time, HUD still wants to use
PHAS scores as the basis for funding decisions, such as Capital Fund
performance bonuses, then it would be wise to consider the workload that
would be required to itself verify the PHAS management operations data that
housing agencies self- certify. Such a consideration on HUD's part need not
assume that there
must be an annual, on- site review of each and every agency's certification
but could reflect the risk associated with each housing agency. Those
housing agencies demonstrating they are low risk- for example, those not
among the agencies the field offices said often certified inaccurately-
could reasonably be subject to such a review less often than others.
The threat of a sanction can itself be an effective way to improve a housing
agency's performance- as some HUD officials told us they have learned from
experience. How often the Department should threaten or actually sanction a
housing agency- in circumstances where the 1998 legislation does not mandate
it- is a determination we believe that HUD is in the best position to make.
We would suggest that the results from our survey might be a start for HUD
to make such a determination. With so many of HUD's field offices indicating
they almost never threaten or initiate sanctions against housing agencies,
and even fewer having experience applying
sanctions, few are likely to have learned first- hand the potential benefits
from holding housing agencies more strictly accountable. Recommendations for
To better ensure that HUD is accurately measuring the performance of all
Executive Action
housing agencies so that it can reward high performers and hold accountable
troubled and substandard agencies, the Secretary of Housing and Urban
Development should
develop and implement a cost- effective approach for HUD's field offices to
verify housing agencies' management operations certifications for accuracy
and completeness, and
ensure that HUD uses adequately verified housing agency performance
information as the basis for (1) the Capital Fund performance bonuses and
(2) bonus points on applications for competitive grants. Agency Comments and
We provided HUD with a draft of this report for its review and comment. Our
Evaluation
While HUD did not directly address the report's recommendations, it did
provide us with additional information for our consideration. This
information primarily dealt with (1) the importance of the other PHAS
indicators for ensuring that HUD accurately assesses housing agencies'
management performance; (2) additional quality assurance steps HUD is taking
to ensure housing agencies' management operations data are
accurate; and (3) training HUD has provided its field offices to improve
their oversight of and assistance to housing agencies.
First, HUD emphasized that the PHAS management operations indicator is not
intended to be a stand- alone measure of housing agencies' management
performance. According to HUD, all four PHAS indicators together- management
operations, physical condition, financial condition, and resident
satisfaction- are used to evaluate management performance. HUD characterized
the four PHAS indicators as providing a system of checks and balances to
ensure HUD accurately assesses housing agencies' management performance. For
example, according to HUD, the PHAS
resident satisfaction indicator will provide information regarding housing
agencies' completion of work orders and security operations, both of which
HUD measures through the management operations indicator. HUD
also noted that the physical condition indicator will provide an independent
assessment of housing agencies' performance on the annual inspections of
units and systems component of the management operations indicator. We agree
with HUD that poor performance by a housing agency on the
components measured through the PHAS management operations indicator will at
some point be likely to affect the agency's performance on the physical
condition, financial condition, or resident satisfaction PHAS indicators.
However, rather than wait for poor performance in management operations to
manifest itself in the other PHAS indicators, we believe it is important for
HUD to have an accurate measure of each housing agency's
performance each year. HUD must do so because (1) housing agencies that
are substandard in any one of the PHAS management operations, physical
condition, or financial condition indicators face the same consequences as
those that are troubled, as determined by an aggregate score for all four
PHAS indicators; and (2) annual performance bonuses for high performers- as
HUD is proposing for the Capital Fund- require an accurate measure of
performance in all of the PHAS indicators each year. Consequently, we have
left intact our recommendation that HUD ensure it uses adequately verified
housing agency performance information for
purposes such as the Capital Fund performance bonuses. Second, HUD stated
that in addition to relying on housing agencies' independent audits to
identify when housing agencies submit inaccurate management operations data,
it is using other quality assurance activities to analyze each agency's data
prior to issuing PHAS scores. We have added information to our report
explaining the activities of its Real Estate Assessment Center (which
receives and processes all housing agencies'
certifications), as well as the field offices, prior to HUD's issuing final
PHAS scores. HUD also noted that its field offices are being provided
training to review and correlate the information for and between all PHAS
indicators to better enable them to identify possible instances of
inaccurate housing agency certifications. We agree with HUD that it makes
sense to turn to its field offices to better
assure itself whether it should issue PHAS scores on the basis of housing
agencies' self- certified management operations data. The field offices have
the most direct, day- to- day experience in overseeing and providing
assistance to the housing agencies. However, we question HUD's reliance
on the quality assurance activities it mentioned. These activities do not
involve verification of management operations data, and as our survey
results indicated, the field offices do not believe that, under PHAS, HUD
currently has the tools and resources necessary to ensure that housing
agencies' management operations data are accurate and complete.
Consequently, we have left intact our recommendation that HUD develop in its
field offices a cost- effective approach to verify housing agencies'
management operations certifications.
Finally, HUD noted that the Department has provided, and plans to continue
providing, training to its field office staff in several areas, including
their new and/ or revised responsibilities under the 1998 reform legislation
and the PHAS indicators. HUD also indicated that through indepth training,
its field office staff will learn how to provide targeted technical
assistance for housing agencies that are not complying with
program rules and to assist the agencies in developing strategies to improve
their performance prior to HUD's using discretionary sanctions. We have
added information to our report explaining in greater detail HUD's plans for
additional field office training.
HUD also provided technical clarifications to our report, which we have
incorporated as appropriate.
Scope and To assess HUD's oversight of the management performance of housing
Methodology
agencies, we conducted a mail survey of all of the directors of public
housing in HUD's Hubs and Program Centers, which are the 43 field offices
responsible for overseeing these agencies. We surveyed these officials
because they have direct experience in providing both oversight and
assistance to public housing agencies in the 50 states and the District of
Columbia. This experience, we believe, makes them qualified to speak to the
issues associated with the effectiveness of HUD's oversight. Because we
believed that the HUD field staff might view some of our questions as
sensitive, we pledged to hold in confidence their individual responses and
not identify any of these HUD officials with his or her responses. We did so
in order to ensure candor in their responses to questions they might
consider sensitive while still allowing us to follow up on the information
they provided. We pretested our questionnaire with three HUD field offices
and modified it on the basis of the feedback and comments we received during
the pretests. We received responses from 41 of the 43 HUD field office
officials, which is a 95- percent response rate. Our survey results reflect
the information provided by, and the opinions of, the HUD field office
officials. We did not independently verify the field offices' responses to
our questions.
Throughout our review, we also met with HUD headquarters and field office
officials involved in oversight of housing agencies. We also visited HUD's
two troubled agency recovery centers in Cleveland, Ohio, and Memphis,
Tennessee. In addition, we reviewed relevant documents from each of these
offices, past GAO and HUD Office of Inspector General reports on issues
associated with oversight of housing agencies, and statutory and regulatory
materials applicable to HUD's oversight. We conducted our review from
October 1999 through October 2000 in accordance with generally accepted
government auditing standards.
As arranged with your office, unless you publicly announce its contents
earlier, we plan no further distribution of this report until 10 days after
the date of this letter. At that time, we will send copies to the
appropriate congressional committees and the Honorable Andrew M. Cuomo,
Secretary
of Housing and Urban Development. We will also make copies available to
others upon request. Please call me at (202) 512- 2834 if you or your staff
have any questions about the information in this report. Key contributors to
this report are listed in appendix II. Sincerely yours,
Stanley J. Czerwinski Director, Physical Infrastructure Issues
Appendi xes Housing Agencies' Management Performance
Appendi x I Survey
Appendi x II
GAO Contacts and Staff Acknowledgments GAO Contacts Stanley J. Czerwinski,
(202) 512- 2834 Paul Schmidt, (312) 220- 7681 Acknowledgments In addition to
those named above, Johnnie E. Barnes, Anne A. Cangi, Sherrill Dunbar,
Jacqueline Garza, Bill MacBlane, Eric Marts, and Luann
Moy made key contributions to this report.
(385827) Lett er
Ordering Information The first copy of each GAO report is free. Additional
copies of reports are $2 each. A check or money order should be made out to
the Superintendent of Documents. VISA and MasterCard credit cards are
accepted, also. Orders for 100 or more copies to be mailed to a single
address are discounted 25 percent.
Orders by mail: U. S. General Accounting Office P. O. Box 37050 Washington,
DC 20013
Orders by visiting: Room 1100 700 4th St. NW (corner of 4th and G Sts. NW)
U. S. General Accounting Office Washington, DC
Orders by phone: (202) 512- 6000 fax: (202) 512- 6061 TDD (202) 512- 2537
Each day, GAO issues a list of newly available reports and testimony. To
receive facsimile copies of the daily list or any list from the past 30
days, please call (202) 512- 6000 using a touchtone phone. A recorded menu
will provide information on how to obtain these lists.
Orders by Internet: For information on how to access GAO reports on the
Internet, send an e- mail message with “info” in the body to:
info@ www. gao. gov or visit GAO's World Wide Web home page at: http:// www.
gao. gov
To Report Fraud,
Contact one:
Waste, or Abuse in Web site: http:// www. gao. gov/ fraudnet/ fraudnet. htm
Federal Programs
e- mail: fraudnet@ gao. gov 1- 800- 424- 5454 (automated answering system)
GAO United States General Accounting Office
Page 1 GAO- 01- 94 Public Housing
Contents
Page 2 GAO- 01- 94 Public Housing
Page 3 GAO- 01- 94 Public Housing United States General Accounting Office
Washington, D. C. 20548 Page 3 GAO- 01- 94 Public Housing
Page 4 GAO- 01- 94 Public Housing
Page 5 GAO- 01- 94 Public Housing
Page 6 GAO- 01- 94 Public Housing
Page 7 GAO- 01- 94 Public Housing
Page 8 GAO- 01- 94 Public Housing
Page 9 GAO- 01- 94 Public Housing
Page 10 GAO- 01- 94 Public Housing
Page 11 GAO- 01- 94 Public Housing
Page 12 GAO- 01- 94 Public Housing
Page 13 GAO- 01- 94 Public Housing
Page 14 GAO- 01- 94 Public Housing
Page 15 GAO- 01- 94 Public Housing
Page 16 GAO- 01- 94 Public Housing
Page 17 GAO- 01- 94 Public Housing
Page 18 GAO- 01- 94 Public Housing
Page 19 GAO- 01- 94 Public Housing
Page 20 GAO- 01- 94 Public Housing
Page 21 GAO- 01- 94 Public Housing
Page 22 GAO- 01- 94 Public Housing
Page 23 GAO- 01- 94 Public Housing
Page 24 GAO- 01- 94 Public Housing
Page 25 GAO- 01- 94 Public Housing
Page 26 GAO- 01- 94 Public Housing
Page 27 GAO- 01- 94 Public Housing
Page 28 GAO- 01- 94 Public Housing
Appendix I
Appendix I Housing Agencies' Management Performance Survey
Page 29 GAO- 01- 94 Public Housing
Appendix I Housing Agencies' Management Performance Survey
Page 30 GAO- 01- 94 Public Housing
Appendix I Housing Agencies' Management Performance Survey
Page 31 GAO- 01- 94 Public Housing
Appendix I Housing Agencies' Management Performance Survey
Page 32 GAO- 01- 94 Public Housing
Appendix I Housing Agencies' Management Performance Survey
Page 33 GAO- 01- 94 Public Housing
Appendix I Housing Agencies' Management Performance Survey
Page 34 GAO- 01- 94 Public Housing
Appendix I Housing Agencies' Management Performance Survey
Page 35 GAO- 01- 94 Public Housing
Appendix I Housing Agencies' Management Performance Survey
Page 36 GAO- 01- 94 Public Housing
Appendix I Housing Agencies' Management Performance Survey
Page 37 GAO- 01- 94 Public Housing
Appendix I Housing Agencies' Management Performance Survey
Page 38 GAO- 01- 94 Public Housing
Appendix I Housing Agencies' Management Performance Survey
Page 39 GAO- 01- 94 Public Housing
Appendix I Housing Agencies' Management Performance Survey
Page 40 GAO- 01- 94 Public Housing
Appendix I Housing Agencies' Management Performance Survey
Page 41 GAO- 01- 94 Public Housing
Appendix I Housing Agencies' Management Performance Survey
Page 42 GAO- 01- 94 Public Housing
Appendix I Housing Agencies' Management Performance Survey
Page 43 GAO- 01- 94 Public Housing
Appendix I Housing Agencies' Management Performance Survey
Page 44 GAO- 01- 94 Public Housing
Appendix I Housing Agencies' Management Performance Survey
Page 45 GAO- 01- 94 Public Housing
Appendix I Housing Agencies' Management Performance Survey
Page 46 GAO- 01- 94 Public Housing
Appendix I Housing Agencies' Management Performance Survey
Page 47 GAO- 01- 94 Public Housing
Appendix I Housing Agencies' Management Performance Survey
Page 48 GAO- 01- 94 Public Housing
Appendix I Housing Agencies' Management Performance Survey
Page 49 GAO- 01- 94 Public Housing
Appendix I Housing Agencies' Management Performance Survey
Page 50 GAO- 01- 94 Public Housing
Appendix I Housing Agencies' Management Performance Survey
Page 51 GAO- 01- 94 Public Housing
Page 52 GAO- 01- 94 Public Housing
Appendix II
United States General Accounting Office Washington, D. C. 20548- 0001
Official Business Penalty for Private Use $300
Address Correction Requested Bulk Rate
Postage & Fees Paid GAO Permit No. GI00
*** End of document. ***