HUD Housing Portfolios: HUD Has Strengthened Physical Inspections but
Needs to Resolve Concerns About Their Liability (Letter Report,
07/25/2000, GAO/RCED-00-168).

Pursuant to a congressional request, GAO reviewed the Real Estate
Assessment Center's (REAC) new physical inspection system for the
Department of Housing and Urban Development's (HUD) public and
multifamily housing properties, focusing on whether: (1) REAC's
inspection system represents an improvement over HUD's prior inspection
systems; and (2) REAC's inspections are reliable.

GAO noted that: (1) under the new physical inspection system, as in the
past, the primary basis for HUD's physical inspection standards is the
statutory requirement for decent, safe, and sanitary housing; (2) HUD's
new standards require that both multifamily and public housing be
decent, safe, sanitary, and in good repair; (3) while the new standards
do not differ substantially from the previous ones, HUD substantially
revised the procedures used to administer the standards; (4) HUD
believes that the new system produces more objective assessments than
those performed under previous systems; (5) these assessments provide a
basis for targeting resources to the properties that need the most
attention; (6) according to REAC's data, about 87 percent of the
multifamily properties and 80 percent of the public housing properties
reviewed as of April 2000 received scores of at least 60, which HUD
considers satisfactory; (7) however, when REAC quality assurance staff
performed on-site follow-up reviews to assess the adequacy of completed
inspections, they often found that the inspections were not carried out
consistently with REAC's requirements; (8) the quality assurance
reviewers determined during follow-up reviews performed in 1999 and the
early part of 2000 that about 35 percent of the inspections did not meet
REAC's standards; (9) however, because these reviews were not performed
on a random basis and the effects of the identified problems on the
inspection scores cannot be readily determined, it is unclear to what
extent the problems with the inspections affected the overall inspection
results; (10) although REAC deserves credit for establishing quality
assurance procedures, GAO found gaps and weaknesses in some of these
procedures that substantially limited their effectiveness; (11)
representatives of public housing authorities and multifamily housing
industry groups that GAO interviewed also had concerns about the
reliability of REAC's inspections; (12) they were concerned that the
scores may not necessarily reflect the actual condition of some
properties, noting that, in some instances, there were substantial
differences in the inspection scores received by properties that had
undergone more than one inspection; (13) they were also concerned that
HUD may lack the resources needed to ensure the efficient operation of
its appeals procedures; and (14) while some housing industry groups
suggested alternatives to the current physical inspection system, none
of the groups GAO contacted had conducted analyses to compare the cost
and accuracy of these alternatives and HUD's current system.

--------------------------- Indexing Terms -----------------------------

 REPORTNUM:  RCED-00-168
     TITLE:  HUD Housing Portfolios: HUD Has Strengthened Physical
	     Inspections but Needs to Resolve Concerns About Their
	     Liability
      DATE:  07/25/2000
   SUBJECT:  Public housing
	     Data integrity
	     Housing programs
	     Rental housing
	     Building inspection
	     Building codes
	     Quality control
	     Quality assurance
IDENTIFIER:  REAC Physical Inspection System
	     HUD Section 8 Housing Assistance Program
	     REAC Quality Assurance Program
	     HUD 2020 Management Reform Plan

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO Testimony.                                               **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
******************************************************************
GAO/RCED-00-168

Appendix I: Results of Baseline Inspections Conducted by
HUD's Real Estate Assessment Center

34

Appendix II: Industries' Concerns and HUD's Responses

39

Appendix III: Objectives, Scope, and Methodology

43

Appendix IV: Comments From the Department of Housing
and Urban Development

45

Appendix V: GAO Contacts and Staff Acknowledgments

50

Table 1: Physical Inspection Results for Properties in HUD's
Baseline Inventory, by U.S. Region 37

Table 2: Physical Inspection Results for Multifamily Properties
in HUD's Baseline Inventory, by Type of Financing
and Assistance 38

Figure 1: Number of Multifamily and Public Housing Properties
With Inspection Scores of Less Than 60 and 60 or Higher 12

Figure 2: Physical Conditions That REAC Quality Assurance
Reviewers Found Contract Inspectors Did Not Record at Properties GAO Visited
16

Figure 3: Percentage of Multifamily and Public Housing Property Inspections,
Sorted by Inspection Scoring Range 35

Figure 4: Percentage of Multifamily and Public Housing
Property Units, Sorted by Inspection Scoring Range 36

HUD Department of Housing and Urban Development

GAO General Accounting Office

REAC Real Estate Assessment Center

Resources, Community, and
Economic Development Division

B-283608

July 25, 2000

Congressional Committees

In 1997, as part of its 2020 Management Reform Plan, the Department of
Housing and Urban Development (HUD) created the Real Estate Assessment
Center (REAC) to obtain consistent information on, among other things, the
physical condition of its public and multifamily housing. In the past, for
most programs, HUD obtained data on the physical condition of its housing
from property inspections, but there were differences in the way the
inspections were performed and in the data that HUD collected on property
conditions.

In late 1998, REAC began implementing a new electronically based system for
inspecting the physical condition of HUD's public and multifamily housing
properties. Under the new system, HUD relies primarily on contractors to
conduct inspections but uses its own staff to monitor and oversee the
contractors through several quality assurance procedures, including on-site
reviews, to determine whether the inspections were carried out in accordance
with REAC's standards and protocols. HUD plans to use the data from REAC's
inspections not only to assess the condition of properties but also to
strengthen its oversight of public and multifamily housing.

You expressed concerns about whether REAC's inspections represent an
improvement over HUD's prior inspection systems and whether REAC's
inspections are reliable. Accordingly, this report

ï¿½ discusses the basis for the new physical inspection standards HUD has
established for public and multifamily housing and any differences between
the new standards and inspection procedures and those formerly used by HUD;

ï¿½ summarizes the results of the first round of physical inspections and
determines what REAC's reviews of these inspections show about their
reliability;

ï¿½ determines whether REAC's quality assurance measures are sufficient to
ensure that the inspections are reliable and that any problems with the
inspections are resolved expeditiously; and

ï¿½ describes concerns expressed by housing industry representatives about the
new system and determines whether they have identified more cost-effective
systems for assessing the physical condition of HUD's properties.

Under the new physical inspection system, as in the past, the primary basis
for HUD's physical inspection standards is the statutory requirement for
decent, safe, and sanitary housing. HUD's new standards require that both
multifamily and public housing be decent, safe, sanitary, and in good
repair. The standards also establish more specific requirements for the
major areas of HUD housing, such as the site, dwelling units, and common
areas. While the new standards do not differ substantially from the previous
ones, HUD substantially revised the procedures used to administer the
standards. For example, it developed a detailed list of items that
inspectors are required to review at properties and specifically defined
what constitutes a deficiency for each inspectable item. During the
inspections, the inspectors identify the deficiencies that they observe at a
property and transmit their findings electronically to REAC headquarters. At
REAC, data submitted by the inspectors are checked and analyzed, and each
property's condition is scored using a scale of 0 to 100. HUD believes that
the new system produces more objective assessments than those performed
under previous systems. These assessments provide a basis for targeting
resources to the properties that need the most attention. Moreover, because
the assessment data are centralized, HUD can track the actions taken to
correct the identified deficiencies.

According to REAC's data, about 87 percent of the multifamily properties and
80 percent of the public housing properties reviewed as of April 2000
received scores of at least 60, which HUD considers satisfactory. However,
when REAC quality assurance staff performed on-site follow-up reviews to
assess the adequacy of completed inspections, they often found that the
inspections were not carried out consistently with REAC's requirements. More
specifically, the quality assurance reviewers determined during follow-up
reviews performed in 1999 and the early part of 2000 that about 35 percent
of the inspections did not meet REAC's standards. However, because these
reviews were not performed on a random basis and the effects of the
identified problems on the inspection scores cannot be readily determined,
it is unclear to what extent the problems with the inspections affected the
overall inspection results.

Although REAC deserves credit for establishing quality assurance procedures,
we found gaps or weaknesses in some of these procedures that substantially
limited their effectiveness. In particular, we found that while REAC
performed on-site reviews to assess the adequacy of the inspections, it did
not have procedures for ensuring that these reviews were performed
systematically, that problems identified during the reviews were resolved
quickly and appropriately, and that its reviews were coordinated with those
that its inspection contractors perform as part of their own quality control
programs. For example, we found that, as of March 2000, REAC had not
reviewed the adequacy of the inspections performed by 21 percent of the
active inspectors. Furthermore, REAC did not always report the results of
its reviews to inspection contractors in a timely manner, and it did not
have the systems and records needed to ensure that corrective actions are
taken after problems have been identified. REAC has recently taken a number
of actions to strengthen its quality assurance procedures. For example, it
drafted a plan that sets forth the objectives for and procedures to be used
in its quality assurance program. While this is a positive step, we are
recommending several additional actions to strengthen the quality assurance
plan. HUD agreed with us that it needs to improve its quality assurance
program and pointed out steps it is taking to do so.

Representatives of public housing authorities and multifamily housing
industry groups we interviewed also had concerns about the reliability of
REAC's inspections. For example, they were concerned that the scores may not
necessarily reflect the actual condition of some properties, noting that, in
some instances, there were substantial differences in the inspection scores
received by properties that had undergone more than one inspection. They
were also concerned that HUD may lack the resources needed to ensure the
efficient operation of its appeals procedures. While some housing industry
groups suggested alternatives to the current physical inspection system,
none of the groups we contacted had conducted analyses to compare the cost
and accuracy of these alternatives and HUD's current system.

Nearly 4 million American families live in rental housing--either public
housing or privately owned multifamily housing--that is owned, insured, or
subsidized by HUD. To ensure that these families have housing that is
decent, safe, sanitary, and in good repair, REAC inspects these properties
annually. Public housing serves low-income families, the elderly, and
persons with disabilities and is operated by public housing authorities
using funds provided by HUD. Privately owned multifamily housing includes
properties that receive some form of rental assistance, including Section 8,
from HUD; properties whose mortgages are insured or held by HUD; and
properties that are financed by HUD.1

REAC's new physical inspection system is based on uniform physical condition
standards and a computer-driven physical inspection protocol. The majority
of REAC's inspections are conducted by inspectors employed by one of the
inspection contractors that HUD has hired for this service; some inspections
are, however, performed by HUD personnel. During an inspection, a contract
inspector enters observations into a hand-held computer and then
electronically transmits the data to REAC for verification and calculation.
The computer program generates a score from 0 to 100, which provides a
snapshot of the condition of the inspected property, including any health
and safety violations observed during the inspection.

To establish uniform standards that could be applied to all HUD housing,
regardless of the source of subsidy or assistance, the Department identified
hundreds of conditions or deficiencies that could be observed at any HUD
housing property. HUD grouped these observable deficiencies into five major
inspectable areas:2

ï¿½ the site (including the fencing, grounds, lighting, and parking lots),

ï¿½ the building exterior (including the foundation, doors, roofs, walls, and
windows),

ï¿½ the building systems (including the electrical systems, elevators, heating
and air-conditioning systems, and water and sanitary systems),

ï¿½ the dwelling units (all the elements within a resident's apartment), and

ï¿½ the common areas (including the halls and stairways, community rooms, rest
rooms, utility and mechanical rooms, trash collection areas, and day care
centers).

HUD also established weights and values to quantify the severity of the
identified deficiencies and determine their relative importance. The more
severe the deficiency or the more important the inspectable item with a
deficiency, the more points are deducted for that deficiency. An area score
is then calculated from all the observed deficiencies for an inspectable
area, and the five area scores are averaged and weighted to produce a single
score for the property's overall physical condition. Generally, a property's
score is based on a sample of units and, less often, on a sample of
buildings. Because the score represents the property's condition at the time
of the inspection, it does not take into account the property's long-term
capital needs--such as whether its roof should be replaced in 5 years or its
reserves are sufficient to meet such needs.

The final score and the observed deficiencies are summarized in a report
that REAC sends to the property owner or public housing authority and to the
appropriate HUD program office. HUD requires that corrective actions be
taken for properties that receive inspection scores below certain
thresholds. However, the contract inspector is required to notify the owner
or manager of any exigent ("life-threatening") health and safety violations
at the end of the inspection. These violations must be corrected
immediately, and notice of the correction must be sent to the appropriate
program office. Health and safety violations are also noted by attaching an
alphabetic symbol to the score classifying the violations as
non-life-threatening or life-threatening.3

To ensure the reliability of the inspections, REAC has established four
quality assurance procedures. They include the following:

ï¿½ Desk reviews. REAC staff review the accuracy of the inspection data, such
as the number of buildings at a property and the number of apartment units
selected for inspection. Errors are reported to the contractor for immediate
resolution. Desk reviews also allow REAC to identify trends and patterns
that might suggest weaknesses in the inspection protocol.

ï¿½ Field reviews of selected contract inspections. These reviews may be
collaborative (a REAC staff member accompanies a contract inspector to
observe the inspector's performance) or follow-up (a REAC staff member
visits a property after it has been inspected to assess whether the
inspection meets REAC's requirements).4 Both types of reviews are used to
identify inspectors who are not adhering to the inspection protocols.
Inspection contractors are then advised to take appropriate corrective
action, which may include reinspecting the property, retraining the contract
inspector, or administering disciplinary action.

ï¿½ Contractor quality control programs. REAC requires each inspection
contractor to establish and maintain its own quality control program for
overseeing its inspectors and to report its findings to REAC. Contractors,
for example, perform collaborative reviews that are similar to REAC's
reviews, survey property owners for their views on the inspections, record
complaints about inspectors' performance, and retrain poorly performing
inspectors.

ï¿½ Appeals procedures. HUD provides multifamily property owners and public
housing authorities with two avenues, technical reviews and data
corrections, for disputing inspection scores that they believe are
incorrect. The technical review process allows owners and housing
authorities to report errors committed by contract inspectors that may
result in inaccurate assessments of the condition of their properties. The
data correction process provides a means for HUD to correct inspection data
when local ordinances or building codes have not been factored into the
scoring process. REAC is responsible for processing technical reviews, while
data corrections are processed by HUD's Multifamily Housing and Public and
Indian Housing program offices. As of April 2000, HUD had finalized the
appeals procedures for public housing but not for multifamily housing.

REAC's goal was to obtain baseline data on the physical condition of all HUD
properties by the end of calendar year 1999. REAC completed its baseline
inspections of public housing properties on schedule but did not finish its
baseline inspections of multifamily properties. As of April 12, 2000, REAC
had inspected 26,528 multifamily properties, or 92 percent of the
multifamily properties scheduled to be inspected.

Procedures Are Different

HUD's new physical inspection standards, called uniform physical condition
standards, are based on a statutory requirement that HUD housing be decent,
safe, and sanitary. Although the new standards are similar to HUD's former
physical inspection standards, the procedures for implementing the standards
have been substantially revised to address problems with HUD's prior
inspection procedures.

Like HUD's prior physical inspection standards, the new uniform standards
are based on statutory language that requires HUD housing to be decent,
safe, and sanitary. HUD believes that housing assisted under its programs
should be subject to uniform physical standards, regardless of the source of
the subsidy or assistance. Consequently, rather than altering the statutory
requirement for the maintenance of HUD housing, HUD aimed to describe the
statutory requirement clearly and make that definition consistent across
applicable HUD programs. The Congress established the standard for HUD
housing in the United States Housing Act of 1937 (42 U.S.C. 1437 et seq.),
providing that federally assisted housing for low-income individuals be
decent, safe, and sanitary.5 This act applies to Section 8 assisted housing
and public housing. For HUD-insured multifamily housing, the mortgagors are
required by contract to maintain the mortgaged premises in good repair and
condition--a standard that, HUD says, is very similar if not identical to
the standard for decent, safe, and sanitary housing.

HUD's new uniform physical condition standards include requirements for the
major areas of HUD housing, such as the site, dwelling units, and common
areas. For example, the standards specify that the site--including the
fencing and retaining walls, grounds, play areas, and walkways--must be free
of health and safety hazards and in good repair. In addition, the site must
not be subject to material adverse conditions, such as abandoned vehicles,
excess accumulations of trash, infestations of vermin or rodents, or fire
hazards. Similarly, the standards specify that the property's dwelling units
must be structurally sound and habitable and that all areas and aspects of
the dwelling units must be free of health and safety hazards, functionally
adequate, operable, and in good repair.

According to HUD, the new uniform physical condition standards do not differ
substantially from the housing quality standards that the Department
formerly used for public housing and still uses for dwelling units rented to
tenants with section 8 tenant-based assistance. HUD officials told us that
the two sets of standards cover essentially the same inspectable items
except that the housing quality standards apply only to the dwelling units
while the new uniform standards apply to the entire property. HUD also
considers the uniform standards more specific and less subjective than the
housing quality standards. While the uniform standards require that, in
addition to being decent, safe, and sanitary, HUD housing must be in good
repair, HUD believes that this is the physical condition standard to which
HUD assisted housing has always been subject.

With Prior Inspections

HUD revised its physical inspection procedures and made them uniform in an
effort to address a number of concerns about its former inspection
programs.6 First, HUD required all inspections to be performed by trained
and certified inspectors. In the past, inspections were performed by
inspectors representing several different entities, depending on the HUD
program--Section 8 contract administrators, mortgagees, HUD field offices,
or public housing authorities. According to HUD officials, these inspectors
were not uniformly trained, and different training sometimes produced
inconsistent results.

Second, HUD identified items that must be observed during inspections,
developed detailed definitions for deficiencies in each of them, and
assigned a specific number of points for each deficiency. Now, when a HUD
inspector identifies a particular deficiency, a certain number of points are
automatically deducted from the inspected property's total physical
condition score. In the past, without such specific guidance and detailed
definitions, some inspectors might have focused on some deficiencies but not
on others. Moreover, without a point system for deficiencies, some
inspectors might have determined that a particular deficiency, such as a
cracked toilet seat, was severe enough for a unit to "fail" its inspection,
while another inspector might have attached less importance to the same
deficiency and allowed the unit to "pass." Under the new inspection
procedures, individual inspectors no longer have such broad discretion to
determine whether the dwelling units they inspect meet HUD's standard for
decent, safe, and sanitary housing. According to HUD, the new procedures are
more objective, ensuring more consistent results.

Third, HUD centralized the reporting, scoring, and storing of property
inspection results. After inspectors electronically transmit the inspection
data they have entered into their hand-held computers to REAC, the data are
processed, the properties receive physical inspection scores, and the data
are maintained in centralized databases. Because the new scores are based on
a point system, they are scalable, allowing HUD to assess the breadth or
severity of the deficiencies found. HUD can then use the scores to evaluate
the condition of the inspected properties and to focus its resources on the
properties needing the most attention. Furthermore, since all the data
obtained during the inspections are centralized, HUD can use the data to
track the actions taken to correct deficiencies found during the
inspections. In the past, the pass/fail determinations that HUD received
from some inspection entities did not provide the information needed to
direct resources to the properties with the greatest deficiencies. Moreover,
HUD received no information on the physical condition of public housing
because it relied on public housing authorities to certify their units'
compliance with the standard for decent, safe, and sanitary housing.

Besides revising the physical inspection procedures, HUD established four
quality assurance procedures to help ensure the reliability of its
contractors' inspections. Previously, HUD had no comprehensive, consistent
quality assurance procedures.

Condition, REAC's Reviews Indicate Some Inspections Were Not Reliable

The results of the first round of physical inspections performed under the
new inspection system indicate that most of HUD's public and multifamily
housing is in satisfactory physical condition. However, REAC's quality
assurance reviews have found that a number of these inspections were not
carried out consistently with REAC's requirements. It is unclear to what
extent the problems with the inspections affected the overall inspection
results.

Physical Condition

As of April 12, 2000, REAC had conducted initial, or "baseline," inspections
for 26,528 multifamily properties and over 3,100 public housing authorities
with 13,607 properties. These figures represent about 92 percent of the
multifamily properties and all of the public housing properties that REAC
intended to include in the baseline inspections.

As shown in figure 1, almost 34,000 properties, or 85 percent of those
inspected, received scores of 60 or higher. A score of 60 indicates that a
property is in satisfactory physical condition. Properties receiving
inspection scores of 60 or higher accounted for just under 1.9 million
units, or about 74 percent of the 2.6 million units receiving assistance in
the properties inspected.

Overall, the inspection scores for multifamily properties were slightly
higher than the scores for public housing properties, but for both types of
properties, a significant percentage of the scores were 60 or higher.7 For
example, about 87 percent of the multifamily properties and 80 percent of
the public housing properties received scores of 60 or higher. The
properties receiving scores of 60 or higher accounted for 84 percent of the
multifamily units and 64 percent of the public housing units in the
properties that were inspected.8

A substantial number of properties received high inspection scores. More
specifically, about 38 percent of the multifamily properties and 25 percent
of the public housing properties received scores of 90 or higher. By
contrast, only about 2 percent of the multifamily and about 3 percent of the
public housing properties received scores of 30 or lower, an indication that
these properties may be in severe physical distress.

Despite the high proportion of satisfactory scores, a substantial number of
properties were cited during the inspections for exigent health and safety
problems. For example, almost one-third of the multifamily and public
housing properties received health and safety citations for defects REAC
categorizes as "life-threatening." Examples of such defects, according to
REAC documents, include missing or inoperable smoke detectors, electrical
system problems, hazards, missing fire extinguishers, and the presence of
security bars. Appendix I provides additional information on the results of
REAC's baseline inspections.

Results Is Unclear

The results of reviews REAC performs as part of its quality assurance
program show that, while the number of inspections that REAC rejects during
desk reviews has declined substantially, REAC has found problems with a
substantial number of the inspections for which it has performed follow-up
reviews.

During desk reviews, REAC staff members review whether contract inspectors
have made errors in such things as the number of buildings at a property and
the number of apartment units selected for inspection. If errors exist, the
inspection report is returned to the inspection contractor for resolution
within 48 hours. If the contractor does not resolve the problems within 48
hours, the inspection is rejected. According to REAC, about 10 percent of
the inspections submitted to REAC in March 1999, when REAC began tracking
rejection rates, were rejected. But by April 2000, the percentage of
inspections rejected during REAC desk reviews had decreased to less than 1
percent.

However, from February 1999 through the early part of 2000, REAC quality
assurance reviewers continued to find problems with inspections when
performing collaborative and follow-up reviews. During the collaborative
reviews, quality assurance reviewers accompany contract inspectors to judge
their performance. Under follow-up reviews, reviewers test the accuracy of
completed inspections by examining building exteriors, common areas,
mechanical and electrical systems, and site conditions, such as parking
areas and fencing. They then make a judgmental determination as to whether
the original inspection met REAC's standards.9

As of March 15, 2000, REAC quality assurance reviewers determined that about
12 percent of the 728 initial inspections they evaluated during
collaborative reviews did not meet REAC's standards. Furthermore, REAC
reviewers assessed 819 more initial inspections during follow-up reviews.
About 35 percent of those 819 inspections did not meet REAC's standards.
Among the problems found during such reviews were failures of contract
inspectors to record physical deficiencies, failures to inspect all
buildings, and omissions of health and safety violations. Furthermore, the
results of these reviews, unlike the results of desk reviews, have not
improved during the first year. Between March 1999 and March 2000, REAC
quality assurance staff found little change in the percentage of inspections
that met REAC's standards.

As part of our review, we accompanied REAC quality assurance inspectors on
10 follow-up reviews. In each of the 10 cases, the REAC quality assurance
reviewer determined that the original inspection did not meet REAC's
standards and required that a new inspection be performed. Among the
problems that the quality assurance inspectors found were adverse site
conditions that were not recorded, areas that were not included during
inspections, and health and safety violations that were not reported. The
photographs in figure 2 illustrate some of the defects in property
conditions that contract inspectors did not document during their
inspections of four of the properties we visited.

While the results of REAC's quality assurance reviews call into question the
adequacy of some of the inspections performed, it is not clear how the
problems identified affect the overall results of the inspections. Because
REAC performs quality assurance reviews on a targeted rather than a random
basis, it is difficult to project the results of these reviews to all of the
inspections. Furthermore, REAC has not performed any analyses to assess how
the problems identified by the quality assurance inspectors affected the
inspection scores. REAC quality assurance reviewers do not rescore the
condition of properties when performing their reviews, but in some instances
they do require the contract inspectors to reinspect the properties. The
reinspection then generates a new inspection score.

While REAC officials agreed that the percentage of inspections identified as
not meeting REAC's standards was unacceptable, they indicated that a precise
tolerance level has not yet been established as a guide in evaluating the
reliability of the inspections. In addition, they noted that because the
classification of inspections into one of two categories was based on the
reviewers' judgments rather than specific criteria, the results of the
reviews might not be as accurate, consistent, or reliable as they should be.
REAC officials told us that they were developing more specific criteria for
reviewers to use in assessing the inspection results.

REAC officials also emphasized that while field reviews are an important
part of the quality assurance process, their results must be considered in
conjunction with other relevant quality assurance information. They did not
believe that the results of the field reviews alone meant that a substantial
portion of the inspections performed were substandard. Furthermore, REAC
officials noted that REAC was in the process of testing changes that have
been made in the inspection protocol. These changes include revisions aimed
at clarifying the definitions for a number of deficiencies. They believe
that after the new inspection protocol is implemented, there will be fewer
problems with the quality of inspections.

Measures, but REAC Is Making Improvements

REAC deserves credit for establishing quality assurance procedures to help
ensure the reliability of its contract inspections, but our review
identified certain weaknesses that substantially limited the effectiveness
of some of these procedures. More specifically, although REAC has
established a well-defined process for performing desk reviews of contract
inspections, we found problems with its processes for performing field
quality assurance reviews of the inspections and for reviewing contractors'
quality control programs. During our review, REAC began making improvements
to address a number of these weaknesses. For example, it drafted an overall
plan for carrying out its quality assurance program. However, REAC has been
slow to develop formal processes for multifamily housing owners and public
housing authority managers to request changes in their property inspection
scores.

Overseeing Contractors Have Weaknesses

REAC's desk review process appears to be a well-defined and thorough quality
control mechanism for determining whether contract inspectors are
appropriately entering data into their hand held computers and complying
with other established protocols at the time they perform their inspections.
In addition to tracking each inspector's error rate, REAC uses the
information to teach the inspection contractors about the types of data
deficiencies that can lead to rejected inspections. For example, REAC
provides contractors with a frequently updated report that shows, by
inspection standard, the types of deficiencies that could result if a
standard is not followed and the action that REAC will take if a requirement
is not met. It also uses the desk reviews to identify trends and patterns
that might suggest a weakness in a particular component of the inspection
protocol. REAC believes that these efforts have helped to lower the
rejection rates for these reviews.

In addition to desk reviews, REAC conducts on-site collaborative and
follow-up reviews to determine whether contract inspections have been
performed in accordance with its standards and protocols. However, we found
that REAC had not established systematic, clearly defined processes for
carrying out these types of reviews. The following illustrate some of the
weaknesses in REAC's quality assurance processes:

ï¿½ REAC's business operating plan for quality assurance did not include
written policies and procedures for REAC's overall quality assurance
program, nor did it explain how REAC's quality assurance program relates to
or interfaces with those of the four inspection contractors.

ï¿½ REAC did not have a systematic process for planning its quality assurance
reviews nor did it have criteria that effectively defined which inspectors
or inspections would be reviewed. For example, REAC set out to review 4
percent of its contract inspections through collaborative and follow-up
reviews, but by the end of 1999, only 47 percent of the inspection force had
received reviews. Accordingly, REAC accelerated its reviews, and by March
2000, it had increased the percentage of inspectors reviewed to 79 percent.
As of that date, it had completed 1,543 reviews, thereby approximating the 4
percent target, but 56 percent of the reviews were associated with just 53
of the more than 300 inspectors that had performed inspections as of that
date.

ï¿½ REAC did not have strict criteria on when collaborative and follow-up
quality assurance reviews were to take place. For example, for 374 public
housing authority follow-up reviews, as of March 2000, the time between the
contractor's original inspection and REAC's follow-up review ranged from 1
to 394 days, with an average elapsed time of 96 days. Several contractors
were concerned that property conditions could change between the date of the
original inspection and the date of the follow-up review, thus limiting the
ability of quality assurance reviewers to determine and evaluate conditions
that existed at the time of original inspection. One inspection contractor
observed that since REAC had waited so long to point out the inspection
problems, it was sometimes difficult to discuss performance issues credibly
with inspectors.

ï¿½ REAC did not maintain the records necessary to document actions taken by
inspection contractors to address problems identified during the reviews.
For example, REAC was often unable to tell us when the results of its
reviews were referred to contractors and when contractors responded with
corrective actions. Furthermore, when documentation was available, there
were often delays in communicating the results of reviews to the inspection
contractors. For example, in mid January 2000, REAC gave contractors
documentation on 75 inspections that did not meet REAC's standards. At least
47 of the quality assurance reviews had taken place before December 1999,
and several dated back to June 1999.

ï¿½ REAC's inspection contractors also raised concerns that they were not
being advised of quality assurance reviews that did meet REAC's standards so
that these results could be considered in the administration of their own
quality control programs.

ï¿½ REAC did not have written criteria outlining specific disciplinary actions
that should be taken against poorly performing inspectors, nor did it have a
monitoring system to record and track disciplinary actions.

We also found weaknesses in REAC's oversight of the quality control programs
that REAC required its inspection contractors to establish. The following
are illustrative:

ï¿½ Although REAC received and reviewed the written quality control plans of
its contractors, it did not perform on-site evaluations to assess the
effectiveness of the contractors' quality control programs during 1998 or
1999. HUD's contract management policies recommend a thorough assessment of
contractors' quality control programs to ensure the adequacy of quality
control systems at the contractor level. These assessments can also be used
to determine the appropriate balance between HUD's quality assurance
programs and those of the contractors.

ï¿½ While REAC's inspection contracts require contractors to submit monthly
reports describing the results of their quality control programs, REAC has
not always received the reports and did not have a systematic process for
analyzing them and using them to monitor contractors' performance when they
were received. Furthermore, REAC did not factor in the results of
contractors' quality control programs when developing its own plans for
carrying out quality assurance reviews. Such coordination could help REAC
better target its resources.

ï¿½ REAC limited its inspection contractors to performing only one type of
review--collaborative reviews. Several contractors have informed REAC that
their quality control programs could be improved if they had the authority
to perform follow-up reviews as well. For example, they did not believe that
the collaborative reviews always give a true indication of the quality of an
inspector's work because most inspectors are usually more conscientious when
they are being observed during an inspection.

Assurance Processes

During the course of our audit, REAC officials began making a number of
changes to address weaknesses we identified. REAC officials noted that their
quality assurance goal is one of continuous improvement and zero defects in
inspectors' performance. They acknowledged that the quality assurance
process itself was not fully developed when the new inspection system began
in 1998. Rather, as they noted, the process has evolved and is continuing to
evolve over time through experience, testing, and analysis.

Among the steps REAC took during our audit to improve its quality assurance
processes were the following:

ï¿½ To strengthen its planning, REAC began, in January 2000, to compare the
number of quality assurance reviews with the total number of inspections
performed by each contract inspector so that inspectors could be targeted
for reviews.

ï¿½ To improve the objectivity, consistency, and credibility of its quality
assurance reviews, REAC was developing definitive criteria for rating the
quality of contract inspections, and it set a policy goal of performing
follow-up reviews within 30 days of the original contract inspections.

ï¿½ Also in January 2000, REAC began conducting on-site evaluations of
contractor's quality control programs and started reporting the results of
every quality assurance review to the contractors, including those reviews
that meet REAC's standards.

ï¿½ REAC allowed one of the four inspection contractors to expand its quality
control program by performing follow-up reviews, as well as the
collaborative reviews it was already performing.

ï¿½ To improve its processes for taking disciplinary action, REAC began
developing criteria for taking specific actions against contract inspectors.
It also started to establish a system to track deficiencies for individual
inspectors and the associated disciplinary actions taken.

More recently, in April 2000, REAC officials told us that they had drafted
an overall plan for carrying out REAC's quality assurance program. The
quality assurance plan is aimed at establishing procedures in four areas
that would allow REAC to evaluate the (1) inspection contractors' compliance
with provisions in their contracts and their quality control programs, (2)
inspectors' performance in applying REAC's inspection protocol, (3) accuracy
of the inspections and the resulting scores, and (4) performance of the
physical inspection program as indicated by the precision and replicability
of the inspection protocol. REAC officials told us that the plan's
development was based on both their own plans to develop a more formalized
quality assurance program and the suggestions we made during our review.

REAC's establishment of an overall plan for its quality assurance activities
is a positive step toward providing the tools needed to ensure the
reliability of its contract inspections. But because the plan is still in
draft and subject to change, it is difficult to fully assess its merits.
According to our preliminary review of the plan, it is a good start in that
it establishes a framework for REAC's quality assurance activities.
Nevertheless, we are concerned that some of the procedures in the plan may
not allow REAC to achieve the plan's four overall objectives.

First, the plan does not specify how REAC will use the information collected
through various quality assurance procedures to measure its performance
against each of the four objectives. For example, the plan directs REAC,
when assessing the performance of the physical inspection system as
indicated by the precision and replicability of the inspection protocol, to
measure and control the level of variance resulting from the application of
the protocol. However, the three procedures that the plan proposes for
accomplishing this task are vague. Specifically, they call for REAC to
identify variances among inspectors, identify variances caused by
inconsistencies in inspectors' performance, and develop risk indicators and
communicate them to appropriate levels for implementation. It is unclear
precisely what data REAC will be collecting or how the data will be used to
measure the inspection protocol's precision and replicability. Furthermore,
the plan does not discuss how REAC will report to HUD management and the
Congress on its performance in meeting each of the four objectives.

Second, although the plan recognizes the importance of the inspection
contractors' compliance with their respective quality control programs, it
does not explain precisely how REAC's quality assurance objectives and
procedures interface with those of the contractors. For example, the plan
directs REAC to review the contractors' quality control activities and
requires the contractors to report regularly on their programs, but it does
not specify how REAC will use the information from these reviews and reports
to target its own quality assurance activities, such as its collaborative
and follow-up reviews.

Third, we have some questions about the criteria REAC is using to assess its
performance in meeting the plan's four overall objectives. For example,
REAC's criterion for assessing inspectors' performance during collaborative
reviews allows an error rate as high as 20 percent in certain areas before
an inspector's performance is classified as not meeting REAC standards. Such
a high error rate appears inconsistent with REAC's goal of zero defects in
inspectors' performance. Furthermore, the plan does not clearly indicate
what criteria REAC will use to assess inspectors' performance when it
conducts follow-up reviews. In addition, although the plan includes steps
for reviewing REAC's processing of property owners' and public housing
authorities' concerns about their inspection scores, it does not specify
what criteria REAC will use to determine whether these reviews have been
handled appropriately and in a timely manner.

Finally, although the plan indicates how contractors should document
inspectors' performance for their quality control programs, it does not
specify what records REAC should maintain to document its own actions when
it identifies poorly performing inspectors during collaborative and
follow-up reviews.

Requesting Changes in Inspection Scores

In January 2000, after nearly a year and a half, HUD finalized regulations
for providing public housing authorities with a means to request changes or
adjustments in their inspection scores. As of April 2000, HUD had not yet
finalized such regulations for multifamily property owners. HUD had been
using interim guidelines to address their concerns. However, even with the
direction provided by these interim guidelines, HUD was not able to address
many public housing authorities' and multifamily property owners' concerns,
and requests from both remain pending, including several that date back to
May and June 1999.

As discussed in the background section of this letter, HUD has established
two procedures for public housing authorities and multifamily property
owners to appeal property inspection scores they believe are incorrect.
Specifically, HUD uses (1) technical reviews to identify errors by contract
inspectors and (2) data correction reviews to factor in local ordinances and
building codes that may not have been considered, or may have been
considered inappropriately, during inspections. For example, the owner or
manager of a property may request a data correction review when REAC's
inspection protocol--applicable nationwide--is not entirely consistent with
a local code requirement or when an owner is incorrectly held responsible
for the condition of items (such as roads and sidewalks) that the owner does
not own and is not responsible for maintaining. Although HUD considers these
procedures critical to the REAC inspection system's credibility and
acceptance within the housing industry, the Department had not established
the procedures when it implemented the inspection system in October 1998.
Instead, Office of Housing and REAC officials explained, HUD's first
priority was to get the inspection system up and running and then to
establish procedures for technical and data correction reviews. As a result,
HUD only recently finalized applicable regulations for public housing
authorities, and, as of April 2000, had not yet finalized such regulations
for multifamily property owners. Consequently, it is too soon for us to
assess the effectiveness of these procedures.

As of April 2000, HUD also lacked guidelines for processing data correction
requests from public housing authorities, although it did have interim
guidelines for processing such requests from multifamily property owners.
REAC also developed interim guidelines for processing technical review
requests from both multifamily property owners and public housing
authorities, but as of April 2000, the guidelines were still in draft
format. According to the draft guidelines, the overall goal of the technical
review process is to provide owners and public housing authorities with a
quick, fair, and consistent assessment of their appeals. Moreover, as an
official from HUD's Office of Housing noted, timely responses to the
concerns of owners and public housing authorities are important because they
help HUD confirm the existence of problems with inspection scores and
determine the need for correction. However, according to HUD's data on
requests from property owners and public housing authorities for technical
and data correction reviews, HUD was often slow in resolving cases under the
interim guidelines. For example, as of April 2000, HUD had received about
400 technical review requests, over half of which were still pending.
Although most of the pending requests were submitted during March and April
2000, several dated back to May and June 1999. Furthermore, of the requests
HUD had resolved, the Department had taken 3 months, on average, to reach a
decision, and had determined, for over half, that changes in the inspection
score were warranted.10 When we asked REAC officials why cases were not
always processed in a timely manner, they cited delays in the rulings for
establishing the review processes and insufficient staff to process the
cases. In addition, one REAC official said that some cases are more complex
and take longer to process. For example, a request from one public housing
authority involved 10 different properties and took REAC nearly 4 months to
complete. REAC officials told us that they recognize the need to reduce
processing times. They said that as REAC gains additional experience in
processing requests for technical reviews and appeals, they anticipate that
the time for completing them will decrease.

Inspections and Other Issues

Although HUD has consulted both the multifamily and public housing
industries during the development and revision of the new physical
inspection process, officials from both industries have periodically
expressed concern about the reliability of the inspections and other aspects
of the inspection process. In response to these concerns, HUD officials have
emphasized that the physical inspection process is relatively new and still
evolving and that HUD intends to continue working with the industry to
improve it. Some industry officials have suggested possible alternatives to
the current system, but none of the groups we contacted had analyzed the
costs or accuracy of these alternatives. The National Academy of Public
Administration is currently reviewing alternative performance measurement
systems for subsidized housing that would include a physical inspection
component.

About Aspects of the Physical Inspection System

During the 2 years that it has been developing and refining its new physical
inspection system, HUD has met with representatives of the multifamily and
public housing industries to obtain their views on the system's design and
their suggestions for improving certain parts of it. However, officials from
both industries still have concerns about aspects of the system, which they
believe is not yet providing a true picture of the condition of some
properties. Some of their concerns include the following:

ï¿½ Some of the properties that have been inspected more than once have
received significantly different scores, even though the condition of the
properties has remained more or less the same.

ï¿½ HUD is not adequately testing changes that it had made in its inspection
software to incorporate revised definitions for a substantial number of
deficiencies.

ï¿½ The training and skills of some contract inspectors need to be improved.

ï¿½ HUD is not giving property owners and managers enough time to prepare and
submit documentation for technical reviews, database corrections, and
appeals; these processes are complex and can lead to confusion; and HUD may
not have sufficient resources at its field offices to implement the
procedures efficiently and effectively.

In addition, representatives of some public housing organizations have a
number of other concerns. For example, they said the scoring system
considers an excessive number of deficiencies, including some that are so
minor they should not be considered; too many points are deducted for some
items relative to their importance; defects found in a very small percentage
of the items inspected could result in a property's receiving a failing
score; and HUD imposes different oversight requirements on public housing
properties and multifamily properties that receive comparable inspection
scores.

In response to these concerns, HUD officials said they are committed to
developing a physical inspection system that accurately portrays the
condition of HUD's subsidized and assisted properties. At the same time,
they recognize that developing such a system will be an evolutionary
process. To ensure that the system accurately portrays the condition of HUD
housing, officials said that HUD has consulted with representatives from
both industries, will continue to do so, and will make appropriate
revisions. They also cited actions HUD was taking to address the industries'
concerns as the inspection process was evolving, such as considering
requests for technical reviews even if they are submitted after HUD's
deadline for filing them. Concerns expressed by the housing industry
representatives and HUD's responses to them are discussed in greater detail
in appendix II.

the National Academy of Public Administration Is Conducting a Review

While some of the housing industry officials we spoke with suggested
possible alternatives to HUD's current system, they had not conducted
detailed analyses of these systems. Among the alternatives they suggested
were (1) returning to HUD's prior system but doing a better job of verifying
the self-certified physical condition information reported by public housing
authorities; (2) turning the current inspection system over to the
owners/managers of the properties, letting the owners/managers conduct their
own inspections using the computer inspection instruments approved by HUD,
and having HUD verify the results on a random basis; (3) establishing a
physical inspection system based on local codes under which housing agencies
or owners would initiate inspections of their properties using inspectors
selected from HUD's list of certified inspectors; and (4) using a peer
review model that would rely on standards developed, and agreed to, by
members of the housing industry and would have evaluations of a property's
physical condition conducted by a commission or board comprised of experts
and practitioners in the field. However, the industry groups did not have
detailed information on how these systems would work, nor did they have
information for comparing these systems with HUD's current system in terms
of cost or accuracy.

The National Academy of Public Administration is currently assessing the
effectiveness of various methods of evaluating public housing authorities
and other providers of federally assisted housing, as required under section
563 of the Quality Housing and Work Responsibility Act of 1999 (P. L.
105-276). The act specified two primary purposes for the study. First, the
study is to identify and examine various methods of evaluating and improving
the performance of public housing authorities in administering public
housing and tenant-based rental assistance programs and of other providers
of federally assisted housing that offer alternatives to oversight by HUD.
Second, the report is to evaluate whether specific monitoring and oversight
activities currently conducted by HUD should be eliminated, expanded,
modified, or transferred to other entities to increase their accuracy and
effectiveness and to improve monitoring. According to Academy officials, the
study will include an assessment of HUD's physical inspection portion of
performance measurement. The Academy is to provide a final report on its
findings and recommendations by September 2000.

The establishment of a new physical inspection system is a positive step by
HUD to address weaknesses in its oversight of multifamily and public housing
properties. In particular, the establishment of uniform standards and
inspection procedures helps to address inconsistencies in the way standards
were applied to HUD properties and in the way physical inspections were
performed. Equally important, the establishment of centralized databases for
collecting information on the physical condition of these properties
provides HUD not only with detailed, readily available data but also with a
mechanism for (1) ensuring that deficiencies identified during inspections
are corrected and (2) helping HUD take appropriate action against property
owners and housing authorities that fail to provide housing that is decent,
safe, sanitary, and in good repair.

However, there are continuing concerns about the extent to which inspections
performed under the new system are reliable--concerns that are substantiated
by the results of HUD's own quality assurance reviews. These concerns
increase the importance of HUD's having a system of quality assurance
procedures that will allow it to assess and report on the reliability of its
inspections and take corrective actions when problems are found. Such a
system includes having effective processes for addressing multifamily
property owners' and public housing authorities' concerns about the results
of inspection scores.

REAC has recently taken a number of actions to strengthen its quality
assurance procedures. These include recently preparing a plan that sets
forth the objectives for and procedures to be used in its quality assurance
program. In our view, this is a positive step toward providing HUD
management, the Congress, and others with information for assessing REAC's
progress in achieving key objectives, such as ensuring that inspection
contractors are complying with the terms of their contracts, inspectors are
performing inspections consistently with REAC's inspection protocol, and
inspection scores accurately reflect the condition of properties being
assessed. Nevertheless, we are concerned that, in some areas, the draft plan
may not provide REAC with the information it needs to assess its progress in
meeting these objectives.

We recommend that the Secretary of Housing and Urban Development direct the
Director of HUD's Real Estate Assessment Center to revise REAC's April 2000
quality assurance plan as necessary to ensure that the quality assurance
activities it contains will provide REAC with the information it needs to
evaluate four key areas--inspection contractors' compliance with provisions
in their contracts and quality control programs; inspectors' performance in
applying REAC's inspection protocol; the accuracy of the inspections and
resulting scores; and the performance of the physical inspection program as
indicated by the precision and replicability of the inspection protocol. The
revisions should include adding information to the plan that describes (1)
how the information obtained through various quality assurance procedures
will be used to assess REAC's performance in meeting each of the plan's
objectives, (2) how REAC's quality assurance activities relate to activities
performed by the inspection contractors as part of their quality control
programs, and (3) what records REAC should maintain to document its actions
when poorly performing inspectors are identified during collaborative and
follow-up reviews. In addition, REAC should reevaluate whether the plan
contains appropriate criteria for quality assurance personnel to use in
assessing each of the activities covered by the plan.

Finally, to provide the Congress with timely information on REAC's progress
in addressing concerns about the reliability of physical inspections, we
recommend that HUD periodically issue reports describing the quality
assurance activities that it has performed and the results of these
activities.

We provided HUD with a draft of this report for its review and comment. HUD
said while its staff and management had themselves identified many of the
items we pointed out during the course of our audit, we helped the
Department focus on specific areas and suggested improvements that further
strengthened the inspection protocol. HUD stated that the inspection
protocol has enabled the Department to determine the condition of its entire
property inventory, which represents a major modification and improvement
over what existed previously. HUD agreed with us that the Department needs
to improve its quality assurance program but pointed out that in the past no
quality assurance program, consistent standards, or reliable inspection
process existed at HUD.

In response to our recommendation that REAC revise its April 2000 quality
assurance plan as necessary to ensure that the quality assurance activities
it contains provide REAC with the information needed to assess its
performance in achieving the plan's four key objectives, HUD pointed out
aspects of the plan that are aimed at addressing our recommendation and
noted some modifications that it had made to the plan. HUD also provided us
with an updated version of its quality assurance plan. While this version is
an improvement over the previous plan, it does not fully address our
recommendation. For example, although the plan includes some indicators for
assessing REAC's performance in meeting the plan's objectives, it still does
not contain indicators for some key quality assurance activities, such as
follow-up reviews for assessing inspectors' performance and activities aimed
at assessing the inspection system's precision and replicability. REAC
officials told us that they were still determining what indicators they will
use to assess the results of quality assurance activities, and they said
they were planning to develop additional indicators. REAC officials also
told us that although the draft plan contains requirements for reviewing and
assessing the results of contractors' quality control programs, they
recognize that they need to improve coordination between contractors'
quality control activities and REAC's quality assurance activities. They
said they are considering revisions to the requirements for contractors'
quality control programs that would more clearly integrate these
requirements with REAC's overall quality assurance program.

In response to our recommendation that HUD periodically issue reports
describing the quality assurance activities that REAC has performed and the
results of these activities, HUD said that it plans to issue a semi-annual
report summarizing the results of REAC's quality assurance activities. REAC
officials told us that they were still deciding what information to include
in the report. The complete text of HUD's comments and our evaluation of
them appear in appendix IV.

We conducted our work from August 1999 through June 2000 in accordance with
generally accepted government auditing standards. (See app. III for a
discussion of our objectives, scope, and methodology).

As arranged with your offices, unless you publicly announce its contents
earlier, we plan no further distribution of this report until 30 days after
the date of this letter. At that time, we will send copies to the Honorable
Andrew M. Cuomo, Secretary of Housing and Urban Development. We will make
copies available to others on request.

If you or your staff have any questions about this report, please call me at
(202) 512-7631. Key contributors to this report are listed in appendix V.

Stanley J. Czerwinski
Associate Director, Housing and
Community Development Issues

List of Requesters

The Honorable Rick A. Lazio
Chairman
The Honorable Barney Frank
Ranking Minority Member
Subcommittee on Housing and
Community Opportunity
Committee on Banking and
Financial Services

The Honorable Christopher S. Bond
Chairman
Subcommittee on VA, HUD and
Independent Agencies
Committee on Appropriations
U.S. Senate

Results of Baseline Inspections Conducted by HUD's Real Estate Assessment
Center

As of April 12, 2000, HUD's Real Estate Assessment Center (REAC) had
completed baseline inspections of 26,528 multifamily properties and 13,607
public housing properties at over 3,100 public housing authorities. These
totals represent about 92 percent of the multifamily properties and all of
the public housing properties that REAC intended to inspect in order to
gather baseline data.

Figure 3 provides information on the percentage of multifamily and public
housing properties whose physical inspection scores fell into one of four
scoring ranges. As the figure shows, a large percentage of the inspection
scores were 60 or above. According to HUD, a score of 60 or above generally
indicates that a property is in satisfactory physical condition.
Furthermore, 38 percent of the multifamily properties and 25 percent of the
public housing properties received a score of 90 or higher. In contrast,
only about 2 percent of the multifamily properties and 3 percent of the
public housing properties received scores of 30 or below. Such scores
indicate that there may be substantial problems with the physical condition
of these properties.

Figure 4 summarizes the percentage of multifamily and public housing units
associated with the properties whose physical inspection scores fall into
one of the four scoring ranges. About 84 percent of the multifamily units
and about 64 percent of the public housing units were in properties with
inspection scores of 60 or higher. About 2 percent of the multifamily units,
and about 9 percent of the public housing units, were in properties with
inspection scores of 30 or below.

Our analysis indicates that the baseline physical inspection scores vary
according to location. Table 1 provides information on the total number of
properties inspected, the number of assisted units in these properties, and
the average inspection score for all property inspections within each of the
four regions of the country. Properties in the Northeast and Midwest regions
received lower average inspection scores than properties located in the
South and West. REAC officials cited several factors that may have
contributed to these regional variations, including the facts that
properties in the Northeast and Midwest tend to be older and weather
conditions in those regions are generally harsher. Nevertheless, REAC
officials noted that the United States Housing Act of 1937 requires that all
HUD-assisted facilities, including both assisted and public housing, be in
decent, safe, and sanitary condition and in a good state of repair.

                                  Average
                       Total      inspection
 Region    Total       assisted   score
           properties
                       units
 Northeast 7,388       698,843    74.95
 Midwest   10,694      621,052    77.78
 South     14,644      855,789    79.60
 West      6,664       299,319    81.25
 Other     581         83,261     70.42
 Total     39,971      2,558,264  78.40

Note: "Other" refers to properties in Puerto Rico and the Virgin Islands.
The average inspection score when these properties are excluded is 78.51.

Our analysis also indicates that physical inspection scores vary according
to the type of financing and assistance received from HUD. Table 2 provides
information on the total number of multifamily properties inspected, the
number of assisted units in these properties, and the average inspection
score for all property inspections according to the type of assistance HUD
provides. Properties that receive funding through HUD's 202/811 programs
received the highest average inspection score. The 202/811 programs are
aimed at expanding the supply of housing and supportive services for the
elderly and handicapped. Properties whose mortgages are "held" by HUD
received the lowest average scores. HUD assumed responsibility for these
properties after they became financially troubled, and their financial
difficulties may have had an impact on their physical condition.

                                             Total
 Type of        Assisted or                  assisted      Average
 financing      unassisted       Total       units         inspection score
                                 properties

 202/811        Assisted         5,956       250,408       86.77
 202/811        Unassisted       57          0             92.61
 HUD-held       Assisted         683         52,733        68.60
 HUD-held       Unassisted       87          0             72.62
 FHA-insured    Assisted         8,448       643,246       77.11
 FHA-insured    Unassisted       4,629       0             81.30
 Noninsured     Assisted         6,504       377,963       78.99
 Total          Not applicable   26,364      1,324,350     Not applicable

Industries' Concerns and HUD's Responses

HUD consulted representatives of the multifamily and public housing
industries during both the development of its physical inspection system and
recent revisions to it. Nevertheless, industry officials whom we spoke with
still had concerns about the system. In general, these officials were
concerned that the inspection results do not consistently provide accurate
information on individual properties. They were also concerned that the two
processes that allow owners and managers to appeal their physical inspection
scores--technical reviews and database corrections--are too complex and do
not allow enough time for housing authorities and owners to prepare appeals.
In addition, public housing industry officials were concerned about
differences in HUD's treatment of public housing properties and multifamily
properties that receive comparable inspection scores. In response to these
concerns, HUD officials noted that they had been working with housing
industry representatives to address their concerns and would continue to do
so.

Properties' True Physical Condition

Multifamily and public housing industry officials told us that REAC's
inspection protocol and scoring algorithm do not consistently produce
inspection scores that accurately reflect the physical condition of the
properties being assessed. They said they were aware of several instances in
which a property's inspection score changed substantially from the initial
inspection to a more recent one. For example, they noted that one property
received a score of 85 in 1999 and a score of 42 in 2000. The officials
believed that so much change in the score from one year to the next, when
there was little change in the condition of the property, calls into
question the reliability of the inspection system.

In addition, officials from both industries believed that the number of
points properties lose for some deficiencies is out of proportion to their
importance because of the weight that HUD assigns to the deficiencies. For
example, one public housing group identified a number of deficiencies with
point deductions that it considered inappropriate. These included fences and
retaining walls that showed signs of deterioration but still served their
intended functions, minimal accumulations of water in parking lots and
driveways, and play equipment that does not operate correctly but does not
pose a safety risk. As a result, some officials believed that the system was
not fair and generated some scores that were lower than they should have
been.

HUD officials said they are aware of significant discrepancies between
recent and first-year scores for some properties. The officials told us they
are reviewing variations in property scores and will develop remedial
alternatives as warranted. In addition, HUD officials said that pending
revisions to the inspection software should make inspection results more
accurate. As to the industries' concerns about the number of points
associated with some deficiencies, HUD officials believed that at least some
have been addressed by changes that HUD recently made to define a large
number of deficiencies more specifically. However, the officials said that
if concerns persist after the new inspection protocol incorporating the
changes in definitions has been tested and implemented, they will consider
looking at the weights assigned to certain deficiencies.

Software

In response to the multifamily and public housing industries' concerns, in
December 1999, HUD made changes to the definitions of a large number of
potential deficiencies that inspectors use in assessing properties' physical
condition. Although HUD has begun testing the revised software that
incorporates these changes, industry officials believed that HUD should
review its testing procedures. The officials believed that HUD should use a
statistically valid sampling methodology for selecting test sites and that
HUD should conduct complete inspections, using both the revised and old
software simultaneously, rather than looking only at the test results for
deficiencies with revised definitions. After completing the test
inspections, they said, HUD should compare the results to determine how the
revised definitions affected the properties' scores and whether the revised
software produces more accurate results.

According to HUD officials, HUD has modified its testing procedures twice to
address the industries' concerns about its selection of test sites and
testing methodology. Initially, HUD conducted tests targeted to deficiencies
with revised definitions, but later it modified its procedures to test the
complete protocol at six public housing authorities. In all, HUD tested the
software at 23 developments in 15 cities. At the 17 properties where HUD
conducted tests using the complete protocol, the scores for 14 increased by
an average of about 5 points when the new software was used.

Both multifamily and public housing industries said that significant
variations in the physical inspection scores received by some properties
show that certain contract inspectors need additional training in how to
implement REAC's inspection protocol. Industry officials also believed that
variations in the scores might mean that HUD is not providing enough on-site
training for the contract inspectors. The officials further suggested that
turnover among inspectors might be negatively affecting the quality of
inspections.

In response to these concerns, HUD officials maintained that the
Department's requirements for inspectors are sufficient to ensure that they
are qualified. The officials noted that the contract inspectors must have a
minimum of 3 years of experience in the building trades and, to be
certified, must pass tests that ensure they can appropriately characterize
deficiencies. While acknowledging that there has been some turnover among
inspectors, HUD officials said the inspection contractors have not reported
problems in hiring replacement inspectors. They also noted that when HUD has
found an inspector's performance unacceptable, it has revoked the
inspector's certification to perform HUD inspections. Finally, they informed
us that HUD will retrain and recertify the entire inspector workforce in the
course of implementing the latest version of its inspection software.

Public Housing Properties

Public housing officials are concerned that public housing properties and
multifamily properties with comparable inspection scores receive different
treatment from HUD. For example, they said HUD's regulations designate
public housing properties that receive physical inspection scores of 60 or
lower as "troubled," whereas multifamily properties do not receive a
comparable designation. They also noted that public housing properties
receiving overall scores of from 60 through 69 points under HUD's Public
Housing Assessment System (which includes REAC's physical inspections as one
of its components) must submit an improvement plan to HUD to eliminate the
deficiencies in the housing authority's performance, whereas no similar
requirement exists for multifamily properties. In addition, they said that
multifamily properties receiving the highest physical inspection scores are
to be reinspected every 3 years, whereas top-performing public housing
authorities will be reinspected every 2 years.

Officials from HUD's Office of Public and Indian Housing said that some of
the differences in how properties are treated, such as the designation of
public housing properties with scores of 60 or below as troubled, is based
on statutory requirements. HUD officials also said that public housing
authorities are required to file improvement plans only if their overall
score under HUD's Public Housing Assessment System is from 60 through 69. If
a housing authority receives a score in this range on the physical
inspection component of the system, it does not need to file an improvement
plan so long as its overall score is 70 or higher. However, the rule
provides that, on a risk-management basis, a field office/program center may
require that a housing authority with a score of not less than 70 submit an
improvement plan. The officials also said they believed allowing
high-performing public housing authorities to be reinspected every 2 years
was an appropriate incentive, but that they would continue to review this
and other incentives as the new assessment system was being implemented.

Prepare and Submit Documentation for Technical Reviews and Database
Corrections

According to the multifamily and public housing industries, HUD's technical
review and database correction processes do not give them enough time to
seek redress for problems with inspections. The technical review process
allows owners and housing authorities to appeal their scores when they
believe contract inspectors have made errors in judgment when assessing the
condition of their properties. The database correction process provides a
means for HUD to correct inspection data when local ordinances or building
codes are not factored into the scoring process. Specifically, industry
groups are concerned that 15 days is not enough time to complete the steps
for requesting technical reviews or database corrections. Industry groups
also fear that HUD will accept appeals for cases only when a change in score
will lead to a change in the property's "designation" and disposition (e.g.,
whether the property will be referred to a HUD office for corrective
action). Finally, industry groups are concerned about the potential for
confusion resulting from HUD's having different processes for handling
technical reviews and database corrections and the possibility that HUD may
not have enough staff to implement the processes efficiently and
effectively.

HUD officials said that the time frames specified in HUD's regulations for
housing authorities and property owners to request technical reviews and
database corrections are short to minimize the prospects of substantial
change in a property's condition between the time an inspection occurs and
the time a request is received. However, the officials said that HUD has
processed the requests even if it has received them after the specified time
frames. They also said that different systems are needed for technical
reviews and database corrections because HUD field offices have to be
involved in the database correction process. They also noted that as they
identify instances calling for database corrections (e.g., when they find
differences between local building codes and REAC's standards), they plan to
modify their inspection database. Over time, corrections in the database
will reduce the burden on field office staff.

Objectives, Scope, and Methodology

Congressional requesters expressed concerns about whether REAC's inspections
represent an improvement over HUD's prior inspection systems and whether
REAC's inspections are reliable. In response to their concerns, we agreed to

ï¿½ discuss the basis for the new physical inspection standards HUD has
established for public and multifamily housing and any differences between
the new standards and inspection procedures and those formerly used by HUD;

ï¿½ summarize the results of the first round of physical inspections and
determine what REAC's reviews of these inspections show about their
reliability;

ï¿½ determine whether REAC's quality assurance measures are sufficient to
ensure that the inspections are reliable and that problems with the
inspections are resolved expeditiously; and

ï¿½ describe concerns expressed by housing industry representatives about the
new system and determine whether they have identified more cost-effective
systems for assessing the physical condition of HUD's properties.

We reviewed HUD's proposed rules for establishing REAC's physical inspection
system; these rules lay out HUD's basis for its new physical inspection
standards. In addition, we reviewed HUD's statutory requirements to provide
decent, safe, and sanitary housing as set forth in the United States Housing
Act of 1937, as amended. We also reviewed the requirements in the
Department's contracts with mortgagors of HUD-insured housing. To determine
how the new standards and procedures differ from prior ones, we interviewed
(1) HUD officials in REAC, the Office of Public and Indian Housing, and the
Office of Housing and (2) representatives of public and multifamily housing
industry associations. We also compared the new standards with HUD's housing
quality standards.

To summarize the results of the baseline inspection data, we obtained two
databases from HUD in early April of this year. These databases contain
information on the first round of inspections of properties in HUD's
multifamily and public housing portfolios. This information includes, for
each property, the type of property inspected, the date of inspection, the
physical inspection score, and the types of exigent health and safety
violations identified during the inspection. We performed tests for
reasonableness and completeness. When we found discrepancies (e.g.,
duplicate entries or data entry errors) we brought them to HUD's attention.
We corrected these discrepancies before conducting our own analyses. To
assess the overall physical condition of these portfolios, we determined the
extent to which properties included in the baseline inventory received
satisfactory physical ratings. To determine what REAC's reviews of these
inspections showed about their reliability, we analyzed the results of
REAC's collaborative and follow-up quality assurance reviews.

To determine whether REAC's quality assurance measures are sufficient to
ensure that inspections are reliable, we interviewed REAC officials,
reviewed contract files and reports, and analyzed data on the different
quality assurance processes and appeals procedures. We analyzed inspection
contracts and visited two of the four inspection contractors, who provided
us with information on their quality control programs. We also judgmentally
selected 10 contract inspections completed at properties in three different
metropolitan areas and accompanied REAC quality assurance inspectors on
follow-up reviews to verify whether each inspection met REAC's standards. To
determine whether inspection problems were being resolved expeditiously, we
reviewed HUD's proposed and final regulations for appeals and comments
submitted on the regulations, and we analyzed appeals data.

To obtain information on the multifamily and public housing industries'
concerns about REAC's inspection system and on whether the industries have
identified more cost-effective systems for assessing properties' physical
condition, we met with representatives of associations for both industries,
including the American Association of Homes and Services for the Aging,
Council of Large Public Housing Authorities, National Association of Home
Builders, National Association of Housing and Redevelopment Officials,
National Affordable Housing Management Association, National Center for
Housing Management, National Housing Conference, National Leased Housing
Association, National Association of Realtors, and Public Housing
Authorities Directors Association. We also discussed concerns and
alternatives with a selected number of executive directors, or their
representatives, for public housing authorities. Finally, we met with
officials from the National Academy of Public Administration to discuss
their ongoing study.

Comments From the Department of Housing and Urban Development

The following are GAO's comments on the Department of Housing and Urban
Development's letter dated June 9, 2000.

1. In response to our recommendation that HUD periodically issue reports
describing the quality assurance activities that it performs and the results
of these activities, REAC updated its quality assurance plan to include the
preparation of a semi-annual report that will summarize the overall results
of REAC's quality assurance activities. REAC also provided us with a 1-page
document containing indicators that it had developed for assessing the
results of its quality assurance activities. However, these indicators do
not cover some key quality assurance activities, such as (1) follow-up
reviews for determining whether inspectors performed inspections completely
and correctly and (2) activities for assessing the inspection system's
precision and replicability. When we later met with REAC officials to
discuss HUD's comments, they told us they were still deciding what
indicators to include in the semi-annual report, and they said they were
planning to add indicators to those included in the document they gave to us
earlier.

2. The revised quality assurance plan that HUD provided us still does not
specify how REAC will use its inspection contractors' quality control data
and information to target its own quality assurance efforts. After giving us
the revised plan, REAC officials told us they recognize that better
coordination is needed between the contractors' and REAC's own quality
assurance programs. They said they are considering revisions in the
requirements for the contractors' quality control programs that would better
integrate them with REAC's quality assurance program. The revisions would
include more specifically defining the information that needs to be included
in the reports that contractors prepare on their quality control activities.
In addition, REAC is looking at ways of providing more information to
contractors for them to use in strengthening their quality control efforts.

3. REAC has taken actions to help ensure that poorly performing inspectors
are dealt with appropriately. For example, it has developed the inspector
tracking system that HUD refers to in its comments, and it has added a step
to its quality assurance plan that is aimed at determining whether an
inspector's substandard performance has been documented and reported to the
contractor. However, given the problems we identified with REAC's
documentation of actions taken against contract inspectors, we believe the
plan should identify the records HUD staff need to maintain to document poor
performance and the actions taken to correct it.

4. Although REAC has developed criteria for assessing some of the activities
covered in the quality assurance plan, it is still not clear how REAC will
assess some other important activities, such as its success in controlling
the level of variance in inspection scores resulting from the application of
its inspection protocol. The plan identifies various tests that will be
performed to assess variations in inspection scores, such as analyses of
data from different inspectors for the same or similar sites, yet it is not
clear what level of variation will be regarded as acceptable. Additionally,
it is not clear what criteria will be used to assess the results of
follow-up reviews aimed at determining whether inspections have been
performed completely and correctly.

5. We continue to have concerns about the criteria REAC plans to use during
collaborative quality assurance reviews for assessing whether inspectors'
performance meets REAC's standards. REAC's quality assurance plan indicates
that inspectors could have an error rate as high as 20 percent in certain
areas before their performance is classified as not meeting REAC's
standards. We recognize that REAC plans to correct data errors in
inspections for those properties where it has actually conducted the
collaborative reviews. However, it is not clear how REAC will determine
whether the inspectors are making similar errors at other properties that
are not reviewed. At our meeting with them to discuss HUD's comments, REAC
officials told us that it is not their intention to allow inspectors with
error rates as high as 20 percent to continue performing inspections and
they would reexamine the criteria used in the plan and define it more
clearly.

GAO Contacts and Staff Acknowledgments

Stanley J. Czerwinski, (202) 512-7631

Richard A. Hale, (202) 512-3090

In addition to those named above, Karen Bracey, Martha Chow, Mark Egger,
Bess Eisenstadt, John McGrail, Sally Moino, Jay Raple, Rose Schuville, and
Don Watson made key contributions to this report.

(385813)

Table 1: Physical Inspection Results for Properties in HUD's
Baseline Inventory, by U.S. Region 37

Table 2: Physical Inspection Results for Multifamily Properties
in HUD's Baseline Inventory, by Type of Financing
and Assistance 38

Figure 1: Number of Multifamily and Public Housing Properties
With Inspection Scores of Less Than 60 and 60 or Higher 12

Figure 2: Physical Conditions That REAC Quality Assurance
Reviewers Found Contract Inspectors Did Not Record at Properties GAO Visited
16

Figure 3: Percentage of Multifamily and Public Housing Property Inspections,
Sorted by Inspection Scoring Range 35

Figure 4: Percentage of Multifamily and Public Housing
Property Units, Sorted by Inspection Scoring Range 36
  

1. HUD's uniform physical condition standards apply to Section 8
project-based assisted housing ("project-based" assistance is attached to
the structure, in contrast to "tenant-based" assistance, which is attached
to the resident).

2. A particular deficiency can be included in more than one of these five
major inspectable areas.

3. Although smoke detector violations are labeled as life-threatening,
points are not deducted for them, and they are noted by an asterisk attached
to the score.

4. REAC officials refer to follow-up reviews as limited quality assurance
inspections.

5. The physical condition standard currently required under the United
States Housing Act of 1937 is referred to as "decent and safe." HUD does not
consider this standard substantively different from the previous statutory
standard of "decent, safe, and sanitary."

6. The new inspection procedures apply to the HUD housing covered by the
uniform physical condition standards.

7. Our analysis excludes the inspection scores for 164 multifamily
properties because we lacked complete information on the characteristics of
these properties.

8. According to data provided by HUD's Office of Public and Indian Housing,
about 87 percent of the nation's public housing authorities received
composite physical inspection scores of 60 or above. These housing
authorities accounted for about 65 percent of the public housing units.

9. Initially, REAC classified the inspections according to four
categories--highly satisfactory, satisfactory, less than satisfactory, or
unsatisfactory--but in March 2000, REAC consolidated the four categories
into two--within standard and outside of standard.

10. HUD also corrected the inspection score or grade for health and safety
violations for another 1,400 properties. REAC officials told us that most of
these corrections were required because of an error in the scoring software
or differences between local building standards and REAC's inspection
standards that REAC identified during the initial months of the inspection
process.
*** End of document. ***