General Services Administration: STAR-PBS' New Program for Tracking and
Managing Real Property (Letter Report, 10/12/1999, GAO/GGD-00-12).

Pursuant to a congressional request, GAO reviewed the Public Buildings
Service's (PBS) System for Tracking and Administering Real Property
(STAR), focusing on: (1) the functions STAR performs; (2) whether users
were having problems using STAR; (3) the steps PBS had taken or is
taking to address any user problems; and (4) the actions PBS had taken
or plans to take to ensure that STAR data are accurate, reliable, and
consistent.

GAO noted that: (1) STAR is a real estate inventory management software
application that maintains data on projects, leases, buildings, and
space assignments--that is, who is in what space--in an integrated
database environment; (2) the data in STAR are used to generate bills to
clients for their assigned space, develop budget plans, track and manage
leases, and evaluate performance; (3) STAR's input/output functions
include: (a) direct on-screen access to the data; (b) the ability to
update the data; (c) access to standard reports; and (d) the ability to
sort data and create ad hoc reports; (4) users identified problems with
STAR as well as benefits derived from using the system; (5) the 128
users GAO interviewed identified 18 types of problems that they had
experienced using STAR; (6) these problems included complaints such as
taking too long to enter data and having too many steps or screens; (7)
according to the STAR project manager, all of the problems that were
reported to GAO by users had been reported to the STAR project team
through the contractor-operated STAR Help Desk; (8) also, STAR users had
submitted nearly 8,000 requests, which were recorded on service tickets,
to the Help Desk between October 1997 and August 1999; (9) these
tickets, many of them concerning the same issue, identified problems
that users reported having when working with STAR as well as STAR
enhancement suggestions and other issues related to STAR's operations;
(10) further, 116 of the 128 users GAO interviewed saw at least 1
benefit from using STAR, such as data being current and readily
available; (11) the STAR Help Desk and the PBS Change Review Team were
set up to solve problems identified by STAR users and to resolve other
issues with STAR; (12) by the end of November 1999, changes and
enhancements will have resulted in eight versions of STAR; (13) further,
additional contract help had been provided to assist the regions with
STAR problems and training; and (14) PBS has taken various actions to
try to ensure that STAR has accurate, reliable, and consistent data: (a)
STAR has been designed with built-in features to help ensure the
reliability of some data elements; (b) PBS ran edits against the data
before they were transferred to STAR and ran additional edits after data
were transferred; (c) PBS initiated an effort to validate all space
assignments; (d) PBS conducted a special study and produced a report
that identified data problems and made recommendations for solving the
problems; (e) a team was set up to determine how to implement the
report's recommendations; and (f) PBS is developing a quality control
system.

--------------------------- Indexing Terms -----------------------------

 REPORTNUM:  GGD-00-12
     TITLE:  General Services Administration: STAR-PBS' New Program for
	     Tracking and Managing Real Property
      DATE:  10/12/1999
   SUBJECT:  Computer software
	     Management information systems
	     Information resources management
	     Data integrity
	     Federal property management
IDENTIFIER:  PBS System for Tracking and Administering Real Property

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO report.  Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved.                 **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
** A printed copy of this report may be obtained from the GAO   **
** Document Distribution Center.  For further details, please   **
** send an e-mail message to:                                   **
**                                                              **
**                                            **
**                                                              **
** with the message 'info' in the body.                         **
******************************************************************

United States General Accounting Office
GAO

Report to Congressional Requesters

October 1999

GAO/GGD-00-12

GENERAL SERVICES ADMINISTRATION
STAR-PBS' New Program for Tracking and Managing

Real Property

Ordering Information

The first copy of each GAO report and testimony
is free. Additional copies are $2 each. Orders
should be sent to the following address,
accompanied by a check or money order made out
to the Superintendent of Documents, when
necessary. VISA and MasterCard credit cards are
accepted, also. Orders for 100 or more copies to
be mailed to a single address are discounted 25
percent.

Order by mail:

U.S. General Accounting Office
P.O. Box 37050
Washington, DC 20013

or visit:

Room 1100
700 4th St. NW (corner of 4th and G Sts. NW)
U.S. General Accounting Office
Washington, DC

Orders may also be placed by calling (202) 512-
6000 or by using fax number (202) 512-6061, or
TDD (202) 512-2537.

Each day, GAO issues a list of newly available
reports and testimony. To receive facsimile
copies of the daily list or any list from the
past 30 days, please call (202) 512-6000 using a
touch-tone phone. A recorded menu will provide
information on how to obtain these lists.

For information on how to access GAO reports on
the INTERNET, send e-mail message with "info" in
the body to:

[email protected]

or visit GAO's World Wide Web Home Page at:

http://www.gao.gov

United States
General Accounting Office
Washington, D.C. 20548-0001
Official Business
Penalty for Private Use $300
Address Correction Requested

                    Bulk Rate
               Postage & Fees Paid
                       GAO
                 Permit No. G100

(240329)

Contents
Page 201GAO/GGD-00-12 STAR-Program for Tracking and Managing Real
Property
Letter                                                                      1
                                                                             
Appendix I                                                                 22
Definitions of STAR Help
Desk Ticket Types
                           Conversion Issues                               22
                           Database Change                                 22
                           Enhancement-Application                         22
                           Enhancement- Ad Hoc Report                      22
                           Enhancement-Report Standard                     22
                           Inquiry                                         22
                           Inquiry Reports                                 22
                           Maintenance-Nonscheduled                        22
                           Maintenance-Scheduled                           22
                           Policy and Procedures                           23
                           Rent Bill                                       23
                           Security Request                                23
                           Table Update                                    23
                           Training                                        23
                           Trouble                                         23
                           Trouble Reports                                 23
                                                                             
Appendix II                                                                24
Data Issues Identified
by PBS' Go Team and Its
Recommendations
                                                                             
Appendix III                                                               26
Comments From the
General Services
Administration
                                                                             
Tables                     Table 1: Description of STAR's Seven             6
                           Major Modules
                           Table 2:  Identified User Problems               8
                           Table 3: PBS' Responses to Examples of           9
                           User Problems
                           Table 4:  Help Desk-Identified User             11
                           Problems, Ranked by the Number of
                           Service Tickets Received for the
                           Problem
                           Table 5:  Examples of Go Team-                  16
                           Identified Issues and Recommendations
                           Table 6: PBS' Prioritized Action Plans          16
                           by Priority
                                                                             

Abbreviations

CRT       Change Review Team
GSA       General Services Administration
IG        Office of Inspector General
PBS       Public Buildings Service
PROMPT    Permanent Record of Managed Property
Transactions
STAR      System for Tracking and Administering
Real Property

B-281769

Page 18GAO/GGD-00-12 STAR-Program for Tracking and
Managing Real Property
     B-281769

October 12, 1999

The Honorable Bob Franks
Chairman
The Honorable Robert Wise, Jr.
Ranking Democratic Member
Subcommittee on Economic Development,
 Public Buildings, Hazardous Materials,
 and Pipeline Transportation
Committee on Transportation and Infrastructure
House of Representatives
 
This report responds to your request for a review
of the General Services Administration's (GSA)
Public Buildings Service's (PBS) System for
Tracking and Administering Real Property (STAR).
Your Subcommittee was concerned about problems
that were being reported to it concerning STAR. As
agreed with the Subcommittee, our objectives were
to determine

ï¿½    the functions STAR performs;
ï¿½    whether users were having problems using
STAR;
ï¿½    the steps PBS had taken or is taking to
address any user problems; and
ï¿½    the actions PBS had taken or plans to take to
ensure that STAR data are accurate, reliable, and
consistent.

As agreed with your offices, we did not review the
technical aspects of STAR because GSA's Office of
Inspector General (IG) is reviewing the technical
and systemic issues relating to STAR. The IG
expects to issue its report by the end of 1999.

Results in Brief
STAR is a real estate inventory management
software application that maintains data on
projects, leases, buildings, and space
assignmentsthat is, who is in what space-in an
integrated database environment. The data in STAR
are used to generate bills to clients for their
assigned space, develop budget plans, track and
manage leases, and evaluate performance. STAR's
input/output functions include

ï¿½    direct on-screen access to the data,
ï¿½    the ability to update the data,
ï¿½    access to standard reports, and
ï¿½    the ability to sort data and create ad hoc
reports.

Users identified problems with STAR as well as
benefits derived from using the system. The 128
users we interviewed identified 18 types of
problems that they had experienced using STAR.
These problems included complaints such as taking
too long to enter data and having too many steps
or screens. According to the STAR project manager,
all of the problems that were reported to us by
users had been reported to the STAR project team
through the contractor-operated STAR Help Desk.
Also, STAR users had submitted nearly 8,000
requests, which were recorded on service tickets,
to the Help Desk between October 1997 and August
1999. These tickets, many of them concerning the
same issue, identified problems that users
reported having when working with STAR as well as
STAR enhancement suggestions and other issues
related to STAR's operations. Further, 116 of the
128 users we interviewed saw at least 1 benefit
from using STAR, such as data being current and
readily available.

The STAR Help Desk and a PBS Change Review Team
(CRT) were set up to solve problems identified by
STAR users and to resolve other issues with STAR.
By the end of November 1999, changes and
enhancements will have resulted in eight versions
of STAR. Further, additional contract help had
been provided to assist the regions with STAR
problems and training.

PBS has taken various actions to try to ensure
that STAR has accurate, reliable, and consistent
data.

ï¿½    STAR has been designed with built-in features
to help ensure the reliability of some data
elements. For example, work queues were to
automatically generate requests for such things as
approvals of leases, corrections, or notifications
of missing information; work queues could also be
used manually to send information.
ï¿½    PBS ran edits against data before they were
transferred to STAR and ran additional edits after
data were transferred.
ï¿½    PBS initiated an effort to validate all space
assignments.
ï¿½    PBS conducted a special study and produced a
report that identified data problems and made
recommendations for solving the problems. 1
ï¿½    A team was set up to determine how to
implement the report's recommendations. PBS has
started implementing some of these
recommendations.
ï¿½    PBS is developing a quality control system.

Background

With the introduction of STAR and other changes, a
major cultural change has occurred in GSA's
facilities management. According to the PBS
report,

"The simultaneous launches of New Pricing, STAR,
Occupancy Agreements, and a host of other re-
engineered business line processes and software
constitutes the most significant change to the way
PBS does its business since the creation of the
Federal Buildings Fund."

The mission of PBS is to effectively manage over
8,000 owned and leased federal real properties in
the United States. PBS had previously used a
computer program called PLUS+ to track and manage
its real properties. In 1994, PBS' Central Office
decided that the systems it used to support its
mission were antiquated and needed to be replaced.
New software was needed to support real property
inventory, leasing, space assignments, and billing-
related activities. PBS did market research
analyses from March 1994 through October 1995,
evaluating various real estate software packages.

In March 1994, PBS began its initial efforts to
obtain new software. It published a notice in the
Commerce Business Daily seeking sources to supply
software products to support management operations
for leased and owned public buildings. Two
responses to this notice were received. However,
PBS determined that neither of these sources met
its functional needs.

PBS also hired a contractor to identify any
existing property management software that might
meet its needs. The contractor developed a list of
400 software products that it said spanned all
facets of PBS' business functions. Using PBS'
needs as criteria, this list was reduced to 36
products, then to 6 products for further study.
However, PBS concluded that none of these products
was capable of satisfying all of its minimum
requirements, such as the ability and capacity to
concurrently manage over 8,000 owned and leased
properties.

During discussions with real estate companies and
Fortune 500 companies about real estate practices
and procedures, PBS discovered a proprietary
system called Permanent Record of Managed Property
Transactions (PROMPT). This system had been
implemented by AT&T during the summer of 1994.
According to PBS officials, PBS ran a 4-month
pilot test in 1995 on PROMPT. During this test, 30
percent of PBS' data were diagrammed and selected
data elements on leases and space assignments, as
well as other areas, were entered into the system.
The test was completed in October 1995. PBS
concluded that although PROMPT could meet its
needs, the data were not in the format that it
wanted.

In July 1996, an acquisition plan for STAR was
approved by GSA's Associate Administrator for
Acquisition Policy. This plan called for a sole
source contract with AT&T for the use of PROMPT,
which PBS renamed STAR. On January 17, 1997, the
contract, which included a license agreement
granting use of PROMPT in perpetuity unless
canceled by the licensee, was signed with AT&T.
The contract was for labor hours on a best-effort
performance basis to convert PROMPT version 2.7
into STAR, which met PBS' requirements. The base
contract was for $8.3 million. Of this amount,
$4.3 million was for the perpetual right to use
the national license; the remaining $4.0 million
was for software maintenance, a STAR reference
guide, training courses and manuals, technical
support services, hotline support, and travel. The
contract also included several options. If all of
these options were exercised, the total cost of
the contract would have been $17.0 million. On
October 19, 1998, the quantities in one of the
options-option four-were increased adding 1,000
users, 310 training classes, 2,900 sets of
training manuals, and 23,200 staff hours of
technical support. This raised the value of the
contract to $21.8 million, which was an increase
of $4.8 million.

PBS began rolling out STAR in October 1997, and it
was available in all regions by January 1998. The
first rent bills generated by STAR were sent to
agencies in July 1998.

PBS officials said that the introduction of STAR,
among other things, has led to a cultural change
that has had many effects in PBS. The officials
noted that three changes particularly, affected
the realty specialists who maintain STAR. First,
realty specialists are now required to enter the
data directly into the STAR system; second, they
are to be held directly responsible for data
accuracy; and third, they are responsible for
billing the client agencies. In the past, the
realty specialists filled out data entry sheets
and data entry clerks entered the data. It was not
clear at that time who was responsible for the
data accuracy. Further, in the past, realty
specialists were not involved in the actual
billing of the clients.

Scope and Methodology
To determine what functions STAR performs, we
observed how STAR operates; interviewed officials
responsible for the development of STAR; and
reviewed contract documents, training manuals, and
the draft desk reference manual for STAR.

To identify problems that users were experiencing
using STAR, we judgmentally selected four regions,
two of which were identified by PBS officials as
having problems implementing STAR, one that was
the lead region for STAR, and one from which
congressional staff had received complaints. We
also included the headquarters unit to get its
perspective on STAR. PBS officials identified the
users of STAR at these locations, of which realty
specialists and realty assistants were identified
as the major users. We then interviewed 128 users,
81 of which were realty specialists or realty
assistants, in the 5 locations and asked them
about the problems they have had using STAR.  Our
results cannot be projected to the total universe
of STAR users.

Further, we reviewed Help Desk reports that
organized user complaints about STAR into subject
categories. Because of the way that the issues
were categorized in the report and the actual
number of items reported to the Help Desk-almost
8,000-we were not able to directly compare our
interview-identified problems with those reported
to the Help Desk. However, we discussed the
problems that users had identified in our
interviews, and problems that had been reported to
the Help Desk, with the STAR project manager.
Further, we reviewed some Help Desk reports that
showed specific problems identified.

To identify the steps that PBS has taken or is
taking to deal with user problems, we interviewed
officials who were responsible for addressing STAR
problems and reviewed documents on changes being
made to correct problems and improve STAR.

Finally, to determine what PBS has done or plans
to do to ensure STAR data integrity, we
interviewed officials and reviewed plans
addressing PBS' efforts to ensure that STAR data
are accurate, reliable, and consistent. We did not
independently assess the accuracy, reliability,
and consistency of STAR data.

     We did our work at GSA/PBS Headquarters and
the National Capital Region in Washington, D.C.;
the Southeast Sunbelt Region in Atlanta, GA; the
New England Region in Boston, MA; and the Mid-
Atlantic Region in Philadelphia, PA. Our work was
done in accordance with generally accepted
government auditing standards. We requested
comments on a draft of this report from the
Administrator of GSA. GSA's written comments are
discussed at the end of this letter.

STAR-A System for Tracking and Administering Real
Property

STAR is a real estate inventory management
software application that maintains data on
projects, leases, buildings, and space
assignmentsthat is, who is in what space-in an
integrated database environment. The data in STAR
have been used to generate bills to clients for
their assigned space, develop budget plans, track
and manage leases, and evaluate performance.
STAR's input/output functions include

ï¿½    direct on-screen access to the data,
ï¿½    the ability to update the data,
ï¿½    access to standard reports, and
ï¿½    the ability to sort data and create ad hoc
reports.

To develop the fiscal year 2001 budget, STAR data
are being used to support the development of the
over $5.5 billion rental revenue estimate and as
the basis for the Galaxy system's roughly $3.1
billion estimate of PBS lease expenditures.2

STAR has seven major modules: (1) Space
Management, (2) Client Billing, (3) Project
Administration, (4) Billing, (5) Work Queue, (6)
Administration, and (7) Reports. Table 1 describes
each of these modules.

Table 1: Description of STAR's Seven Major Modules
Module                    Description
Space Management          Supports the recording and processing of data
                         related to buildings, space, occupants, and
                         occupancy. Data stored include building location,
                         square footage, lease terms, types of space,
                         occupant, and payment schedules.
Client Billing            Supports the development, documentation, and
                         implementation of specific occupancy agreements
                         with GSA clients. The data stored include client
                         terms, space identification, and client-specific
                         rates.
Project Administration    Supports the tracking of actions taken to satisfy
                         a federal agency's space request. These actions
                         include new lease acquisitions, consolidations,
                         and constructing a new room or expanding space.
Billing                   Generates monthly bills to client agencies on the
                         basis of their space assignment; supports
                         miscellaneous adjustments, such as debits or
                         credits, to bills that have already been sent to
                         the client. The data stored include rent to be
                         billed and miscellaneous debits and credits.
Work Queue                Supports an on-line to do list for realty services
                         users and for interoffice communications. It has
                         five types of work queues-approval, copy,
                         corrections, action required, and notification.
Administration            Manages administrative tasks related to STAR.
                         Accessible only by contractor.
Reports                   Supports standardized reports on space management,
                         project administration, and billing. Also provides
                         report writer for writing ad hoc reports.
Source: GAO analysis of PBS documents.

Each of these modules has data entry screens. For
example, the Space Management module has a
Building Details screen for entering specific
building information and a Lease Details screen
for entering specific lease information. STAR is
supported by a contractor-operated Help Desk to
aid users if they have problems with the STAR
modules and screens.

PBS identified the following benefits of STAR.

ï¿½    Movement of data and information systems
closer to the end user.
ï¿½    Direct access to project, lease, building,
and space assignment data.
ï¿½    Electronic notification of expiring leases,
unassigned space, and other information.
ï¿½    Integration of information technology into
business practices.
ï¿½    Direct input eliminates the need for data
transcribers and precludes duplicative
keypunching.
ï¿½    Easy report generation and exporting of data
to another program.
ï¿½    No need to memorize data element numbers, it
uses English.
ï¿½    Tutorials on policies and procedures can be
placed into help screens of the system.
ï¿½    Will allow for direct update of agency
information.
ï¿½    Improves corporate competitiveness.

     In the future, PBS hopes to expand STAR's
capabilities by adding more features, such as
direct access by an agency to view the information
that STAR maintains concerning the agency's
assigned space.

Users Identified Problems and Benefits for STAR
The 128 users we interviewed identified problems
they had experienced using STAR as well as several
benefits they saw being derived from the use of
STAR.3 Of the problems identified by users, 17
were mentioned by at least 10 of the users. One
problem was mentioned by only one user but was
included in this report because it involves
accounting for rent revenues.4 According to the
STAR project manager, each type of problem that
was reported to us had been reported to the STAR
project team through the STAR Help Desk and logged
into 16 categories. According to the project
manager, the PBS project team has taken actions to
(1) resolve the reported problems that the team
believed were legitimate and (2) add the requested
enhancements that the team believed were
appropriate. Further, 116 of the users we
interviewed saw at least 1 benefit from using
STAR.

Table 2 gives a breakdown of the problems that
individuals we interviewed identified.

Table 2:  Identified User Problems
Type of problem                                                   Number of
                                                          users identifying
                                                                    problem
Takes long time to enter data, too many steps/screens.a                  99
System slow at times, you get system errors.                             92
Have to enter the same information repeatedly in different               87
modules because modules are not linked.
Printing requires multiple steps.                                        73
Not user friendly.                                                       71
Realty specialist can go into any lease in region, data                  55
security issues.
STAR will not let you save work until certain information                55
has been filled in.
No history for R620 reports, authorizes payment by                       39
finance, only a digest. Realty specialists believed a
history is needed in STAR.
Cannot enter same date twice in the payment schedule                     29
screen. This results in the dummying of dates on the
Payment Schedule screen.
Hard-to-use report writer for ad hoc reports because you                 27
have to download data to another program module by module.
Cannot access all fields in STAR.a
STAR prompts are not clear.                                              27
Work queues-You are getting too many that you do not need,               24
getting work queues on items that are not the realty
specialist's responsibility. These waste the user's time.
a
Bills-need to explain the manual adjustments on the bill.                22
Need to calculate joint use space to the same number of
decimal places for the bill as STAR does internally.
Billing adjustments cumbersome to make.a
Once a change has been approved, you cannot correct an                   21
error without initiating a database change through the
STAR Help Desk.
STAR lacks a Desk Reference Manual.                                      20
Realty specialist does not use STAR often enough to become               19
proficient.
STAR lacks a standard report called a combined space                     13
report.
Cannot enter rent for National Antenna Program.b                          1
aWe have combined nine identified problems into
four generic problems.
bThis problem was included because it involves
accounting for rent revenues.
Source:  Compiled by GAO from interviews with STAR
users.

The project manager said that the STAR project
team had taken action or action was being taken on
some of the identified problems. Other problems
relate to inherent parts of STAR that cannot be
easily fixed or occurred because of decisions made
during the STAR developmental process. Further,
according to another PBS official, user problems
may also be a reaction to learning new business
processes, such as New Pricing, and other changes.

Table 3 contains PBS' responses to examples of the
problems we identified through interviews.

Table 3: PBS' Responses to Examples of User
Problems
Problem                   PBS response to the problem
Printing requires         STAR has excellent data manipulation capabilities
multiple steps.           but the users are not used to downloading in order
                         to print. Also, it would be very expensive to make
                         STAR print directly from the screen.
STAR will not let you     To ensure that information was entered in STAR, it
save work until           was designed not to let you save work until
certain information has   certain data entry blanks have been filled in.
been filled in.
Realty specialist can go  All realty specialists in a region can go into all
into any lease            leases, if they have been granted access to
in region, data security  leases. It will be up to each region to decide if
issues.                   it wants to limit access to a greater extent. It
                         can be done.
No history for R620       It was decided during development not to have a
reports, authorizes       history of the R620 reports in STAR. There is a
payment by finance, only  digest of the reports on STAR and a hard copy of
a digest.                 the individual R620s can be located in the lease
Realty specialists        file.
believed a history
is needed in STAR.
Work queues-You are       Work queues serve an important role in STAR by
getting too               keeping users informed of problems and other
many that you do not      issues. The work queue process is being
need, getting             streamlined in a release scheduled for November
work queues on items that 1999.
are not
the realty specialist's
responsibility.
These waste the user's
time.
Once a change has been    Because of the approval process in STAR for making
approved,                 changes, you have to go to the Help Desk for a
you cannot correct an     database change to correct an error once a change
error without             has been approved.
initiating a database
change through
the STAR Help Desk.
Source: Table 2 of this report and interview with
the STAR project manager.

The STAR developmental process also affected STAR,
particularly in relation to the user friendliness
problem. According to the project manager, the
time frame for developing STAR covered from
January 1997 until October 1997. A different team
developed each module in STAR, and communications
among the teams may not have been effective
because of the time constraints. The project
manager thought that the lack of communications
may have resulted in inconsistencies and
duplication among modules. Further, she said that
STAR's development was hindered because during the
development phase it was learned that STAR would
have to handle both the old pricing policy for
client space and the new pricing policy for client
space, not just the new pricing policy as
originally planned. This resulted in having to
change STAR's design. Also, she suggested that PBS
may have released STAR to the regions before it
was ready, resulting in some of the problems that
have been identified by users.

     The project manager said that a cultural
change occurred for realty specialists that
probably affected their view of STAR. Before STAR
and other business changes, realty specialists
were not responsible for entering and maintaining
data using a computer or for billing client
agencies for their assigned space. With the
introduction of STAR, New Pricing, New Billing,
and the Occupancy Agreement Tool, additional
workloads were placed on the realty specialist.5
Also, each initiative had its own implementation
problems that the realty specialists had to deal
with in doing their jobs.

The nearly 8,000 service tickets handled by the
Help Desk reflect PBS-wide problems and other
issues reported by users. PBS had classified these
tickets into 16 different categories. According to
PBS officials, the same problem may have been
reported on many different service tickets. Table
4 shows the Help Desk's breakdown of these service
tickets received and the status of the tickets by
the 16 categories, as of August 12, 1999.
Definitions of the categories can be found in
appendix I.

Table 4:  Help Desk-Identified User Problems,
Ranked by the Number of Service Tickets Received
for the Problem
Category of problem                              Number of   Closed    Open
                                                   service
                                                   tickets
                                                  received
Troublea                                             3,540    3,496      44
Database change                                      2,080    2,033      47
Inquiry                                                659      652       7
Enhancement-application                                590      429     161
Rent bill                                              364      359       5
Trouble reports                                        189      178      11
Enhancement-ad hoc reports                              94       50      44
Inquiry reports                                         90       89       1
Enhancement-report standard                             88       61      27
Security request                                        78       78       0
Table update                                            59       55       4
Policy and procedures                                   40       33       7
Training                                                28       27       1
Maintenance-nonscheduled                                14       14       0
Conversion issues                                       11       11       0
Maintenance-scheduled                                    4        3       1
Total                                                7,928    7,568     360
aOriginally, this category was used as a catchall
until other categories were defined.
Source: Help Desk contractor document.

Trouble tickets refer to any part of the system
that the user may be having problems with, except
for the report function, including essential
nonfunctioning software. The contractor is
responsible for fixing the problems associated
with all tickets categorized as trouble at no cost
to PBS. The three enhancement categories contain
requests to make changes to STAR. Any changes to
STAR resulting from these tickets categorized as
enhancements have to be negotiated between PBS and
the contractor, with PBS' paying for the change.
The CRT leader said the team had processed 511
enhancement tickets, of which about 44 were still
open. The official said that this tally and the
contractor's tally of tickets categorized as
enhancements, as shown in table 4, do not match
because the two had not been recently reconciled.
She said the contractor's system probably had some
items in the enhancement categories that were not
enhancements and CRT had probably closed some
other tickets that were not yet recorded in the
contractor's computer database as closed.

While the 128 users we interviewed reported
problems with STAR, 116 of them saw some positive
benefit from STAR. Users identified the following
as the top five benefits.

ï¿½    Information is current and readily
accessible.
ï¿½    Can see that changes to data have been
accepted by the system.
ï¿½    STAR makes it easy to make minor changes.
ï¿½    STAR directly controls billing and lease
payments.
ï¿½    It is good to have control over the data for
which you are responsible.

PBS' Efforts to Improve STAR
As previously mentioned, as part of the STAR
project, a contractor-operated Help Desk and PBS'
CRT were set up to solve STAR problems. The Help
Desk and CRT have handled about 8,000 service
tickets, of which 95 percent have been closed.
Further, other actions, such as additional
contract help, are now available to the regions to
help with STAR problems and training.

The Help Desk is to record user complaints,
enhancement suggestions, or other issues
concerning the use of STAR on service tickets. CRT
reviews and determines how to handle enhancement
tickets and trouble tickets that require a fix
through a new release of STAR. Guidance on the
review process states that no action on a service
ticket usually means that a ticket resulted from a
user's momentary frustration with the system, a
passing thought of a user, or that other users had
already identified the problem.

If the enhancement or trouble service ticket is
accepted, CRT is to edit and define the ticket to
clearly identify the issue. CRT is also to assign
each of these tickets a priority-high, normal, or
low. Working with the contractor, CRT then decides
how to implement any change considered needed. By
the end of November 1999, eight versions of STAR
will have been issued. According to PBS reports,
STAR releases 3.0, and 4.0, combined, have about
40 enhancements. Version 4.0, which is to be
released by the end of November 1999, for example,
is to address the problems of (1) having to enter
the same information repeatedly in different
modules because modules are not linked and (2) not
having the ability to enter the same date twice in
the Payment Schedule screen. A category named
"zero square foot leases" is also to be added to
STAR to cover the national antenna program in a
release scheduled for the spring of 2000.6 As of
August 12, 1999, additional enhancement tickets
are still open, although the exact number was
unclear because PBS had not reconciled the data
between the CRT and the computer-based Help Desk
report. Other efforts to improve STAR include the
following:

ï¿½    A desk reference manual for STAR is to be
issued in November 1999. This manual should help
with the problem of the need to have better
prompts that make where to go clearer. The users
should be able to look up what to do.
ï¿½    A report is under development that would be
similar to the combined space report, which
provided such information as building address,
square footage, and lease-specific information.
ï¿½    Renewed emphasis has been placed on examining
a preview of the bill in STAR before actual bills
are sent.
ï¿½    An ad hoc exception report on bills is being
sent to the regions each month to catch errors
more quickly.
ï¿½    PBS has set a specific accuracy goal that
targets a specific data problem for improvement
and rewards regions for meeting the improvement
goal. We were told that this data accuracy goal
would be changed on a regular basis. It currently
focuses on the issue of chargebacks and missing
rates.

PBS' Efforts to Ensure Accurate, Reliable, and
Consistent Data in STAR
PBS knows that it has data problems, and these
problems have been well documented. Examples of
the problems follow:

ï¿½    The GSA IG concluded that over half of the
space assignment drawings are off by more than 5
percent.
ï¿½    Price Waterhouse Coopers, in its audit of
GSA's fiscal year 1998 financial statements, found
a reportable condition that improvements are
needed in the controls over the integrity of rent
and leasing data used to manage the Federal
Buildings Fund.
ï¿½    Exception reports and queries indicate that
there is significant regional variation in how
well data in STAR are maintained, how well PBS
produces occupancy agreements, and the accuracy of
rent billings. PBS had to do extensive
reconciliation of all congressional rent bills,
the Federal Bureau of Investigation had to tell
PBS that it was not being billed for $3.0 million
worth of assignments, and 40 percent of the first
wave of Census Bureau offices that PBS had
delivered had not been billed as of January 1999.

PBS said it has taken various actions to ensure
that STAR has accurate, reliable, and consistent
data.

ï¿½    STAR was designed with built-in features to
help ensure the reliability of some data elements.
ï¿½    PBS ran edits against the data to be
transferred to STAR and ran additional edits after
data were transferred.
ï¿½    PBS has an initiative to validate all space
assignments.
ï¿½    A special study was done that produced a
report identifying data problems and made
recommendations for solving the problems. 7
ï¿½    A team was set up to determine how to
implement the report's recommendations.
ï¿½    PBS is developing a quality control system.

Controls have been built into STAR to make the
program provide more accurate, reliable, and
consistent data. For example, STAR has fields that
have to be filled in before you can go to the next
step of your task. This is to ensure that all of
the required data for a task are entered. Another
example is work queues, which automatically
generate requests for such things as approvals of
leases, corrections, or notifications of missing
information and may also be used manually to send
information. This feature is also designed to
automatically notify the budget analyst
responsible for rent expenditures of changes so
that the analyst may anticipate the effect of the
change on expenditures. STAR makes certain
calculations using data entered in the database to
reduce the chance of human error.

During the transition to STAR, a PBS team was
formed to work with each of the teams that was
developing STAR modules to assist in data
transfer. According to the team leader, this team
developed diagnostic edits for each module that
were run on a region-by-region basis to determine
missing or incomplete data before the transfer of
data to STAR. Further, AT&T wrote a conversion
program that included a list of edits to run
against the data once they were transferred to
STAR to identify missing or incomplete data.
Finally, the post-conversion STAR team was tasked
with regularly identifying queries, exception
reports, and flags that are to be added to STAR to
help identify pockets of bad data.

To further ensure accurate, reliable, and
consistent data, PBS established a Go Team in
January 1999 to identify and quantify the real and
perceived problems with inventory data-space
assignment drawings, lease files, assignment
files, rent rates, bills, and information in STAR
and Occupancy Agreements. For example, as
previously mentioned, the GSA IG found half of the
space assignment drawings were off by more than 5
percent. The Go Team found considerable problems
with how well PBS was creating final rent
rates-the proper calculation and application of
fees and other charges. In a spot check of 20
Client Billing Records, the team found minor
errors in 100 percent of the cases. Also, the team
found errors in parking information. In one
region, a discrepancy between total parking and
assigned parking, which should match in STAR, was
found in about 5 percent of the buildings. This
problem could affect revenues. Further, the team
found errors in design gross square feet and gross
square feet, which should match in STAR.  In three
regions in particular, substantial variances
between the design gross square feet and the gross
square feet were found. Again, this could affect
revenues.

In the team's April 1999 report, it identified the
causes of known data problems and identified
actions that needed to be taken to fix data
problems. The Go Team report identified the
following as the root causes of data problems:

ï¿½    the lack of accountability and ownership of
data;
ï¿½    the lack of clear, consistent, complete
policies associated with PBS' new business
practices;
ï¿½    incomplete training;
ï¿½    bad data to begin with;
ï¿½    PBS' reorganization that moved or de-
emphasized assignment functions;
ï¿½    migration from the old computer program to
STAR;
ï¿½    personnel turnover, downsizing, and workload;
and
ï¿½    changing standards, such as moving from
occupiable square feet to usable square feet to
rentable square feet.

In addition to those actions already under way
that were discussed under the previous section of
this report, the Go Team identified issues that it
said must be addressed to improve data quality.
These issues were

ï¿½    management responsibilities,
ï¿½    accountability and ownership,
ï¿½    training and tools, and
ï¿½    reports and measures.

Table 5 lists examples of the suggested changes
made by the team. Appendix II lists all of the
suggested changes.

Table 5:  Examples of Go Team-Identified Issues
and Recommendations
Type of issue             Recommendations
Management's              Establish national policies, answer questions, and
Responsibilities          resolve disputes in a timely manner.
                         Change the Central Office culture of "Information
                         is power so I won't share it."
                         Ensure that changed business practices and new
                         tools are adopted at the working level.
Accountability and        Job descriptions should include data integrity.
Ownership                 Competency testing and certification should be
                         established for employees who need to understand
                         New Pricing, Occupancy Agreements, and STAR.
                         Performance ratings should address the issue of
                         employee proficiency in the use of tools and
                         databases required in their specific job
                         functions. Employees should be evaluated on the
                         integrity of their data and the promptness and
                         accuracy of their input.
Training and Tools        Need to make sure that everyone who uses STAR gets
                         the right kind of training.
                         Cross training opportunities between business
                         lines should be encouraged.
                         More changed business practices training should be
                         scheduled.
Reports and Measures      Develop meaningful performance measures for data
                         integrity. Any measure that is adopted should
                         identify the magnitude of the data problems and
                         encourage additional accountability and ownership.
                         Possible performance measures for data include the
                         number of exceptions/flags generated for each
                         region in STAR and the number and dollar value of
                         chargebacks.
Source: Data Issues in PBS, PBS, April 1999.

In May 1999, another team was established to
determine how to implement the first team's
recommendations. The second team developed a set
of six prioritized action plans that include both
temporary stop-gap fixes and permanent solutions.
Table 6 lists the prioritized action plans, in
priority order.

Table 6: PBS' Prioritized Action Plans by Priority
Action plan               Purpose
1. Education and          Initially educate internally and externally for
Involvement               implementation of new billing in October 1999.
                         Address long-term educational and training needs
                         associated with data accuracy.
2. Making Decision        Address the mechanics of decisionmaking, suggest
Process More              some standards that should exist for seeking
    Effective             decisions, and propose some solutions for
                         publicizing decisions once made.
3. Ensuring Data Accuracy Establish standards for accuracy, and provide
and                       tools for people to use in dealing with PBS data.
    Appropriate Use
4. Business Process       Address approaches ranging from process
Integration and           improvementse.g., billing processto systems
    Simplification        improvementse.g., software standardizationto
                         correction of current data and prevention of
                         future data corruption.
5. Motivational Issues    Suggest various rewards/consequences and
With Data and             approaches to motivate users and customers to
    Systems               clean up and maintain data in all PBS systems.
6. Overcoming and         Identify regional variances, and decide which
Capitalizing Upon         variances are problems and which are not.
    Regional Variances
Source:  PBS Go Team implementation paper.

The team developed the education and involvement
action plan to a higher degree of detail than the
other five plans due to the extreme time
constraints for delivering results on these
issues. The PBS Commissioner advised us that under
this action plan, as of September 23, 1999, an
intense training effort, called Educate 2000, was
completed in each region and the central office on
the effects of full implementation of New Pricing
in October 1999. Also, according to the
Commissioner, regions began holding customer
workshops in late September to inform them of the
changes in their STAR bills and provide updates on
Occupancy Agreements.  Further, he said that a
Customer Advisory Board has been established, and
its first meeting is scheduled for October 29,
1999.

For action plans two, five, and six, team leaders
have been appointed. Action plans three and four
are being run by the Office of the Chief
Information Officer and were delayed while a new
Chief Information Officer was appointed. A new
Chief Information Officer was appointed in July
1999.

PBS is also in the final stages of hiring a
contractor to develop and document a quality
control system that validates real estate and
billing data residing in STAR. The contractor will
provide on-site support to regions in such areas
as developing diagnostic reports. PBS expects to
award the contract in early fiscal year 2000.

Agency Comments and Our Evaluation
     On September 23, 1999, the PBS Commissioner
provided written comments on a draft of this
report. He stated that the report is largely
factual and balanced and does a good job of
capturing the magnitude of the effort undertaken
to move GSA and PBS from 1970s batch process
computing to a modern relational database
platform. In the comments, the Commissioner also
briefly summarized (1) PBS' transition from its
1970s system to STAR; (2) the cultural changes
that drove the need to provide its customers with
more reliable information and to measure its own
performance in hard, quantifiable terms as
envisioned by the Government Performance and
Results Act; and (3) actions being taken to
improve the data in STAR and elsewhere, including
a spatial validation effort using computer-
assisted drawings and the development of a quality
control system to validate real estate and billing
data residing in STAR. He also provided technical
clarifications that have been included in this
report where appropriate. The Commissioner's
letter is reproduced in appendix III.

We are sending copies of this report to Senator
Ben Nighthorse Campbell, Chairman, and Senator
Byron L. Dorgan, Ranking Minority Member,
Subcommittee on Treasury and General Government,
Senate Committee on Appropriations; Senator George
V. Voinovich, Chairman, and Senator Max S. Baucus,
Ranking Minority Member, Subcommittee on
Transportation and Infrastructure, Senate
Committee on Environment and Public Works;
Representative Jim Kolbe, Chairman, and
Representative Steny H. Hoyer, Ranking Minority
Member, Subcommittee on Treasury, Postal Service,
and General Government, House Committee on
Appropriations; the Honorable David J. Barram,
Administrator, GSA; and to others upon request.

If you have any questions regarding this report,
please call me or Ron King at (202) 512-8387. Key
contributors to this assignment were Tom Keightley
and Mike Yacura.

Bernard L. Ungar
Director, Government Business
 Operations Issues

_______________________________
1 Data Issues in PBS, PBS, April 1999.
2 Galaxy is a program that is the primary tool
used to manage the rental-of-space account. Galaxy
will receive source data from STAR. A proposal to
enhance Galaxy is currently under review.  A
statement of work to accomplish this objective is
currently being developed.
3 Of the approximately 46 problems identified by
users, we combined 9 problems into 4 generic
problems because of their similarity. We then
reported only those problems identified by 10 or
more users, except in 1 case. All four of the
generic problems were included in our report.
4 One person identified the problem that STAR
could not record the rent revenues from the
National Antenna Program.
5 Through New Pricing, PBS has introduced a new
way of calculating the rent that client agencies
must pay. Through New Billing, PBS has changed its
billing process, such as billing on a monthly
basis instead of quarterly and making the reality
specialists responsible for billing. The Occupancy
Agreement Tool is a new software package that is
to be used to document agreements on such things
as square footage and cost between PBS and its
client agencies.
6Zero square foot leases will also cover other non-
space revenue producing items, such as docks and
wareyards.
7 Data Issues in PBS.

Appendix I
Definitions of STAR Help Desk Ticket Types
Page 23GAO/GGD-00-12 STAR-Program for Tracking and
Managing Real Property
Conversion Issues
Problems or situations pertaining to data that
have been converted from the old system.

Database Change
Requests to have data changed within the database,
requires written approval.

Enhancement-Application
     Requests to change the software.

Enhancement-
Ad Hoc Report
     Requests to change an existing ad hoc report
or add a new ad hoc report.

Enhancement-Report Standard
     Requests to change an existing standard
report or add a new standard report.

Inquiry
     Any question that is being asked to gather
information.

Inquiry Reports
     Calls specifically related to questions about
reports.

Maintenance-Nonscheduled
     Requests for maintenance for unexpected
problems.

Maintenance-Scheduled
     Issues regarding the monthly maintenance and
operation schedule.

Policy and Procedures
     Questions concerning policies and procedures.

Rent Bill
     Requests regarding a rent bill or the rent
billing process.

Security Request
     Requests on any security issue coming through
the Help Desk.

Table Update
     Request to change information in a table that
supports the System for Tracking and Administering
Real Property (STAR).

Training
     Requests that can be solved or partially
solved by additional training.

Trouble
     Calls about any part of the system the user
may be having problems with, except reports,
including bugs and essential nonfunctioning
software.

Trouble Reports
     Calls about any problems the user may be
having regarding reports.

Appendix II
Data Issues Identified by PBS' Go Team and Its
Recommendations
Page 25GAO/GGD-00-12 STAR-Program for Tracking and
Managing Real Property
Type of issue             Recommendations
Management's              Establish national policies, answer questions, and
Responsibilities          resolve disputes in a timely manner.
                         Change the Central Office culture of "Information
                         is power so I won't share it."
                         Ensure that changed business practices and new
                         tools are adopted at the working level.
                         Prepare employees for the business practices and
                         system changes and emphasize the importance of
                         needed training.
                         Take a proactive position regarding data integrity
                         and commit the resources needed to get the job
                         done.
                         Resolve issues quickly, so employees do not
                         disengage completely from the issue, making it
                         even worse.
                         Instill in all employees a big picture perspective
                         to avoid the "That's not my job attitude."
                         Become familiar with the systems the employees
                         must use on a daily basis in order to appreciate
                         the effort needed to learn the system.
                         Address, buy-into, and communicate a policy on
                         data input by contractors.
                         Close the gaps that exist between the
                         organizational vision of upper management and the
                         changes that need to take place in business
                         processes and roles.
                         Decide and publish Central Office roles and
                         organizational structure on who ultimately decides
                         what. Establish accountability for business
                         decisions.
                         Managers must see the advantages of acting
                         proactively-this must become a core value in PBS.
                         Establish a management information steering
                         committee that will establish policy, fiscal
                         controls and resolve conflicts.
                         Start having regular meetings to go over project
                         data.
Accountability and        Job descriptions should include data integrity.
Ownership                 Competency testing and certification should be
                         established for employees who need to understand
                         New Pricing, Occupancy Agreements, and STAR.
                         Performance ratings should address the issue of
                         employee proficiency in the use of tools and
                         databases required in their specific job
                         functions. Employees should be evaluated on the
                         integrity of their data and the promptness and
                         accuracy of their input.
                         Deal with the perception that STAR and Computer-
                         aided Design are wonderful systems that absolve us
                         of our individual responsibility to review, track,
                         and know our data.
                         If individual specialists are to be responsible
                         for their data, then any data manipulation that
                         takes place system-wide must be coordinated and
                         communicated.
                         Stop continually shifting employees from one area
                         to another.
                         Decide how much access to propriety data our
                         contractors can/should have.
Training and Tools        Need to make sure that everyone who uses STAR gets
                          the right kind of training.
                          Cross training opportunities between business
                          lines should be encouraged.
                          More changed business practices training should be
                          scheduled.
                          Develop systems that can get information to the
                          people who need it.
                          Set up recurring training/certification programs.
                          Expand STAR training beyond realty specialists.
                          Conduct more billing training, emphasizing how and
                          when to do billing adjustments.
                          Plan just-in-time training and train internally
                          before taking it to the customers.
                          Regional business lines need to identify their
                          areas for immediate training.
                          Training for affected personnel must be mandatory
                         and enforced by directors, supervisors, and team
                         leaders.
                         Look at integrated training of multiple
                         tools-teach STAR, Occupancy Agreements, and the
                         client billing policy as an integrated training
                         session.
                         Regions need to take advantage of the available
                         training resources recently procured by the STAR
                         national training team.
Training and Tools        Regional "Power Users" should be utilized in
(Cont.)                   training and instructing other regional users in
                          some type of regular scheduled sessions to keep
                          users up to date on changes and enhancements to
                          the databases and internal processes.
                          Identify any strong regional training programs as
                          best practices, and have the rest of PBS learn
   Medium/long term       from them.
actions                   
                             Some type of evaluation or testing of users
                          needs to be developed to ensure that they
                          understand the database(s) and process required
                          for them.
   Customer training and     A training infrastructure should be developed
marketing                 in each region that provides for regularly
                         scheduled sessions on all of the various databases
                         utilized by GSA. Training sessions should be
                         interactive.
                         
                            Develop national, regional, and local
                         dialogs/workshops with our customers so they fully
                         understand new pricing, billing, and the changes
                         coming to existing assignments before they arrive
                         October 1, 1999.
Reports and Measures      Develop meaningful performance measures for data
                         integrity. Any measure that is adopted should
                         identify the magnitude of the data problems and
                         encourage additional accountability and ownership.
                         Possible performance measures for data include the
                         following:
                            Number of exceptions/flags generated for each
                         region in STAR.
                            Number and dollar value of chargebacks.
                            Number of unopened work queues over some number
                         of days old in STAR.
                            Develop a workload measure that would help
                         management determine when to deploy additional
                         resources, if possible.
                            Number of billing adjustments.
                            Number of customer complaints with response
                         times for resolution.

Appendix III
Comments From the General Services Administration
Page 27GAO/GGD-00-12 STAR-Program for Tracking and
Managing Real Property

*** End of Document ***