General Services Administration: STAR-PBS' New Program for Tracking and
Managing Real Property (Letter Report, 10/12/1999, GAO/GGD-00-12).
Pursuant to a congressional request, GAO reviewed the Public Buildings
Service's (PBS) System for Tracking and Administering Real Property
(STAR), focusing on: (1) the functions STAR performs; (2) whether users
were having problems using STAR; (3) the steps PBS had taken or is
taking to address any user problems; and (4) the actions PBS had taken
or plans to take to ensure that STAR data are accurate, reliable, and
consistent.
GAO noted that: (1) STAR is a real estate inventory management software
application that maintains data on projects, leases, buildings, and
space assignments--that is, who is in what space--in an integrated
database environment; (2) the data in STAR are used to generate bills to
clients for their assigned space, develop budget plans, track and manage
leases, and evaluate performance; (3) STAR's input/output functions
include: (a) direct on-screen access to the data; (b) the ability to
update the data; (c) access to standard reports; and (d) the ability to
sort data and create ad hoc reports; (4) users identified problems with
STAR as well as benefits derived from using the system; (5) the 128
users GAO interviewed identified 18 types of problems that they had
experienced using STAR; (6) these problems included complaints such as
taking too long to enter data and having too many steps or screens; (7)
according to the STAR project manager, all of the problems that were
reported to GAO by users had been reported to the STAR project team
through the contractor-operated STAR Help Desk; (8) also, STAR users had
submitted nearly 8,000 requests, which were recorded on service tickets,
to the Help Desk between October 1997 and August 1999; (9) these
tickets, many of them concerning the same issue, identified problems
that users reported having when working with STAR as well as STAR
enhancement suggestions and other issues related to STAR's operations;
(10) further, 116 of the 128 users GAO interviewed saw at least 1
benefit from using STAR, such as data being current and readily
available; (11) the STAR Help Desk and the PBS Change Review Team were
set up to solve problems identified by STAR users and to resolve other
issues with STAR; (12) by the end of November 1999, changes and
enhancements will have resulted in eight versions of STAR; (13) further,
additional contract help had been provided to assist the regions with
STAR problems and training; and (14) PBS has taken various actions to
try to ensure that STAR has accurate, reliable, and consistent data: (a)
STAR has been designed with built-in features to help ensure the
reliability of some data elements; (b) PBS ran edits against the data
before they were transferred to STAR and ran additional edits after data
were transferred; (c) PBS initiated an effort to validate all space
assignments; (d) PBS conducted a special study and produced a report
that identified data problems and made recommendations for solving the
problems; (e) a team was set up to determine how to implement the
report's recommendations; and (f) PBS is developing a quality control
system.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: GGD-00-12
TITLE: General Services Administration: STAR-PBS' New Program for
Tracking and Managing Real Property
DATE: 10/12/1999
SUBJECT: Computer software
Management information systems
Information resources management
Data integrity
Federal property management
IDENTIFIER: PBS System for Tracking and Administering Real Property
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO report. Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
** A printed copy of this report may be obtained from the GAO **
** Document Distribution Center. For further details, please **
** send an e-mail message to: **
** **
** **
** **
** with the message 'info' in the body. **
******************************************************************
United States General Accounting Office
GAO
Report to Congressional Requesters
October 1999
GAO/GGD-00-12
GENERAL SERVICES ADMINISTRATION
STAR-PBS' New Program for Tracking and Managing
Real Property
Ordering Information
The first copy of each GAO report and testimony
is free. Additional copies are $2 each. Orders
should be sent to the following address,
accompanied by a check or money order made out
to the Superintendent of Documents, when
necessary. VISA and MasterCard credit cards are
accepted, also. Orders for 100 or more copies to
be mailed to a single address are discounted 25
percent.
Order by mail:
U.S. General Accounting Office
P.O. Box 37050
Washington, DC 20013
or visit:
Room 1100
700 4th St. NW (corner of 4th and G Sts. NW)
U.S. General Accounting Office
Washington, DC
Orders may also be placed by calling (202) 512-
6000 or by using fax number (202) 512-6061, or
TDD (202) 512-2537.
Each day, GAO issues a list of newly available
reports and testimony. To receive facsimile
copies of the daily list or any list from the
past 30 days, please call (202) 512-6000 using a
touch-tone phone. A recorded menu will provide
information on how to obtain these lists.
For information on how to access GAO reports on
the INTERNET, send e-mail message with "info" in
the body to:
[email protected]
or visit GAO's World Wide Web Home Page at:
http://www.gao.gov
United States
General Accounting Office
Washington, D.C. 20548-0001
Official Business
Penalty for Private Use $300
Address Correction Requested
Bulk Rate
Postage & Fees Paid
GAO
Permit No. G100
(240329)
Contents
Page 201GAO/GGD-00-12 STAR-Program for Tracking and Managing Real
Property
Letter 1
Appendix I 22
Definitions of STAR Help
Desk Ticket Types
Conversion Issues 22
Database Change 22
Enhancement-Application 22
Enhancement- Ad Hoc Report 22
Enhancement-Report Standard 22
Inquiry 22
Inquiry Reports 22
Maintenance-Nonscheduled 22
Maintenance-Scheduled 22
Policy and Procedures 23
Rent Bill 23
Security Request 23
Table Update 23
Training 23
Trouble 23
Trouble Reports 23
Appendix II 24
Data Issues Identified
by PBS' Go Team and Its
Recommendations
Appendix III 26
Comments From the
General Services
Administration
Tables Table 1: Description of STAR's Seven 6
Major Modules
Table 2: Identified User Problems 8
Table 3: PBS' Responses to Examples of 9
User Problems
Table 4: Help Desk-Identified User 11
Problems, Ranked by the Number of
Service Tickets Received for the
Problem
Table 5: Examples of Go Team- 16
Identified Issues and Recommendations
Table 6: PBS' Prioritized Action Plans 16
by Priority
Abbreviations
CRT Change Review Team
GSA General Services Administration
IG Office of Inspector General
PBS Public Buildings Service
PROMPT Permanent Record of Managed Property
Transactions
STAR System for Tracking and Administering
Real Property
B-281769
Page 18GAO/GGD-00-12 STAR-Program for Tracking and
Managing Real Property
B-281769
October 12, 1999
The Honorable Bob Franks
Chairman
The Honorable Robert Wise, Jr.
Ranking Democratic Member
Subcommittee on Economic Development,
Public Buildings, Hazardous Materials,
and Pipeline Transportation
Committee on Transportation and Infrastructure
House of Representatives
This report responds to your request for a review
of the General Services Administration's (GSA)
Public Buildings Service's (PBS) System for
Tracking and Administering Real Property (STAR).
Your Subcommittee was concerned about problems
that were being reported to it concerning STAR. As
agreed with the Subcommittee, our objectives were
to determine
� the functions STAR performs;
� whether users were having problems using
STAR;
� the steps PBS had taken or is taking to
address any user problems; and
� the actions PBS had taken or plans to take to
ensure that STAR data are accurate, reliable, and
consistent.
As agreed with your offices, we did not review the
technical aspects of STAR because GSA's Office of
Inspector General (IG) is reviewing the technical
and systemic issues relating to STAR. The IG
expects to issue its report by the end of 1999.
Results in Brief
STAR is a real estate inventory management
software application that maintains data on
projects, leases, buildings, and space
assignmentsthat is, who is in what space-in an
integrated database environment. The data in STAR
are used to generate bills to clients for their
assigned space, develop budget plans, track and
manage leases, and evaluate performance. STAR's
input/output functions include
� direct on-screen access to the data,
� the ability to update the data,
� access to standard reports, and
� the ability to sort data and create ad hoc
reports.
Users identified problems with STAR as well as
benefits derived from using the system. The 128
users we interviewed identified 18 types of
problems that they had experienced using STAR.
These problems included complaints such as taking
too long to enter data and having too many steps
or screens. According to the STAR project manager,
all of the problems that were reported to us by
users had been reported to the STAR project team
through the contractor-operated STAR Help Desk.
Also, STAR users had submitted nearly 8,000
requests, which were recorded on service tickets,
to the Help Desk between October 1997 and August
1999. These tickets, many of them concerning the
same issue, identified problems that users
reported having when working with STAR as well as
STAR enhancement suggestions and other issues
related to STAR's operations. Further, 116 of the
128 users we interviewed saw at least 1 benefit
from using STAR, such as data being current and
readily available.
The STAR Help Desk and a PBS Change Review Team
(CRT) were set up to solve problems identified by
STAR users and to resolve other issues with STAR.
By the end of November 1999, changes and
enhancements will have resulted in eight versions
of STAR. Further, additional contract help had
been provided to assist the regions with STAR
problems and training.
PBS has taken various actions to try to ensure
that STAR has accurate, reliable, and consistent
data.
� STAR has been designed with built-in features
to help ensure the reliability of some data
elements. For example, work queues were to
automatically generate requests for such things as
approvals of leases, corrections, or notifications
of missing information; work queues could also be
used manually to send information.
� PBS ran edits against data before they were
transferred to STAR and ran additional edits after
data were transferred.
� PBS initiated an effort to validate all space
assignments.
� PBS conducted a special study and produced a
report that identified data problems and made
recommendations for solving the problems. 1
� A team was set up to determine how to
implement the report's recommendations. PBS has
started implementing some of these
recommendations.
� PBS is developing a quality control system.
Background
With the introduction of STAR and other changes, a
major cultural change has occurred in GSA's
facilities management. According to the PBS
report,
"The simultaneous launches of New Pricing, STAR,
Occupancy Agreements, and a host of other re-
engineered business line processes and software
constitutes the most significant change to the way
PBS does its business since the creation of the
Federal Buildings Fund."
The mission of PBS is to effectively manage over
8,000 owned and leased federal real properties in
the United States. PBS had previously used a
computer program called PLUS+ to track and manage
its real properties. In 1994, PBS' Central Office
decided that the systems it used to support its
mission were antiquated and needed to be replaced.
New software was needed to support real property
inventory, leasing, space assignments, and billing-
related activities. PBS did market research
analyses from March 1994 through October 1995,
evaluating various real estate software packages.
In March 1994, PBS began its initial efforts to
obtain new software. It published a notice in the
Commerce Business Daily seeking sources to supply
software products to support management operations
for leased and owned public buildings. Two
responses to this notice were received. However,
PBS determined that neither of these sources met
its functional needs.
PBS also hired a contractor to identify any
existing property management software that might
meet its needs. The contractor developed a list of
400 software products that it said spanned all
facets of PBS' business functions. Using PBS'
needs as criteria, this list was reduced to 36
products, then to 6 products for further study.
However, PBS concluded that none of these products
was capable of satisfying all of its minimum
requirements, such as the ability and capacity to
concurrently manage over 8,000 owned and leased
properties.
During discussions with real estate companies and
Fortune 500 companies about real estate practices
and procedures, PBS discovered a proprietary
system called Permanent Record of Managed Property
Transactions (PROMPT). This system had been
implemented by AT&T during the summer of 1994.
According to PBS officials, PBS ran a 4-month
pilot test in 1995 on PROMPT. During this test, 30
percent of PBS' data were diagrammed and selected
data elements on leases and space assignments, as
well as other areas, were entered into the system.
The test was completed in October 1995. PBS
concluded that although PROMPT could meet its
needs, the data were not in the format that it
wanted.
In July 1996, an acquisition plan for STAR was
approved by GSA's Associate Administrator for
Acquisition Policy. This plan called for a sole
source contract with AT&T for the use of PROMPT,
which PBS renamed STAR. On January 17, 1997, the
contract, which included a license agreement
granting use of PROMPT in perpetuity unless
canceled by the licensee, was signed with AT&T.
The contract was for labor hours on a best-effort
performance basis to convert PROMPT version 2.7
into STAR, which met PBS' requirements. The base
contract was for $8.3 million. Of this amount,
$4.3 million was for the perpetual right to use
the national license; the remaining $4.0 million
was for software maintenance, a STAR reference
guide, training courses and manuals, technical
support services, hotline support, and travel. The
contract also included several options. If all of
these options were exercised, the total cost of
the contract would have been $17.0 million. On
October 19, 1998, the quantities in one of the
options-option four-were increased adding 1,000
users, 310 training classes, 2,900 sets of
training manuals, and 23,200 staff hours of
technical support. This raised the value of the
contract to $21.8 million, which was an increase
of $4.8 million.
PBS began rolling out STAR in October 1997, and it
was available in all regions by January 1998. The
first rent bills generated by STAR were sent to
agencies in July 1998.
PBS officials said that the introduction of STAR,
among other things, has led to a cultural change
that has had many effects in PBS. The officials
noted that three changes particularly, affected
the realty specialists who maintain STAR. First,
realty specialists are now required to enter the
data directly into the STAR system; second, they
are to be held directly responsible for data
accuracy; and third, they are responsible for
billing the client agencies. In the past, the
realty specialists filled out data entry sheets
and data entry clerks entered the data. It was not
clear at that time who was responsible for the
data accuracy. Further, in the past, realty
specialists were not involved in the actual
billing of the clients.
Scope and Methodology
To determine what functions STAR performs, we
observed how STAR operates; interviewed officials
responsible for the development of STAR; and
reviewed contract documents, training manuals, and
the draft desk reference manual for STAR.
To identify problems that users were experiencing
using STAR, we judgmentally selected four regions,
two of which were identified by PBS officials as
having problems implementing STAR, one that was
the lead region for STAR, and one from which
congressional staff had received complaints. We
also included the headquarters unit to get its
perspective on STAR. PBS officials identified the
users of STAR at these locations, of which realty
specialists and realty assistants were identified
as the major users. We then interviewed 128 users,
81 of which were realty specialists or realty
assistants, in the 5 locations and asked them
about the problems they have had using STAR. Our
results cannot be projected to the total universe
of STAR users.
Further, we reviewed Help Desk reports that
organized user complaints about STAR into subject
categories. Because of the way that the issues
were categorized in the report and the actual
number of items reported to the Help Desk-almost
8,000-we were not able to directly compare our
interview-identified problems with those reported
to the Help Desk. However, we discussed the
problems that users had identified in our
interviews, and problems that had been reported to
the Help Desk, with the STAR project manager.
Further, we reviewed some Help Desk reports that
showed specific problems identified.
To identify the steps that PBS has taken or is
taking to deal with user problems, we interviewed
officials who were responsible for addressing STAR
problems and reviewed documents on changes being
made to correct problems and improve STAR.
Finally, to determine what PBS has done or plans
to do to ensure STAR data integrity, we
interviewed officials and reviewed plans
addressing PBS' efforts to ensure that STAR data
are accurate, reliable, and consistent. We did not
independently assess the accuracy, reliability,
and consistency of STAR data.
We did our work at GSA/PBS Headquarters and
the National Capital Region in Washington, D.C.;
the Southeast Sunbelt Region in Atlanta, GA; the
New England Region in Boston, MA; and the Mid-
Atlantic Region in Philadelphia, PA. Our work was
done in accordance with generally accepted
government auditing standards. We requested
comments on a draft of this report from the
Administrator of GSA. GSA's written comments are
discussed at the end of this letter.
STAR-A System for Tracking and Administering Real
Property
STAR is a real estate inventory management
software application that maintains data on
projects, leases, buildings, and space
assignmentsthat is, who is in what space-in an
integrated database environment. The data in STAR
have been used to generate bills to clients for
their assigned space, develop budget plans, track
and manage leases, and evaluate performance.
STAR's input/output functions include
� direct on-screen access to the data,
� the ability to update the data,
� access to standard reports, and
� the ability to sort data and create ad hoc
reports.
To develop the fiscal year 2001 budget, STAR data
are being used to support the development of the
over $5.5 billion rental revenue estimate and as
the basis for the Galaxy system's roughly $3.1
billion estimate of PBS lease expenditures.2
STAR has seven major modules: (1) Space
Management, (2) Client Billing, (3) Project
Administration, (4) Billing, (5) Work Queue, (6)
Administration, and (7) Reports. Table 1 describes
each of these modules.
Table 1: Description of STAR's Seven Major Modules
Module Description
Space Management Supports the recording and processing of data
related to buildings, space, occupants, and
occupancy. Data stored include building location,
square footage, lease terms, types of space,
occupant, and payment schedules.
Client Billing Supports the development, documentation, and
implementation of specific occupancy agreements
with GSA clients. The data stored include client
terms, space identification, and client-specific
rates.
Project Administration Supports the tracking of actions taken to satisfy
a federal agency's space request. These actions
include new lease acquisitions, consolidations,
and constructing a new room or expanding space.
Billing Generates monthly bills to client agencies on the
basis of their space assignment; supports
miscellaneous adjustments, such as debits or
credits, to bills that have already been sent to
the client. The data stored include rent to be
billed and miscellaneous debits and credits.
Work Queue Supports an on-line to do list for realty services
users and for interoffice communications. It has
five types of work queues-approval, copy,
corrections, action required, and notification.
Administration Manages administrative tasks related to STAR.
Accessible only by contractor.
Reports Supports standardized reports on space management,
project administration, and billing. Also provides
report writer for writing ad hoc reports.
Source: GAO analysis of PBS documents.
Each of these modules has data entry screens. For
example, the Space Management module has a
Building Details screen for entering specific
building information and a Lease Details screen
for entering specific lease information. STAR is
supported by a contractor-operated Help Desk to
aid users if they have problems with the STAR
modules and screens.
PBS identified the following benefits of STAR.
� Movement of data and information systems
closer to the end user.
� Direct access to project, lease, building,
and space assignment data.
� Electronic notification of expiring leases,
unassigned space, and other information.
� Integration of information technology into
business practices.
� Direct input eliminates the need for data
transcribers and precludes duplicative
keypunching.
� Easy report generation and exporting of data
to another program.
� No need to memorize data element numbers, it
uses English.
� Tutorials on policies and procedures can be
placed into help screens of the system.
� Will allow for direct update of agency
information.
� Improves corporate competitiveness.
In the future, PBS hopes to expand STAR's
capabilities by adding more features, such as
direct access by an agency to view the information
that STAR maintains concerning the agency's
assigned space.
Users Identified Problems and Benefits for STAR
The 128 users we interviewed identified problems
they had experienced using STAR as well as several
benefits they saw being derived from the use of
STAR.3 Of the problems identified by users, 17
were mentioned by at least 10 of the users. One
problem was mentioned by only one user but was
included in this report because it involves
accounting for rent revenues.4 According to the
STAR project manager, each type of problem that
was reported to us had been reported to the STAR
project team through the STAR Help Desk and logged
into 16 categories. According to the project
manager, the PBS project team has taken actions to
(1) resolve the reported problems that the team
believed were legitimate and (2) add the requested
enhancements that the team believed were
appropriate. Further, 116 of the users we
interviewed saw at least 1 benefit from using
STAR.
Table 2 gives a breakdown of the problems that
individuals we interviewed identified.
Table 2: Identified User Problems
Type of problem Number of
users identifying
problem
Takes long time to enter data, too many steps/screens.a 99
System slow at times, you get system errors. 92
Have to enter the same information repeatedly in different 87
modules because modules are not linked.
Printing requires multiple steps. 73
Not user friendly. 71
Realty specialist can go into any lease in region, data 55
security issues.
STAR will not let you save work until certain information 55
has been filled in.
No history for R620 reports, authorizes payment by 39
finance, only a digest. Realty specialists believed a
history is needed in STAR.
Cannot enter same date twice in the payment schedule 29
screen. This results in the dummying of dates on the
Payment Schedule screen.
Hard-to-use report writer for ad hoc reports because you 27
have to download data to another program module by module.
Cannot access all fields in STAR.a
STAR prompts are not clear. 27
Work queues-You are getting too many that you do not need, 24
getting work queues on items that are not the realty
specialist's responsibility. These waste the user's time.
a
Bills-need to explain the manual adjustments on the bill. 22
Need to calculate joint use space to the same number of
decimal places for the bill as STAR does internally.
Billing adjustments cumbersome to make.a
Once a change has been approved, you cannot correct an 21
error without initiating a database change through the
STAR Help Desk.
STAR lacks a Desk Reference Manual. 20
Realty specialist does not use STAR often enough to become 19
proficient.
STAR lacks a standard report called a combined space 13
report.
Cannot enter rent for National Antenna Program.b 1
aWe have combined nine identified problems into
four generic problems.
bThis problem was included because it involves
accounting for rent revenues.
Source: Compiled by GAO from interviews with STAR
users.
The project manager said that the STAR project
team had taken action or action was being taken on
some of the identified problems. Other problems
relate to inherent parts of STAR that cannot be
easily fixed or occurred because of decisions made
during the STAR developmental process. Further,
according to another PBS official, user problems
may also be a reaction to learning new business
processes, such as New Pricing, and other changes.
Table 3 contains PBS' responses to examples of the
problems we identified through interviews.
Table 3: PBS' Responses to Examples of User
Problems
Problem PBS response to the problem
Printing requires STAR has excellent data manipulation capabilities
multiple steps. but the users are not used to downloading in order
to print. Also, it would be very expensive to make
STAR print directly from the screen.
STAR will not let you To ensure that information was entered in STAR, it
save work until was designed not to let you save work until
certain information has certain data entry blanks have been filled in.
been filled in.
Realty specialist can go All realty specialists in a region can go into all
into any lease leases, if they have been granted access to
in region, data security leases. It will be up to each region to decide if
issues. it wants to limit access to a greater extent. It
can be done.
No history for R620 It was decided during development not to have a
reports, authorizes history of the R620 reports in STAR. There is a
payment by finance, only digest of the reports on STAR and a hard copy of
a digest. the individual R620s can be located in the lease
Realty specialists file.
believed a history
is needed in STAR.
Work queues-You are Work queues serve an important role in STAR by
getting too keeping users informed of problems and other
many that you do not issues. The work queue process is being
need, getting streamlined in a release scheduled for November
work queues on items that 1999.
are not
the realty specialist's
responsibility.
These waste the user's
time.
Once a change has been Because of the approval process in STAR for making
approved, changes, you have to go to the Help Desk for a
you cannot correct an database change to correct an error once a change
error without has been approved.
initiating a database
change through
the STAR Help Desk.
Source: Table 2 of this report and interview with
the STAR project manager.
The STAR developmental process also affected STAR,
particularly in relation to the user friendliness
problem. According to the project manager, the
time frame for developing STAR covered from
January 1997 until October 1997. A different team
developed each module in STAR, and communications
among the teams may not have been effective
because of the time constraints. The project
manager thought that the lack of communications
may have resulted in inconsistencies and
duplication among modules. Further, she said that
STAR's development was hindered because during the
development phase it was learned that STAR would
have to handle both the old pricing policy for
client space and the new pricing policy for client
space, not just the new pricing policy as
originally planned. This resulted in having to
change STAR's design. Also, she suggested that PBS
may have released STAR to the regions before it
was ready, resulting in some of the problems that
have been identified by users.
The project manager said that a cultural
change occurred for realty specialists that
probably affected their view of STAR. Before STAR
and other business changes, realty specialists
were not responsible for entering and maintaining
data using a computer or for billing client
agencies for their assigned space. With the
introduction of STAR, New Pricing, New Billing,
and the Occupancy Agreement Tool, additional
workloads were placed on the realty specialist.5
Also, each initiative had its own implementation
problems that the realty specialists had to deal
with in doing their jobs.
The nearly 8,000 service tickets handled by the
Help Desk reflect PBS-wide problems and other
issues reported by users. PBS had classified these
tickets into 16 different categories. According to
PBS officials, the same problem may have been
reported on many different service tickets. Table
4 shows the Help Desk's breakdown of these service
tickets received and the status of the tickets by
the 16 categories, as of August 12, 1999.
Definitions of the categories can be found in
appendix I.
Table 4: Help Desk-Identified User Problems,
Ranked by the Number of Service Tickets Received
for the Problem
Category of problem Number of Closed Open
service
tickets
received
Troublea 3,540 3,496 44
Database change 2,080 2,033 47
Inquiry 659 652 7
Enhancement-application 590 429 161
Rent bill 364 359 5
Trouble reports 189 178 11
Enhancement-ad hoc reports 94 50 44
Inquiry reports 90 89 1
Enhancement-report standard 88 61 27
Security request 78 78 0
Table update 59 55 4
Policy and procedures 40 33 7
Training 28 27 1
Maintenance-nonscheduled 14 14 0
Conversion issues 11 11 0
Maintenance-scheduled 4 3 1
Total 7,928 7,568 360
aOriginally, this category was used as a catchall
until other categories were defined.
Source: Help Desk contractor document.
Trouble tickets refer to any part of the system
that the user may be having problems with, except
for the report function, including essential
nonfunctioning software. The contractor is
responsible for fixing the problems associated
with all tickets categorized as trouble at no cost
to PBS. The three enhancement categories contain
requests to make changes to STAR. Any changes to
STAR resulting from these tickets categorized as
enhancements have to be negotiated between PBS and
the contractor, with PBS' paying for the change.
The CRT leader said the team had processed 511
enhancement tickets, of which about 44 were still
open. The official said that this tally and the
contractor's tally of tickets categorized as
enhancements, as shown in table 4, do not match
because the two had not been recently reconciled.
She said the contractor's system probably had some
items in the enhancement categories that were not
enhancements and CRT had probably closed some
other tickets that were not yet recorded in the
contractor's computer database as closed.
While the 128 users we interviewed reported
problems with STAR, 116 of them saw some positive
benefit from STAR. Users identified the following
as the top five benefits.
� Information is current and readily
accessible.
� Can see that changes to data have been
accepted by the system.
� STAR makes it easy to make minor changes.
� STAR directly controls billing and lease
payments.
� It is good to have control over the data for
which you are responsible.
PBS' Efforts to Improve STAR
As previously mentioned, as part of the STAR
project, a contractor-operated Help Desk and PBS'
CRT were set up to solve STAR problems. The Help
Desk and CRT have handled about 8,000 service
tickets, of which 95 percent have been closed.
Further, other actions, such as additional
contract help, are now available to the regions to
help with STAR problems and training.
The Help Desk is to record user complaints,
enhancement suggestions, or other issues
concerning the use of STAR on service tickets. CRT
reviews and determines how to handle enhancement
tickets and trouble tickets that require a fix
through a new release of STAR. Guidance on the
review process states that no action on a service
ticket usually means that a ticket resulted from a
user's momentary frustration with the system, a
passing thought of a user, or that other users had
already identified the problem.
If the enhancement or trouble service ticket is
accepted, CRT is to edit and define the ticket to
clearly identify the issue. CRT is also to assign
each of these tickets a priority-high, normal, or
low. Working with the contractor, CRT then decides
how to implement any change considered needed. By
the end of November 1999, eight versions of STAR
will have been issued. According to PBS reports,
STAR releases 3.0, and 4.0, combined, have about
40 enhancements. Version 4.0, which is to be
released by the end of November 1999, for example,
is to address the problems of (1) having to enter
the same information repeatedly in different
modules because modules are not linked and (2) not
having the ability to enter the same date twice in
the Payment Schedule screen. A category named
"zero square foot leases" is also to be added to
STAR to cover the national antenna program in a
release scheduled for the spring of 2000.6 As of
August 12, 1999, additional enhancement tickets
are still open, although the exact number was
unclear because PBS had not reconciled the data
between the CRT and the computer-based Help Desk
report. Other efforts to improve STAR include the
following:
� A desk reference manual for STAR is to be
issued in November 1999. This manual should help
with the problem of the need to have better
prompts that make where to go clearer. The users
should be able to look up what to do.
� A report is under development that would be
similar to the combined space report, which
provided such information as building address,
square footage, and lease-specific information.
� Renewed emphasis has been placed on examining
a preview of the bill in STAR before actual bills
are sent.
� An ad hoc exception report on bills is being
sent to the regions each month to catch errors
more quickly.
� PBS has set a specific accuracy goal that
targets a specific data problem for improvement
and rewards regions for meeting the improvement
goal. We were told that this data accuracy goal
would be changed on a regular basis. It currently
focuses on the issue of chargebacks and missing
rates.
PBS' Efforts to Ensure Accurate, Reliable, and
Consistent Data in STAR
PBS knows that it has data problems, and these
problems have been well documented. Examples of
the problems follow:
� The GSA IG concluded that over half of the
space assignment drawings are off by more than 5
percent.
� Price Waterhouse Coopers, in its audit of
GSA's fiscal year 1998 financial statements, found
a reportable condition that improvements are
needed in the controls over the integrity of rent
and leasing data used to manage the Federal
Buildings Fund.
� Exception reports and queries indicate that
there is significant regional variation in how
well data in STAR are maintained, how well PBS
produces occupancy agreements, and the accuracy of
rent billings. PBS had to do extensive
reconciliation of all congressional rent bills,
the Federal Bureau of Investigation had to tell
PBS that it was not being billed for $3.0 million
worth of assignments, and 40 percent of the first
wave of Census Bureau offices that PBS had
delivered had not been billed as of January 1999.
PBS said it has taken various actions to ensure
that STAR has accurate, reliable, and consistent
data.
� STAR was designed with built-in features to
help ensure the reliability of some data elements.
� PBS ran edits against the data to be
transferred to STAR and ran additional edits after
data were transferred.
� PBS has an initiative to validate all space
assignments.
� A special study was done that produced a
report identifying data problems and made
recommendations for solving the problems. 7
� A team was set up to determine how to
implement the report's recommendations.
� PBS is developing a quality control system.
Controls have been built into STAR to make the
program provide more accurate, reliable, and
consistent data. For example, STAR has fields that
have to be filled in before you can go to the next
step of your task. This is to ensure that all of
the required data for a task are entered. Another
example is work queues, which automatically
generate requests for such things as approvals of
leases, corrections, or notifications of missing
information and may also be used manually to send
information. This feature is also designed to
automatically notify the budget analyst
responsible for rent expenditures of changes so
that the analyst may anticipate the effect of the
change on expenditures. STAR makes certain
calculations using data entered in the database to
reduce the chance of human error.
During the transition to STAR, a PBS team was
formed to work with each of the teams that was
developing STAR modules to assist in data
transfer. According to the team leader, this team
developed diagnostic edits for each module that
were run on a region-by-region basis to determine
missing or incomplete data before the transfer of
data to STAR. Further, AT&T wrote a conversion
program that included a list of edits to run
against the data once they were transferred to
STAR to identify missing or incomplete data.
Finally, the post-conversion STAR team was tasked
with regularly identifying queries, exception
reports, and flags that are to be added to STAR to
help identify pockets of bad data.
To further ensure accurate, reliable, and
consistent data, PBS established a Go Team in
January 1999 to identify and quantify the real and
perceived problems with inventory data-space
assignment drawings, lease files, assignment
files, rent rates, bills, and information in STAR
and Occupancy Agreements. For example, as
previously mentioned, the GSA IG found half of the
space assignment drawings were off by more than 5
percent. The Go Team found considerable problems
with how well PBS was creating final rent
rates-the proper calculation and application of
fees and other charges. In a spot check of 20
Client Billing Records, the team found minor
errors in 100 percent of the cases. Also, the team
found errors in parking information. In one
region, a discrepancy between total parking and
assigned parking, which should match in STAR, was
found in about 5 percent of the buildings. This
problem could affect revenues. Further, the team
found errors in design gross square feet and gross
square feet, which should match in STAR. In three
regions in particular, substantial variances
between the design gross square feet and the gross
square feet were found. Again, this could affect
revenues.
In the team's April 1999 report, it identified the
causes of known data problems and identified
actions that needed to be taken to fix data
problems. The Go Team report identified the
following as the root causes of data problems:
� the lack of accountability and ownership of
data;
� the lack of clear, consistent, complete
policies associated with PBS' new business
practices;
� incomplete training;
� bad data to begin with;
� PBS' reorganization that moved or de-
emphasized assignment functions;
� migration from the old computer program to
STAR;
� personnel turnover, downsizing, and workload;
and
� changing standards, such as moving from
occupiable square feet to usable square feet to
rentable square feet.
In addition to those actions already under way
that were discussed under the previous section of
this report, the Go Team identified issues that it
said must be addressed to improve data quality.
These issues were
� management responsibilities,
� accountability and ownership,
� training and tools, and
� reports and measures.
Table 5 lists examples of the suggested changes
made by the team. Appendix II lists all of the
suggested changes.
Table 5: Examples of Go Team-Identified Issues
and Recommendations
Type of issue Recommendations
Management's Establish national policies, answer questions, and
Responsibilities resolve disputes in a timely manner.
Change the Central Office culture of "Information
is power so I won't share it."
Ensure that changed business practices and new
tools are adopted at the working level.
Accountability and Job descriptions should include data integrity.
Ownership Competency testing and certification should be
established for employees who need to understand
New Pricing, Occupancy Agreements, and STAR.
Performance ratings should address the issue of
employee proficiency in the use of tools and
databases required in their specific job
functions. Employees should be evaluated on the
integrity of their data and the promptness and
accuracy of their input.
Training and Tools Need to make sure that everyone who uses STAR gets
the right kind of training.
Cross training opportunities between business
lines should be encouraged.
More changed business practices training should be
scheduled.
Reports and Measures Develop meaningful performance measures for data
integrity. Any measure that is adopted should
identify the magnitude of the data problems and
encourage additional accountability and ownership.
Possible performance measures for data include the
number of exceptions/flags generated for each
region in STAR and the number and dollar value of
chargebacks.
Source: Data Issues in PBS, PBS, April 1999.
In May 1999, another team was established to
determine how to implement the first team's
recommendations. The second team developed a set
of six prioritized action plans that include both
temporary stop-gap fixes and permanent solutions.
Table 6 lists the prioritized action plans, in
priority order.
Table 6: PBS' Prioritized Action Plans by Priority
Action plan Purpose
1. Education and Initially educate internally and externally for
Involvement implementation of new billing in October 1999.
Address long-term educational and training needs
associated with data accuracy.
2. Making Decision Address the mechanics of decisionmaking, suggest
Process More some standards that should exist for seeking
Effective decisions, and propose some solutions for
publicizing decisions once made.
3. Ensuring Data Accuracy Establish standards for accuracy, and provide
and tools for people to use in dealing with PBS data.
Appropriate Use
4. Business Process Address approaches ranging from process
Integration and improvementse.g., billing processto systems
Simplification improvementse.g., software standardizationto
correction of current data and prevention of
future data corruption.
5. Motivational Issues Suggest various rewards/consequences and
With Data and approaches to motivate users and customers to
Systems clean up and maintain data in all PBS systems.
6. Overcoming and Identify regional variances, and decide which
Capitalizing Upon variances are problems and which are not.
Regional Variances
Source: PBS Go Team implementation paper.
The team developed the education and involvement
action plan to a higher degree of detail than the
other five plans due to the extreme time
constraints for delivering results on these
issues. The PBS Commissioner advised us that under
this action plan, as of September 23, 1999, an
intense training effort, called Educate 2000, was
completed in each region and the central office on
the effects of full implementation of New Pricing
in October 1999. Also, according to the
Commissioner, regions began holding customer
workshops in late September to inform them of the
changes in their STAR bills and provide updates on
Occupancy Agreements. Further, he said that a
Customer Advisory Board has been established, and
its first meeting is scheduled for October 29,
1999.
For action plans two, five, and six, team leaders
have been appointed. Action plans three and four
are being run by the Office of the Chief
Information Officer and were delayed while a new
Chief Information Officer was appointed. A new
Chief Information Officer was appointed in July
1999.
PBS is also in the final stages of hiring a
contractor to develop and document a quality
control system that validates real estate and
billing data residing in STAR. The contractor will
provide on-site support to regions in such areas
as developing diagnostic reports. PBS expects to
award the contract in early fiscal year 2000.
Agency Comments and Our Evaluation
On September 23, 1999, the PBS Commissioner
provided written comments on a draft of this
report. He stated that the report is largely
factual and balanced and does a good job of
capturing the magnitude of the effort undertaken
to move GSA and PBS from 1970s batch process
computing to a modern relational database
platform. In the comments, the Commissioner also
briefly summarized (1) PBS' transition from its
1970s system to STAR; (2) the cultural changes
that drove the need to provide its customers with
more reliable information and to measure its own
performance in hard, quantifiable terms as
envisioned by the Government Performance and
Results Act; and (3) actions being taken to
improve the data in STAR and elsewhere, including
a spatial validation effort using computer-
assisted drawings and the development of a quality
control system to validate real estate and billing
data residing in STAR. He also provided technical
clarifications that have been included in this
report where appropriate. The Commissioner's
letter is reproduced in appendix III.
We are sending copies of this report to Senator
Ben Nighthorse Campbell, Chairman, and Senator
Byron L. Dorgan, Ranking Minority Member,
Subcommittee on Treasury and General Government,
Senate Committee on Appropriations; Senator George
V. Voinovich, Chairman, and Senator Max S. Baucus,
Ranking Minority Member, Subcommittee on
Transportation and Infrastructure, Senate
Committee on Environment and Public Works;
Representative Jim Kolbe, Chairman, and
Representative Steny H. Hoyer, Ranking Minority
Member, Subcommittee on Treasury, Postal Service,
and General Government, House Committee on
Appropriations; the Honorable David J. Barram,
Administrator, GSA; and to others upon request.
If you have any questions regarding this report,
please call me or Ron King at (202) 512-8387. Key
contributors to this assignment were Tom Keightley
and Mike Yacura.
Bernard L. Ungar
Director, Government Business
Operations Issues
_______________________________
1 Data Issues in PBS, PBS, April 1999.
2 Galaxy is a program that is the primary tool
used to manage the rental-of-space account. Galaxy
will receive source data from STAR. A proposal to
enhance Galaxy is currently under review. A
statement of work to accomplish this objective is
currently being developed.
3 Of the approximately 46 problems identified by
users, we combined 9 problems into 4 generic
problems because of their similarity. We then
reported only those problems identified by 10 or
more users, except in 1 case. All four of the
generic problems were included in our report.
4 One person identified the problem that STAR
could not record the rent revenues from the
National Antenna Program.
5 Through New Pricing, PBS has introduced a new
way of calculating the rent that client agencies
must pay. Through New Billing, PBS has changed its
billing process, such as billing on a monthly
basis instead of quarterly and making the reality
specialists responsible for billing. The Occupancy
Agreement Tool is a new software package that is
to be used to document agreements on such things
as square footage and cost between PBS and its
client agencies.
6Zero square foot leases will also cover other non-
space revenue producing items, such as docks and
wareyards.
7 Data Issues in PBS.
Appendix I
Definitions of STAR Help Desk Ticket Types
Page 23GAO/GGD-00-12 STAR-Program for Tracking and
Managing Real Property
Conversion Issues
Problems or situations pertaining to data that
have been converted from the old system.
Database Change
Requests to have data changed within the database,
requires written approval.
Enhancement-Application
Requests to change the software.
Enhancement-
Ad Hoc Report
Requests to change an existing ad hoc report
or add a new ad hoc report.
Enhancement-Report Standard
Requests to change an existing standard
report or add a new standard report.
Inquiry
Any question that is being asked to gather
information.
Inquiry Reports
Calls specifically related to questions about
reports.
Maintenance-Nonscheduled
Requests for maintenance for unexpected
problems.
Maintenance-Scheduled
Issues regarding the monthly maintenance and
operation schedule.
Policy and Procedures
Questions concerning policies and procedures.
Rent Bill
Requests regarding a rent bill or the rent
billing process.
Security Request
Requests on any security issue coming through
the Help Desk.
Table Update
Request to change information in a table that
supports the System for Tracking and Administering
Real Property (STAR).
Training
Requests that can be solved or partially
solved by additional training.
Trouble
Calls about any part of the system the user
may be having problems with, except reports,
including bugs and essential nonfunctioning
software.
Trouble Reports
Calls about any problems the user may be
having regarding reports.
Appendix II
Data Issues Identified by PBS' Go Team and Its
Recommendations
Page 25GAO/GGD-00-12 STAR-Program for Tracking and
Managing Real Property
Type of issue Recommendations
Management's Establish national policies, answer questions, and
Responsibilities resolve disputes in a timely manner.
Change the Central Office culture of "Information
is power so I won't share it."
Ensure that changed business practices and new
tools are adopted at the working level.
Prepare employees for the business practices and
system changes and emphasize the importance of
needed training.
Take a proactive position regarding data integrity
and commit the resources needed to get the job
done.
Resolve issues quickly, so employees do not
disengage completely from the issue, making it
even worse.
Instill in all employees a big picture perspective
to avoid the "That's not my job attitude."
Become familiar with the systems the employees
must use on a daily basis in order to appreciate
the effort needed to learn the system.
Address, buy-into, and communicate a policy on
data input by contractors.
Close the gaps that exist between the
organizational vision of upper management and the
changes that need to take place in business
processes and roles.
Decide and publish Central Office roles and
organizational structure on who ultimately decides
what. Establish accountability for business
decisions.
Managers must see the advantages of acting
proactively-this must become a core value in PBS.
Establish a management information steering
committee that will establish policy, fiscal
controls and resolve conflicts.
Start having regular meetings to go over project
data.
Accountability and Job descriptions should include data integrity.
Ownership Competency testing and certification should be
established for employees who need to understand
New Pricing, Occupancy Agreements, and STAR.
Performance ratings should address the issue of
employee proficiency in the use of tools and
databases required in their specific job
functions. Employees should be evaluated on the
integrity of their data and the promptness and
accuracy of their input.
Deal with the perception that STAR and Computer-
aided Design are wonderful systems that absolve us
of our individual responsibility to review, track,
and know our data.
If individual specialists are to be responsible
for their data, then any data manipulation that
takes place system-wide must be coordinated and
communicated.
Stop continually shifting employees from one area
to another.
Decide how much access to propriety data our
contractors can/should have.
Training and Tools Need to make sure that everyone who uses STAR gets
the right kind of training.
Cross training opportunities between business
lines should be encouraged.
More changed business practices training should be
scheduled.
Develop systems that can get information to the
people who need it.
Set up recurring training/certification programs.
Expand STAR training beyond realty specialists.
Conduct more billing training, emphasizing how and
when to do billing adjustments.
Plan just-in-time training and train internally
before taking it to the customers.
Regional business lines need to identify their
areas for immediate training.
Training for affected personnel must be mandatory
and enforced by directors, supervisors, and team
leaders.
Look at integrated training of multiple
tools-teach STAR, Occupancy Agreements, and the
client billing policy as an integrated training
session.
Regions need to take advantage of the available
training resources recently procured by the STAR
national training team.
Training and Tools Regional "Power Users" should be utilized in
(Cont.) training and instructing other regional users in
some type of regular scheduled sessions to keep
users up to date on changes and enhancements to
the databases and internal processes.
Identify any strong regional training programs as
best practices, and have the rest of PBS learn
Medium/long term from them.
actions
Some type of evaluation or testing of users
needs to be developed to ensure that they
understand the database(s) and process required
for them.
Customer training and A training infrastructure should be developed
marketing in each region that provides for regularly
scheduled sessions on all of the various databases
utilized by GSA. Training sessions should be
interactive.
Develop national, regional, and local
dialogs/workshops with our customers so they fully
understand new pricing, billing, and the changes
coming to existing assignments before they arrive
October 1, 1999.
Reports and Measures Develop meaningful performance measures for data
integrity. Any measure that is adopted should
identify the magnitude of the data problems and
encourage additional accountability and ownership.
Possible performance measures for data include the
following:
Number of exceptions/flags generated for each
region in STAR.
Number and dollar value of chargebacks.
Number of unopened work queues over some number
of days old in STAR.
Develop a workload measure that would help
management determine when to deploy additional
resources, if possible.
Number of billing adjustments.
Number of customer complaints with response
times for resolution.
Appendix III
Comments From the General Services Administration
Page 27GAO/GGD-00-12 STAR-Program for Tracking and
Managing Real Property
*** End of Document ***