Government Printing: Comparison of DOD and GPO Prices for Printing and
Duplicating Work (Letter Report, 02/17/95, GAO/NSIAD-95-65).
GAO examined the prices charged to customers for printing and
duplication work at the Government Printing Office (GPO) and the Defense
Printing Service (DPS). GAO found that GPO prices for printing work were
about 9 to 11 percent lower than those of DPS. For printing requisitions
of more than $500--about 78 percent of the printing dollars--GPO's
prices were about 21 percent lower than those of DPS. For requisitions
of $500 and less--22 percent of the total printing dollars--DPS' prices
were about 31 percent lower than GPO's. GAO estimates that overall, GPO
prices for DPS' fiscal year 1994 printing workload would have been about
$2.4 million lower than those of DPS. DPS prices for duplicating
requisitions were about four to five percent lower than GPO prices. For
those requisitions priced at more than $500, there was about 0.4-percent
difference favoring GPO, and for those of $500 or less, the difference
was about 57 percent in favor of DPS. GAO estimates that overall, DPS
prices for the DPS fiscal year 1994 duplicating workload would have been
about $9.3 million lower than the GPO prices.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: NSIAD-95-65
TITLE: Government Printing: Comparison of DOD and GPO Prices for
Printing and Duplicating Work
DATE: 02/17/95
SUBJECT: Printing costs
Government publications
Price adjustments
Cost analysis
Federal procurement
Contract costs
Information dissemination operations
Comparative analysis
Cost accounting
IDENTIFIER: Federal Printing Procurement Program
Defense Business Operations Fund
**************************************************************************
* This file contains an ASCII representation of the text of a GAO *
* report. Delineations within the text indicating chapter titles, *
* headings, and bullets are preserved. Major divisions and subdivisions *
* of the text, such as Chapters, Sections, and Appendixes, are *
* identified by double and single lines. The numbers on the right end *
* of these lines indicate the position of each of the subsections in the *
* document outline. These numbers do NOT correspond with the page *
* numbers of the printed product. *
* *
* No attempt has been made to display graphic images, although figure *
* captions are reproduced. Tables are included, but may not resemble *
* those in the printed version. *
* *
* A printed copy of this report may be obtained from the GAO Document *
* Distribution Facility by calling (202) 512-6000, by faxing your *
* request to (301) 258-4066, or by writing to P.O. Box 6015, *
* Gaithersburg, MD 20884-6015. We are unable to accept electronic orders *
* for printed documents at this time. *
**************************************************************************
Cover
================================================================ COVER
Report to Congressional Committees
February 1995
GOVERNMENT PRINTING - COMPARISON
OF DOD AND GPO PRICES FOR PRINTING
AND DUPLICATING WORK
GAO/NSIAD-95-65
Government Printing
Abbreviations
=============================================================== ABBREV
DCI - data collection instrument
DOD - Department of Defense
DPS - Defense Printing Service
GPO - Government Printing Office
NCA - National Capital Area
PRMIS - Printing Resources Management Information System
Letter
=============================================================== LETTER
B-259522
February 17, 1995
Congressional Committees
As set forth by the House report accompanying the fiscal year 1994
Department of Defense (DOD) Appropriations Act and as requested by
the former Chairmen of the Subcommittee on Defense and the
Subcommittee on Legislative, House Committee on Appropriations, we
reviewed issues related to the costs, prices, services, and
operations of the Government Printing Office (GPO) and the Defense
Printing Service (DPS). Our April 1994 report addressed issues
related to GPO and DPS costs, services, and operations.\1 In this
report, we address the prices charged to customers for acquiring
printing and duplicating work from those agencies. Our objective was
to determine whether there was a difference in the prices charged by
the two agencies. As stated in our April 1994 report, questions
about the adequacy of DPS' cost information precluded a cost
comparison of the agencies' operations.
--------------------
\1 Government Printing: Legal and Regulatory Framework Is Outdated
for New Technological Environment (GAO/NSIAD-94-157, Apr. 15, 1994).
BACKGROUND
------------------------------------------------------------ Letter :1
In our April 15, 1994, report, we reported that in recent years,
controversy has arisen over the printing operations of various
agencies. This was because some agencies wanted to publish their
work independent of GPO involvement. This controversy is largely the
result of significant advances in publishing technologies. In
presenting matters for congressional consideration, we noted that the
framework of laws and regulations used to manage the government's
publishing activities has been in place for many years and now seems
the appropriate time for a reassessment. We further stated that as
Congress continues to review the various legislative proposals, it
may wish to consider an alternative framework built on sound business
processes and changing publishing technologies. Appendix I is an
extract from our April 15, 1994, report.
By law, GPO, a legislative branch agency, provides printing-related
services to all branches of the federal government--either by
producing the work in-house or contracting with private vendors. For
the most part, GPO uses its in-house resources to produce
printing-related work for Congress, while it contracts with the
private sector to provide similar services for executive branch
agencies. In fiscal year 1994, GPO provided $724.4 million in
printing-related services for the government--$197.6 million through
in-house resources and $526.8 million through commercial procurement.
DPS is DOD's single manager for printing and duplicating operations.
It was established in April 1992 when the printing-related operations
of the military services and defense agencies were consolidated. At
its inception, DPS had an authorized staffing level of 3,694 persons
and 350 printing-related facilities. DPS has since reduced its
infrastructure as DOD continues to downsize. As of January 1995, DPS
reported that it had 2,343 persons on its staff and 256
printing-related facilities. Under current procedures, DOD customers
submit printing-related requirements to DPS, which satisfies most of
these needs either through its in-house resources or through GPO
contracts with the private sector. In fiscal year 1994, DPS produced
$220.4 million of work in-house and procured $177.4 million of
work--$167.8 million of the latter was procured through the GPO.\2
DPS sends the majority of its printing work to private vendors on
contract to the GPO and maintains most of its duplicating work
in-house. For the most part, in-house work comprises relatively low
dollar value work. For example, in fiscal year 1993, 75 percent of
the DPS' duplicating requisitions were priced under $103, and about
50 percent were under $28.
The prices DOD customers pay for their printing and duplicating work
vary according to the nature of the work, the provider of the work,
and the business arrangements in place to provide it. For work
performed in DPS facilities, DPS uses a uniform nationwide pricing
schedule that is based on the various production processes and costs
associated with producing a product. Customers are charged schedule
prices, which are adjusted annually, and the revenue is used to
offset costs associated with producing the work. Prices are adjusted
throughout the year to reflect changes in paper prices. For work
procured under GPO contracts, customers are usually charged a private
vendor's fee and administrative surcharges assessed by GPO and DPS.
Contractor prices are largely dictated by the economic forces of the
marketplace. In those cases in which GPO decides to produce the work
in-house rather than contract for it, GPO most often prices the work
at a level to cover the production costs. However, in some cases, it
offers discounts and may charge the prevailing commercial rate.
By comparing prices charged to the customer rather than costs of the
services provided, our analysis took the perspective of the DOD
customer who is interested in obtaining a particular service at the
lowest price. To determine the extent to which price differences
reflect differences in costs would require a detailed analysis of
GPO's and DPS' cost accounting systems. As discussed in our April
1994 report on government printing issues, we attempted but could not
make a meaningful comparison of GPO and DPS costs because (1) the two
agencies capture costs differently and (2) there are concerns
regarding the completeness, accuracy, and reliability of DPS' cost
accounting system.
--------------------
\2 The DPS in-house figure includes $29.6 million of cost-per-copy
contract work and self-service copying.
PRICE COMPARISON METHODOLOGY
---------------------------------------------------------- Letter :1.1
Whether DOD customers would be best served by having their printing
and duplicating work produced in DOD facilities or provided by GPO
has been the subject of debate for many years. Several price
comparison studies have been performed in recent years; however,
because of perceived or actual limitations in the conduct of these
studies, none has been widely accepted.
As we developed our price comparison methodology, we took into
account the criticisms of prior studies, the concerns of GPO and DPS,
and the comments of external printing consultants. Our methodology
was (1) based on a representative sample of DPS' $221 million fiscal
year 1993 nationwide in-house workload, (2) designed to capture
prices based on existing business conditions, and (3) reviewed by the
consultants. During the conduct of the study, we also implemented
various controls to minimize the possibility of agency bias on the
results.
Once we developed our sampling framework, our overall conceptual
approach for performing the price comparison was to have DPS send the
sample requisitions actually received from its customers during
fiscal year 1994 to GPO, who for the most part contracted to have the
work done. The contractors actually performed the work and billed
GPO for that work. We then took the price charged by GPO to DPS'
customers and compared it with the price DPS estimated it would have
charged to have performed the work. The results of this sample
comparison were then projected against DPS' fiscal year 1994
workload.
Our price comparison was based on a statistically representative
sample of 303 printing and 685 duplicating DPS customer-requested
requisitions produced between April and September 1994.
Collectively, these requisitions represented 327 printing and 1,193
duplicating products. We based our conclusions on a dollar-weighted
sample of printing and duplicating requisitions. The fiscal year
1994 customer-requested products submitted by DPS plant officials
were based upon criteria that we established from the products
described on a set of validated fiscal year 1993 requisitions.
The submissions were reviewed by an in-house panel of printing and
methodology specialists and the consultants. As DOD customers
routinely submitted requisitions, DPS officials chose those
requisitions that were similar to our criteria and, rather than
producing the work in DPS facilities, sent it to GPO, who largely had
the work produced by contractors. Plant officials sent us the
documentation associated with this work. Work passing our panel's
and consultants' review was used as the sample for this study. The
GPO bills for our sample were then compared with the prices DPS would
have charged if it had done the work in-house. Appendix II contains
a technical description of our methodology and lists the external
consultants we used.
RESULTS IN BRIEF
------------------------------------------------------------ Letter :2
A meaningful cost comparison of the services provided by GPO and DPS
cannot be made because these agencies capture costs differently and
there are longstanding questions regarding the reliability of DPS'
cost accounting system. Consequently, as agreed, we used price as a
comparative measure. We recognize that a price comparison does not
show which agency's services are more economical to the government.
Instead, it shows what customers are paying for services. Table 1
shows the DPS' 1993 workload and the results of our price comparison
projected to DPS' 1994 workload.\3
Table 1
Analysis of GPO and DPS Prices
(Dollars in millions)
Dollar value Number of By
of Dollar requisitio By dollar
requisitions s ns Favors percent s\a
------------ ------ ---------- ------ ---------- ------
Printing
------------------------------------------------------------
Over $500 $19.5 11,000 GPO 21.0 to $4.2
21.6
$500 or less 5.4 51,000 DPS 30.5 to 1.7
31.3
============================================================
Total $24.9 62,000 GPO 9.9 to $2.5
10.1
Duplicating
------------------------------------------------------------
Over $500 $178.0 25,700 GPO 0.6 to 0.8 $1.3
$500 or less 17.6 286,800 DPS 57.0 to 10.1
57.6
============================================================
Total $195.6 312,500 DPS 4.0 to 5.0 $8.7
------------------------------------------------------------
Note: The terms "printing" and "duplicating" represent our
categorization of DPS' workload. See appendix II.
\a Totals may not add due to rounding.
During our review, GPO officials pointed out that they believed if
more use were made of term contracts rather than one-time buys, there
would be a potential for lower contract prices through GPO. Term
contracts provide for the purchase of specific products or classes of
products from vendors during a specified period of time. Our
analysis included a post-stratification of the 1994 DPS sample
products to find out the extent to which DPS' use of term contracts
resulted in savings to DPS customers. We found that GPO contractor
prices were significantly--about 37 percent--lower than DPS printing
prices. However, we did not find significant differences between
GPO's duplicating term contracts and DPS' prices.
GPO provided price estimate information based on term contracts used
by DOD or other federal agencies. This information indicated that,
in most cases, these term contract prices would be lower than DPS'
prices. However, whether suitable term contracts could be
established to handle the DPS workload and satisfy specific customer
needs (e.g., quick turnaround) at the projected prices is uncertain.
Lastly, because our study was based on a fiscal year 1994 sample, it
represents the situation for that period. Relative price differences
will change as prices are adjusted by GPO, GPO contractors, or DPS.
The magnitude and direction of the changes are difficult to predict
because of uncertainty in the marketplace. Recent information shows
that GPO and DPS will both experience price increases during fiscal
year 1995.
--------------------
\3 For purposes of our analysis, we assumed the workload mix was the
same in 1994 as it was in 1993.
DPS AND GPO PRICES UNDER
EXISTING CONDITIONS
------------------------------------------------------------ Letter :3
Our analysis showed that, under conditions existing during our sample
period, GPO's prices for printing work were about 10 percent lower
than those of DPS'. For printing requisitions more than $500, GPO's
prices were about 21 percent lower than DPS' prices. For
requisitions $500 and less, DPS' prices were about 31 percent lower
than GPO's. We estimate that, in aggregate, GPO's prices for DPS'
fiscal year 1994 printing workload would have been about $2.5 million
lower than those of DPS'.
DPS' prices for duplicating requisitions were about 4 to 5 percent
lower than GPO's prices. For those requisitions priced over $500,
there was about 0.7 percent difference favoring GPO, and for those
$500 or less the difference was about 57 percent in favor of DPS. We
estimate that, in aggregate, DPS' prices for its fiscal year 1994
duplicating workload would have been about $8.7 million lower than
the GPO's prices.
PRINTING
---------------------------------------------------------- Letter :3.1
Our analysis of printing prices was based on a sample of 303
requisitions representing 327 products that we categorized as
printing. The sample comprises large (more than $500) and small
(equal to or less than $500) dollar requisitions. About 96 percent
of the work was done through private vendors on contract to GPO; the
remainder was produced in one or more of GPO's facilities. Although
we could not conclusively determine the major factors for the price
difference for printing, our results show that large dollar
requisitions were about 21 percent lower at GPO, representing about
78 percent of the printing dollars. However, the price difference
for small dollar requisitions showed that DPS' prices were about 31
percent lower. About 82 percent of DPS' fiscal year 1993 workload
for printing was for small dollar requisitions, but this category
contained about 22 percent of the total printing dollars.
DUPLICATING
---------------------------------------------------------- Letter :3.2
Our analysis of duplicating prices was based on a sample of 685
requisitions representing 1,193 products that we categorized as
duplicating. Like printing, the sample comprises large and small
dollar requisitions, many of which required quick turnaround times.
Our analysis of DPS' fiscal year 1993 workload for duplicating showed
that small dollar requisitions accounted for about 92 percent of the
requisitions, but 9 percent of the duplicating dollars. Large dollar
requisitions accounted for 8 percent of the requisitions, but 91
percent of the duplicating dollars.
For low dollar requisitions, DPS' prices were about 57 percent lower
than GPO's prices. For high dollar requisitions, GPO's prices were
about 0.7 percent lower than the DPS' prices.
GPO BELIEVES THAT USING TERM
CONTRACTS WOULD RESULT IN LOWER
GPO PRICES
------------------------------------------------------------ Letter :4
During the course of our review, GPO officials stated that they
believed that making maximum use of term contracts could provide DPS
with significantly lower prices for its printing and duplicating
work. GPO officials told us that about 75 percent of its work for
government agencies is performed under term contracts. According to
GPO officials, vendors may not have charged the favorable rates
available on term contracts for the one-time buys that were part of
our study. Although not part of our price comparison methodology,
GPO asked that we include in this report a discussion of how the use
of long-term contracts might affect contractors' prices.
During our test period, about 46 percent of the printing products and
41 percent of the duplicating products produced by GPO contractors
were completed under term contracts. The remaining products were
completed using one-time buys. Because GPO believes that term
contracts with its vendors result in substantial savings to the
government, we reanalyzed our data to try to detect these savings.
For printing, our data supported the conclusion that savings could
accrue. We found a significant difference-- about 37
percent--between GPO's contractor prices under printing term
contracts and DPS' printing prices. However, we did not find
significant differences when duplicating term contracts were used.
Under another approach, which is the most optimistic scenario, GPO
repriced our sample requisitions using prices from its term contracts
in existence across the country--even though they may have been for
agencies other than DOD. We examined GPO's repricing effort to
determine the relative level of GPO's prices versus those of DPS'.
We found that GPO's prices were lower than DPS' prices for 940 of
1,157 sample items that GPO repriced and were in our analysis.
Moreover, GPO's prices resulting from the repricing effort were often
lower than actual GPO's prices for sample items procured under GPO
term contracts during our study. This occurred in 495 of 630 cases
where a term contract was used.
We selectively verified orders repriced by GPO and found a number of
errors. Although these errors would not appear to materially alter
the supposition of lower prices, the errors do raise questions about
the relative magnitude of the prices. Some of the deficiencies we
identified included
minor errors in pricing various printing or duplicating processes,
omission of applicable GPO surcharges, and
misapplication of selected contracts for repricing estimates.
Lastly, whether suitable term contracts could be established to
handle DPS' workload and satisfy specific customer needs (e.g., quick
turnaround) at competitive prices is uncertain. GPO was unable to
reprice 48 of the sample requisitions because there were no existing
term contracts in effect that were appropriate for pricing purposes.
In previous price comparison studies and in DPS' comments on our
current analysis, the use of GPO's prices based on existing term
contracts with other agencies was questioned.
RECENT DPS AND GPO VENDOR PRICE
INCREASES
------------------------------------------------------------ Letter :5
Our analysis was based on DPS' and GPO's prices for work during the
latter half of fiscal year 1994 and, therefore, represents a
point-in-time comparison. Recent price increases taking effect
during fiscal year 1995 could change the relative price difference
between GPO- and DPS-provided work. The magnitude and direction of
the difference is difficult to predict, however, because of
uncertainty in the marketplace.
In October 1994, DPS increased its overall prices by 18.5 percent and
its surcharge for processing work to GPO from 1.83 percent to 5.5
percent. According to DPS officials, the increases were intended, in
large part, to offset prior years' printing-related operation losses.
DPS is part of the Defense Business Operations Fund--a revolving fund
that is used to fund operations for DPS and many other DOD
activities.
In an August 1994 correspondence to the Joint Committee on Printing,
GPO forecasted that GPO prices obtainable through private contractors
would most likely increase by about 3.8 percent during fiscal year
1995. However, in October 1994, GPO officials advised agencies that
private sector price increases may be greater. This was because of
substantial paper cost increases. GPO officials stated that prices
in new term contracts involving significant amounts of paper could
experience 20 to 40 percent price increases. Paper cost increases
may also affect DPS' prices, because DPS can adjust the price it
charges its customers during the fiscal year to reflect paper price
increases.
AGENCY COMMENTS AND OUR
EVALUATION
------------------------------------------------------------ Letter :6
In its official comments on a draft of this report, DOD agreed with
our audit methodology, execution, and conclusions. The comments are
included as appendix III.
GPO, in its official comments, did not agree with many aspects of our
report and the methodology we used to perform our work. The comments
are reprinted in their entirety as appendix IV. GPO commented that
our draft report contained material deficiencies that provided an
undue bias in favor of DPS and that understated the
cost-effectiveness of the GPO Printing Procurement Program and its
private sector printers. GPO cited a number of methodological
concerns (such as our reliance on what GPO characterizes as an
incomplete and questionable DPS database) that it believed raise
questions about the reliability of our work. Further, GPO stated
that we did not always exercise due professional care during the
planning and performance of the study. To illustrate this point, GPO
stated that we (1) allowed DPS to select the sample for the study,
(2) accepted sample jobs outside the criteria established for
selecting the sample, (3) allowed DPS to control the timing and means
for delivering sample jobs to GPO, and (4) allowed DPS to provide the
source documentation for the comparative price analysis.
We disagree with GPO's primary concerns. We believe that our draft
report does not contain material deficiencies and that we exercised
all due care in the conduct of our work. Because of the sensitivity
and continuing controversy surrounding various printing and
duplicating management issues, we exercised extreme care in the
planning and conduct of our work in order to provide results that are
indicative of real world conditions. With regard to GPO's specific
concerns, we disagree for the reasons as detailed in the succeeding
paragraphs.
We did not, as GPO stated, use data from a questionable database
without first testing the reliability of the data we needed for our
study. For example, in using the fiscal year 1993 DPS management
information database as a baseline for our work, we selected 440
printing and 300 duplicating automated requisition entries and
subsequently compared the associated automated data with data found
on the corresponding hard-copy requisitions. In over 90 percent of
the cases, the data were appropriate and we judged them to be
reliable for the purposes of our study.
We disagree that DPS selected the sample for our study and that we
accepted many jobs that, GPO states, were not "twins" based on our
job specification criteria. If DPS were permitted to select jobs
without proper controls, it would have created an opportunity for
unwanted bias. To avoid this, we had DPS plant officials nominate
candidate jobs for our sample, using preestablished criteria that we
provided to them. The criteria provided to DPS were never intended
to restrict DPS to obtaining exact matches, or twins as GPO states.
Certain specifications (e.g., major production processes used,
estimated price, classification level, and time available to produce
the job) were key to DPS' sample nomination process. Other criteria
(e.g., technical job characteristics such as stitching) were provided
as guidelines for DPS to use in attempting to nominate products that
were similar, but not necessarily a direct match. We subsequently
had DPS' nominations reviewed by two panels to ensure that the
nominations (1) fit within the statistical model for our work and (2)
did not have any technical printing requirements or other
considerations that would unduly bias either agency. One panel
consisted of internal printing and methodology specialists, while the
other comprised external printing consultants. As a result of these
reviews, we rejected about 9 percent of DPS' sample candidates. For
rejected cases, we asked DPS to submit additional candidates, which
were subjected to the same review process.
With regard to GPO's statement that DPS controlled the timing and
means of delivering sample jobs to GPO, we believe we exercised
proper oversight of the process. We instructed DPS to use its normal
business procedures to obtain its work through GPO so that our
comparison was realistic. To ensure that DPS did not unduly delay
the delivery of jobs to GPO, we examined, through the panel process,
the timing (DPS receipt of a customer request and subsequent
submission to GPO) for each job in our sample. We also conducted a
separate subsample analysis to further examine the issue. Our
analysis showed no systemic bias in the process. As to the means of
delivering jobs to GPO, we have no indication that DPS intentionally
biased the process, given the requirement to meet the customers'
request dates.
GPO stated that DPS provided the source documentation to us for the
comparative analysis. However, while DPS did provide much of the
data related to the customer requisitions, DPS price estimates, and
GPO billings, we also obtained source documentation from GPO. We
examined and verified the source documentation where reasonable and
where we determined the risk of bias was high. For example, we had
external printing consultants review DPS' price estimates. To verify
the GPO invoice prices provided by DPS, we extracted GPO bills from
database files provided by GPO. We also performed a quality
assurance analysis to examine cases where it appeared that
discrepancies in our data may exist. Where errors were detected, we
made the appropriate corrections in our database.
GPO also questioned our presentation of information in the report.
For example, GPO believes that our interchangeable use of such terms
as "requisitions," "jobs," and "orders" in our draft report created
confusion. Further, GPO stated that our draft methodology did not
sufficiently disclose what we actually did during the review.
Because of GPO's apparent misunderstanding of our methodology, we
have included a more technical description of our methodology and
clarified our report presentation.
GPO also questioned whether it was appropriate for us to have GPO
comment on our report prior to the completion of all audit work.
While we agree that this situation was not ideal, we did not release
the draft report for comment until we had sufficient data to project
the results of our work. The small amount of additional data
collected after we released our draft report for comment did not
materially affect the results of our study.
As to specific concerns noted in GPO's official comments, we have
annotated the comments and have provided our views following the
reprint of GPO's comments in appendix IV.
---------------------------------------------------------- Letter :6.1
We are sending copies of this report to the Secretary of Defense, the
Public Printer of the United States, the Director of DPS, the
Director of the Office of Management and Budget, and other
appropriate congressional committees. We will also make copies
available to others upon request.
The report was prepared under the direction of Donna M. Heivilin,
Director, Defense Management and NASA Issues, who can be reached on
(202) 512-8412 if you or your staff have any questions. Other major
contributors are listed in appendix V.
Henry L. Hinton, Jr.
List of Congressional Committees
The Honorable Strom Thurmond
Chairman
The Honorable Sam Nunn
Ranking Minority Member
Committee on Armed Services
United States Senate
The Honorable Ted Stevens
Chairman
The Honorable Daniel K. Inouye
Ranking Minority Member
Subcommittee on Defense
Committee on Appropriations
United States Senate
The Honorable Floyd Spence
Chairman
The Honorable Ronald V. Dellums
Ranking Minority Member
Committee on National Security
House of Representatives
The Honorable C. W. Bill Young
Chairman
The Honorable John P. Murtha
Ranking Minority Member
Subcommittee on National Security
Committee on Appropriations
House of Representatives
The Honorable Ron Packard
Chairman
The Honorable Vic Fazio
Ranking Minority Member
Subcommittee on Legislative
Committee on Appropriations
House of Representatives
(See figure in printed edition.)Appendix I
EXTRACT FROM OUR APRIL 15, 1994,
REPORT ON GOVERNMENT PRINTING
============================================================== Letter
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
SCOPE AND METHODOLOGY
========================================================== Appendix II
Debate continues as to whether DOD can save money by contracting its
printing and duplicating work through GPO rather than producing it
in-house. Several studies have been conducted on this topic, but
they do not support generalizable findings. Accordingly, we set out
to determine whether DOD customers would pay more or less to obtain
printing and duplicating services through GPO and its private sector
contractors rather than through DPS.
STUDY APPROACH
-------------------------------------------------------- Appendix II:1
Our study is based on the printing and duplicating workload
undertaken by DPS in response to customer requisitions received
during fiscal year 1993. Because it is requisition based, the study
excludes consideration of that portion of DPS' workload associated
with self-service or copy center operations. Work performed in
overseas plants is also excluded because GPO does not typically
service those locations. DPS' fiscal year 1993 workload is used to
specify a statistically matched set of fiscal year 1994 requisitions.
The difference between DPS' and GPO's fiscal year 1994 prices for
this matched set serves as the basis for the study's price
comparisons.
Our work was conducted between September 1993 and January 1995 in
accordance with generally accepted auditing standards.
TERMINOLOGY DIFFERENCES
------------------------------------------------------ Appendix II:1.1
In recent years, the proliferation of computers, laser printers, and
other digital technologies has blurred the distinctions between
printing and duplicating. Today the terms are often used
synonymously, but their precise definition has been the subject of
disagreement within the printing community. DPS, for example, may
classify a piece of work as duplicating, while GPO might classify the
identical product as printing. Similarly, DPS and GPO frequently
disagree about the production processes required to produce products
at different quality levels.
Our study is based on DPS' workload, and we used DPS' production
classifications. Accordingly, the projected price differences for
printing or duplicating are expressed in terms of DPS'
classifications. Our price comparisons, however, are product-based.
Thus, while DPS and GPO may disagree as to whether a particular work
process should be classified as printing or duplicating, this study
reports solely the price differences charged for products that the
customer would perceive as identical.
GEOGRAPHIC CONSIDERATIONS
------------------------------------------------------ Appendix II:1.2
During our sample period, DPS comprised a headquarters and field
activities located within eight areas. DPS prices in-house work at
the time a requisition for printing or duplicating is accepted using
a standard rate schedule. DPS uses this schedule to price each of
the processes used to produce a job. With the exception of paper
prices, the rate schedule is identical for all areas and is updated
on an annual basis. Paper prices are set at the plant level and may
vary throughout the year.
GPO also has a headquarters and regional structure. For executive
agency printing and duplicating work, GPO usually contracts the work
with local or regional commercial vendors. Consequently, GPO's
prices are affected by market conditions. The price for identical
work may vary between regions and, seasonally, within the same
region. The final invoice (including GPO's contract administration
surcharge) is provided after completion of the work and receipt of
the contractor's billing.
GPO's regional boundaries do not coincide with the geographic
boundaries of the DPS areas. Thus, there is not a one-to-one
relationship between a DPS area office and a GPO regional office. A
DPS area office may contract support from multiple GPO regions, and a
single GPO region may support multiple DPS area offices.
DEFINING DPS' FISCAL YEAR
1993 WORKLOAD
------------------------------------------------------ Appendix II:1.3
Members of the printing community often assert that every printing
and duplicating product is unique. Although the products may be
unique, it is clear that there is a finite, countable sequence of
production steps involved in their manufacture. The identification
of these sequences is the core of our statistical design.
DPS' Printing Resources Management Information System (PRMIS) tracks
requisitions and prices for 184 separate work processes that may be
involved in providing customers with requested products. We
realigned DPS' fiscal year 1993 PRMIS data to reflect DPS' job
pricing system--one that is designed to capture the prices charged
for the various manufacturing processes used to produce a product.
By tracking requisitions through the various processes, we were able
to categorize DPS' overall fiscal year 1993 in-house workload into
nine major production processes. Table II.1 lists these processes
and the revenue associated with them in fiscal year 1993. Each of
these major processes was further stratified to reflect additional
production processes (e.g., stitching) and price strata. In total,
our study comprised 37 printing and 32 duplicating strata.
Table II.1
DPS' Major Production Processes (Fiscal
Year 1993)
(Dollars in millions)
Requisitio
Major production process ns Price
---------- ------------------------ ---------- ----------
Printing
------------------------------------------------------------
1 10x15/11x17 press 38,503 $5.0
2 Offset press 14x19 width 17,095 9.6
3 Offset press 22x29/ 3,675 5.8
22x34
4 Offset press 35x45 896 2.0
5 Two-color press 35x45 911 1.8
6 Roll-fed 22x34 116 0.5
7 Thermo/envelope/ 999 0.2
letterpress
Duplicating
------------------------------------------------------------
8 Electrostatic 276,812 188.0
duplicating
9 Offset duplicating 35,724 7.6
10x15/11x17
============================================================
Total 374,731 $220.5
------------------------------------------------------------
Note: Overseas plant figures are excluded.
VALIDATION OF DPS' PRMIS
DATABASE
------------------------------------------------------ Appendix II:1.4
We relied extensively on computer-processed data contained in the
DPS' PRMIS database. We assessed the reliability of the data by
comparing them with data from fiscal year 1993 hard-copy
requisitions. This comparison was conducted using a statistically
valid sample. Table II.2 lists the sample sizes by major production
process.
Table II.2
Sample Used to Validate PRMIS
(Dollars in millions)
Requisitio Sample
Major production process ns size
---------- ------------------------ ---------- ----------
Printing
------------------------------------------------------------
1 10x15/11x17 press 38,503 123
2 Offset press 14x19 width 17,095 93
3 Offset press 22x29/ 3,675 69
22x34
4 Offset press 35x45 896 45
5 Two-color press 35x45 911 35
6 Roll-fed 22x34 116 25
7 Thermo/envelope/ 999 25
letterpress
============================================================
Subtotal 415
Duplicating (National Capital and Western areas)
------------------------------------------------------------
8 Electrostatic
duplicating
National Capital Area 13,679 38
Western Area 94,056 112
9 Offset duplicating
10x15/11x17
National Capital Area 536 38
Western Area 7,426 112
============================================================
Subtotal 300
============================================================
Total 177,892 715
------------------------------------------------------------
Note: Overseas plant work is excluded.
With regard to printing, all fiscal year 1993 requisitions were
grouped according to DPS' seven major printing production processes
and stratified into 37 cells to reflect the influence of production
line characteristics (major combinations of press requirements,
offset plates, etc.) and requisition price (greater than $500 or $500
and less). Statistics for each cell were derived, and a hard copy of
the original customer request was obtained for a dollar-weighted
sample of requisitions.
With regard to duplicating, PRMIS provided a count of requisitions
and prices for electrostatic and offset duplicating work in all DPS
areas. However, only the National Capital and Western areas retained
the individual hard-copy requisitions needed to support our
validation. Thus, our validation of the PRMIS' duplicating workload
focused on DPS' National Capital and Western areas. Within these
areas, duplicating requisitions were grouped according to DPS' two
major production processes and stratified by requisition price
(greater than $500 or $500 and less). Statistics for each cell were
derived, and a hard copy of the original customer request was
obtained for a dollar-weighted sample of requisitions.
We concluded that the data were sufficiently complete and accurate
and thus could serve as a reliable description of DPS' fiscal year
1993 workload. Hard-copy requisitions for 385 of the 415 sampled
PRMIS printing records were obtained and found to validate the PRMIS'
data on price, production sequence, and requisition number. We
obtained hard-copy requisitions for 273 of the 300 sampled
duplicating records and validated the PRMIS' data on price,
production sequence, and requisition number.
STRUCTURING THE FISCAL YEAR
1994 PRICE COMPARISONS
-------------------------------------------------------- Appendix II:2
Our analysis assumes that DPS' fiscal year 1993 workload is
characteristic of DOD's annual demand for printing and duplicating
services. This is not to say that DPS will complete the same volume
of work each year, but rather that its mix of work will remain
relatively constant.
We took a statistically representative sample of DPS' mix of work
from the fiscal year 1994 incoming work requests. DPS priced the
requisitions according to its rate schedule and then sent them to GPO
for production. GPO provided an initial price estimate upon
acceptance of the work and a final invoice price (the contractor's
price plus a surcharge for GPO's administration of the contract)
following completion of the work.
PROTECTION AGAINST SELECTION
BIAS
------------------------------------------------------ Appendix II:2.1
To match fiscal year 1994 jobs with the fiscal year 1993 work mix, we
sought to ensure requisite statistical conditions (necessary for the
preservation of fiscal year 1993 strata properties) and reduce or
eliminate any perception of selection bias on the part of DPS or GPO.
The criteria for a matched requisition were specified by the
statistical cell from which a fiscal year 1993 requisition was drawn.
Attributes obtained from the fiscal year 1993 requisition (e.g.,
number of copies, original impressions, available workdays to perform
the work, paper weight and finished size, type of binding, and
distribution) served as subordinate guidelines to protect against
selection bias. We listed these attributes on the data collection
instrument (DCI) that DPS plant officials used to nominate a
requisition for inclusion in our study. An example of the DCI used
for a printing requisition is shown in figure II.1.
Figure II.1: Sample Form
Listing Attributes of a
Printing Requisition
(See figure in printed
edition.)
After DPS plant officials identified a requisition that met the
parameters of our DCI, they submitted the requisition to GPO for
production and then provided us with documentation to support their
nomination (a completed copy of the DCI, the customer's requisition,
and DPS' schedules used for pricing the product). Before including
the nominations in our sample, we tasked a panel comprising our
personnel with printing, evaluation, and methodological expertise to
examine them. The panel ensured that the nominations fit the
necessary statistical parameters and that, from the viewpoint of the
printing community, the selection would not put GPO at a
disadvantage. All acceptable matches were subsequently reviewed by
external consultants with printing expertise. Specifically, we used
two private sector consultants--Willard Brown, Printing Consultant,
and Herbert Langford, Langford and Associates. They validated the
panel's decisions and reviewed DPS' price estimates, thereby
providing an additional safeguard against systemic bias on the part
of DPS areas or plants.
About 9 percent of DPS-proposed matches failed the review process.
In these cases, the work was not included in our sample and DPS plant
officials were requested to nominate replacements that more
accurately reflected the statistical criteria and attributes
specified within our DCI.
THE FISCAL YEAR 1994
SAMPLING FRAME
------------------------------------------------------ Appendix II:2.2
Table II.3 presents an overview of the study's sampling frame. The
table is organized by major production process and lists the sample
size, responses, and the number of individual products produced using
each process.
Table II.3
Sample Overview (Fiscal Year 1994)
Number
Major production Response of
process Sample s products
-------- -------------------- -------- -------- --------
Printing
------------------------------------------------------------
1 10x15/11x17 press 115 96 98
2 Offset press 14x19 90 74 77
width
3 Offset press 22x29/ 61 44 53
22x34
4 Offset press 35x45 39 30 30
5 Two-color press 33 25 35
35x45
6 Roll-fed 22x34 24 17 17
7 Thermo/envelope/ 23 17 17
letterpress
Duplicating
------------------------------------------------------------
8 Electrostatic 661 571 946
duplicating
9 Offset duplicating 147 114 247
10x15/11x17
============================================================
Total 1,193 988 1,520
------------------------------------------------------------
PRINTING SAMPLE
---------------------------------------------------- Appendix II:2.2.1
We asked DPS to match 385 of our validated fiscal year 1993 printing
requisitions (stratified by major product line, subproduct, and
requisition price) with incoming fiscal year 1994 requisitions. DPS
successfully matched 316 of the 385 requisitions, but at the
conclusion of our study, GPO's contractors had provided final
invoices for only 303 requisitions. Thus, we used a 79-percent
response rate in estimating differences between the DPS' printing
price and GPO's final invoice.
DUPLICATING SAMPLE
---------------------------------------------------- Appendix II:2.2.2
An overview of our sampling scheme for duplicating requisitions is
shown in table II.4.
Table II.4
Sampled Duplicating Requisitions by DPS
Area
Sample Sample
DPS area size Responses size Responses
---------------- --------- --------- --------- ---------
Central Area 91 67 16 8
National Capital 31 25 38 35
Area
Northeast Area 91 68 16 12
Northwest Area 91 86 16 10
Southeast Area 80 78 14 11
Southern Area 91 83 16 11
Southwest Area 91 78 16 13
Western Area 95 86 15 14
============================================================
Total 661 571 147 114
------------------------------------------------------------
Because the National Capital Area (NCA) was unique in its
distribution of duplicating work, we treated it separately from the
other DPS areas--both in our validation of the PRMIS' workload and in
the selection of requisitions for our fiscal year 1994 sample. We
asked DPS to match 69 of our validated fiscal year 1993 requisitions
from NCA (31 electrostatic and 38 offset duplicating, as stratified
by major production process and requisition price) with an incoming
fiscal year 1994 duplicating requisition from the same plant.
To protect against selection bias in the remaining DPS areas, we used
the properties of the Western Area's validated requisitions to create
templates for our DCIs. This allowed us to control DPS' nominations.
We sent a combined total of 739 requisitions to the seven areas
(excluding NCA). From our prior work with PRMIS, we knew the
statistical properties of the mix of work for each of these areas.
Using the statistical properties of the mix of work from each of the
areas, we calculated our estimates.
COMBINED BILLINGS AND
OPEN REQUISITIONS
---------------------------------------------------- Appendix II:2.2.3
In the process of validating PRMIS, we identified two classes of
requisitions that deserved special treatment. The first class,
combined billings, was made up of a collection of individual products
that were batched for production on a single requisition. The second
class, open requisitions, usually comprised multiple products or
recurring tasks for the same customer that were billed to the same
account. Open requisitions were, in effect, term contracts to
provide continuing support to one customer over a period of time.
In our validation of PRMIS, we found 13 combined billings (2 printing
and 11 duplicating requisitions). These requisitions consisted of 59
individual products, each with its own attributes. For data
collection purposes, we separated each combined billing into an
individual DCI for each product. The responses to these DCIs were
ultimately combined to reflect the original 13 combined billing
requisitions.
In our validation of PRMIS, we found 26 open requisitions (2
printing, and 24 duplicating requisitions). These requisitions were
different in that, as blanket purchase agreements, their attributes
were more those of a contract than an individual product. For
sampling purposes, we asked DPS to match the original fiscal year
1993 requisition with a fiscal year 1994 open requisition having
approximately the same dollar value and requiring similar production
processes and delivery schedules--preferably a fiscal year 1994
contract supporting the same customer. As with individual
requisitions, each of DPS' open requisition nominations were subject
to the review and approval of our panel. In our study, DPS
successfully matched 23 of the 26 open requisitions (1 printing and
22 duplicating requisitions, collectively representing a total of 310
products).
For each requisition that was approved by the panel, we identified a
start date and tasked DPS to provide for our sample a prescribed
sequence of products submitted against that contract. As products
were submitted against these requisitions, DPS forwarded the work to
GPO for production and provided us with the customer's requisition
and DPS' schedules used for pricing the product.
Though open requisitions are analogous to GPO's direct-deal term
contracts, DPS does not provide a price reduction on these
requisitions. GPO states that if the work were done by GPO, however,
the DOD customer would receive a price discount. For this reason, we
asked two agencies to price all of the open requisition products as
though the work were performed under their existing GPO direct-deal
term contracts. Thus, we obtained three sets of prices for each open
requisition: DPS' price, GPO's final invoice price, and the
agencies' direct-deal contract price. The direct-deal price was used
as a surrogate for GPO's price in our basic price comparisons. GPO's
final invoice was used to support a separate analysis of term
contract prices. Because open requisitions are billed monthly, we
integrated the prices obtained for each sample in order to reflect
the monthly charge for each requisition.
SAMPLE NONRESPONSE
---------------------------------------------------- Appendix II:2.2.4
To maximize response rates, we visited 47 of the 99 DPS plants
involved in our study to review the progress DPS was making with its
nominations. On a continuing basis, we made phone calls to encourage
high response rates. Weekly, we informed DPS area officials of the
number of successful nominations made by their subordinate plants.
We investigated apparent difficulties or inadequacies in achieving
our response goals. Through these efforts, we achieved an 83-percent
response rate for the combined count of printing and duplicating
requisitions. To ensure that the missing requisitions did not change
the representativeness of our sample, we examined the potential bias
that might be introduced by nonresponse. Categories of nonresponse
are shown in table II.5.
Table II.5
Categories of Nonresponse
Number of
requisitio
Nonresponse reason ns Direction of bias
----------------------- ---------- -----------------------
Sensitive or classified 43 Favors GPO: Production
material of classified or
sensitive material
would likely increase
the contractor price.
Downsizing 24 No apparent bias.
GPO rejected production 32 Favors GPO: Most cases
were the result of our
demand for a
requisition with a
rapid turnaround time.
Miscellaneous 76 No apparent bias:
Nonresponse was
related to sample size
in each area.
Missing GPO final 30 No apparent bias: DPS'
invoice last day for data
collection was
September 30, 1994.
The final invoices
were not received by
January 15, 1995.
------------------------------------------------------------
As may be seen, except for the special provisions needed to safeguard
national security and other sensitive materials, our research did not
indicate the presence of systemic bias against GPO in our sample.
Thus, the study's final estimates were calculated by reweighting the
sample to ignore nonresponse.
THE ANALYSIS
-------------------------------------------------------- Appendix II:3
DPS' fiscal year 1993 mix of work was used to define a statistically
matched set of fiscal year 1994 requisitions. The difference between
DPS' prices and GPO's invoice prices for this set served as the basis
for the study's price comparisons. Based on dollar-weighted sampling
strata, requisition price differences were used to estimate the
fiscal year 1994 population values.
POST-STRATIFICATION AND THE
ANALYSIS OF TERM CONTRACTS
------------------------------------------------------ Appendix II:3.1
GPO officials stated that about 75 percent of GPO contract work for
government agencies is done through term contracts rather than
through one-time purchases. These officials pointed out that these
contracts usually result in considerably lower prices for the
agencies. Our analyses included a post-stratification of the 1994
DPS products, as distinct from requisitions, to determine the extent
to which DPS' use of term contracts resulted in savings to the DPS
customer. Table II.6 shows the results of these analyses.
Table II.6
Effect of Term Contract Use
GPO price index
based on DPS price
Term contract type = 100 Sampling error
-------------------- ------------------ ------------------
Printing 63 31
Duplicating 112 35
------------------------------------------------------------
GPO'S REPRICING EFFORT
------------------------------------------------------ Appendix II:3.2
At the request of GPO officials, we allowed GPO to use any of its
term contract information to reprice the DPS work--after GPO had
completed and billed DPS for our test cases. GPO officials believed
that, in most cases, they could have provided DPS with more favorable
prices if term contracts had been in place and used to price the
work. According to GPO, about 75 percent of its commercial work is
done through term contracts--at prices considerably lower than the
prices charged for one-time purchases.
We selectively verified the pricing of a sample of the jobs that were
repriced by GPO, and we compared those findings with the results of
our own independent analysis of term contract prices that was
described earlier.
QUALITY ASSURANCE EFFORTS
-------------------------------------------------------- Appendix II:4
The structured DCIs, our internal review panel, the printing
consultants, and the site visits of our staff played an important
role in ensuring the credibility of the data that were finally used
in our analyses. These efforts were supported by additional quality
assurance efforts. Specifically, we took two samples. One was a
statistical subsample of our fiscal year 1994 responses. The other
was a sample of instances in which we found extreme percentage
differences between the DPS and GPO duplicating prices. Our goal was
to examine product quality, timing, and pricing issues to detect
errors and any indication of bias.
STATISTICAL SUBSAMPLE
ANALYSIS
------------------------------------------------------ Appendix II:4.1
We took a subsample of 60 of our fiscal year 1994 responses. We
compared requisition information from DPS with GPO's product
information database to determine whether (1) GPO was given the
DCI-specified time to perform the work and (2) the number of GPO's
and DPS' product copies were the same. In all but one case, GPO's
data agreed with the range specified on our DCI.
We also used the GPO product information database to determine the
extent to which DPS requested quality level III work. (GPO's
technical specifications range from quality level I for highest
quality to quality level V for lowest.) GPO told us that by
specifying quality level III work, DPS forced GPO commercial vendors
to print the work rather than duplicate it. GPO states that this
would put them at a disadvantage in some cases. We found four jobs
in our subsample where DPS requested quality level III work--about 7
percent of our sample. The majority of the work requested by DPS in
our subsample consisted of quality levels IV and V.
EXTREME PERCENTAGE
DIFFERENCES BETWEEN DPS' AND
GPO'S DUPLICATING PRICES
------------------------------------------------------ Appendix II:4.2
We looked at 104 requisitions with extreme price differences (in
percentages). We used GPO's product information database to verify
requisition information and obtain both quality levels and pertinent
dates to determine whether there was any bias against GPO with
respect to quality levels of work requested and the amount of time
given GPO to produce the work. We located 96 of the requisitions in
the database and investigated each of these cases with telephone
calls to the DPS plants from which they originated. Where we
detected errors, we made appropriate changes.
(See figure in printed edition.)Appendix III
COMMENTS FROM THE DEPARTMENT OF
DEFENSE
========================================================== Appendix II
(See figure in printed edition.)Appendix IV
COMMENTS FROM THE GOVERNMENT
PRINTING OFFICE
========================================================== Appendix II
See comment 1.
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
See comment 2.
(See figure in printed edition.)
See comment 3.
(See figure in printed edition.)
(See figure in printed edition.)
See comment 4.
(See figure in printed edition.)
(See figure in printed edition.)
See comment 5.
(See figure in printed edition.)
See comment 6.
(See figure in printed edition.)
See comment 7.
(See figure in printed edition.)
(See figure in printed edition.)
See comment 8.
(See figure in printed edition.)
See comment 9.
(See figure in printed edition.)
(See figure in printed edition.)
See comment 10.
(See figure in printed edition.)
(See figure in printed edition.)
See comment 11.
(See figure in printed edition.)
See comment 12.
(See figure in printed edition.)
See comment 13.
(See figure in printed edition.)
(See figure in printed edition.)
See comment 14.
(See figure in printed edition.)
See comment 15.
(See figure in printed edition.)
See comment 16.
(See figure in printed edition.)
(See figure in printed edition.)
See comment 17.
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
See comment 18.
(See figure in printed edition.)
See comment 19.
(See figure in printed edition.)
See comment 20.
(See figure in printed edition.)
(See figure in printed edition.)
See comment 21.
(See figure in printed edition.)
See comment 22.
(See figure in printed edition.)
(See figure in printed edition.)
See comment 23.
(See figure in printed edition.)
(See figure in printed edition.)
See comment 24.
See comment 25.
(See figure in printed edition.)
See comment 26.
(See figure in printed edition.)
(See figure in printed edition.)
See comment 27.
See comment 28.
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
See comment 29.
See comment 30.
(See figure in printed edition.)
The following are GAO's comments on the Government Printing Office's
(GPO) letter dated February 8, 1995.
GAO COMMENTS
1. The points raised in GPO's transmittal letter are addressed in the
Agency Comments and Our Evaluation section of this report.
2. In our April 1994 report, we noted that the emergence of various
electronic technologies has blurred the distinction between printing
and duplicating. We also stated that the legal and regulatory
framework used to manage many aspects of government publishing has
become outdated. For the purposes of our study, we chose to
replicate DPS' fiscal year 1993 mix of work. Therefore, we selected
our sample based on DPS' printing and duplicating processes, and have
annotated table 1 to reflect this. The definitions for these
services may differ from those established by the Joint Committee on
Printing. See appendix II for additional details on our methodology.
3. As stated in our April report, we are aware of the limitations in
DPS' accounting system. It is for this reason that we were unable to
perform a meaningful cost comparison. We chose, with full agreement
from our congressional requesters, to perform a strictly controlled
pricing study. The requesters' staffs were briefed on the reasons
for and the limitations of this pricing study.
4. GPO's statement that we did not assess the reliability of DOD
workload data is not correct. When we initially received the
database from DPS, we reorganized it so that we could select our
fiscal year 1993 sample. We selected our sample and then asked DPS
to find the actual fiscal year 1993 requisition that matched our
selection. DPS was able to find the identical requisition from its
printing facilities that matched our request in over 90 percent of
the cases. Likewise, the dollar amounts of these requisitions
substantially agreed with calculations we initially developed. The
fact that we found requisition and dollar amounts that were accurate
assured us that DPS' fiscal year 1993 database was reliable and
accurate enough that we could base our work on it.
5. We agree with GPO's comments in this section and have made
appropriate clarifications in the report.
6. This concern is addressed in the body of the report.
7. The data we requested from DPS did not include self-service
duplicating. The only work included in our sample was work sent to
DPS printing facilities on a requisition. We disagree with GPO that
we should exclude all DPS requisitions with prices under $25 because
we believe this would not reflect DPS' actual work mix.
8. Appropriate changes have been made in the report text.
9. GPO's statement that our classification of DPS' workload was
simplistic is incorrect. We chose to report our results in only four
categories out of concern for the clarity of the report. Our
analysis of the DPS database, however, entailed the examination of 37
printing and 32 duplicating strata, as described in our methodology.
10. As reported in our April 1994 report, the distinction between
printing and duplicating has become blurred. We discussed quality
level issues with our in-house printing staff, printing consultants,
and with DPS and GPO officials. They told us that the differences
between quality levels were not always readily distinguishable
without the benefit of specialized inspection. Newer technology
enables non-printing processes to produce print-quality work. Our
post-analysis review showed about 7 percent of DPS requisitions
listed as duplicating, which GPO believed required contractors to
produce at quality level III ("good" quality as defined by GPO's
Quality Assurance Through Attributes Program). We reviewed instances
where GPO was concerned with quality level III selections and found
no evidence of systemic bias attempts by DPS. Furthermore, in
several instances, DPS plant personnel told us that their equipment
could produce the quality necessary to satisfy the customer.
11. We disagree with GPO's statement that we allowed DPS to choose
our sample, and we discuss this in the Agency Comments and Our
Evaluation section of this report.
12. We disagree with GPO's statement that we allowed DPS to control
the timing and means of delivering sample jobs to GPO. Again, we
discuss this in the Agency Comments and Our Evaluation section of
this report. We partially agree with GPO that DPS provided the
source documentation for our analysis. GPO was also the source of
some documentation. Where we thought there was a risk for bias, we
attempted to verify both DPS and GPO source documentation.
13. We understand GPO's concerns about paper price increases;
however, this is a situation that affected both GPO contractors and
DPS during the course of our study. We have no evidence to either
disagree or agree with GPO's concern that commercial vendor prices
are more sensitive to paper price changes than are DPS' prices. Much
like GPO contractors, DPS paper prices are not set on a nationwide
basis, but are established individually by each DPS printing
facility. DPS printing facilities charge according to a rolling
average price, which is maintained for each type of paper in their
inventory. The price level changes as paper purchases are added and
as paper inventories are used.
14. We did not test our methodology prior to implementing the survey.
We did take several steps, however, to assure ourselves that we could
gather the information we needed. First, we prepared a methodology
plan that was discussed extensively with GPO and DPS, and reviewed by
our methodologists, printing professionals, and external consultants.
Second, we designed our DCI based on the same characteristics as the
DCI used in our last printing survey. We pretested the DCI with DPS
plant officials and provided GPO with examples of our DCI. After
these reviews, we made necessary changes to the forms and procedures
that we believed were warranted on the basis of the information and
comments we received.
15. We used the distribution of duplicating work processes within
DPS' Western Area fiscal year 1993 workload to select a sample of
duplicating requisitions from each of DPS' areas, except for the NCA.
We obtained requisitions from DPS' Western Area fiscal year 1993
workload to determine how many products were included in the sample
and summarized these products on our DCIs. These were sent to seven
DPS areas, where plant officials nominated requisitions from incoming
requests and priced and forwarded them to GPO for production
following normal plant procedures. Thus, while the sampling criteria
were derived from Western Area's workload, the actual fiscal year
1994 requisitions came from seven DPS areas and were produced by GPO
contractors serving those areas. We also used the distribution of
duplicating work processes within DPS' NCA fiscal year 1993 workload
to select a sample of duplicating requisitions from that area.
16. When we sampled the duplicating requisitions, we obtained 26 open
requisitions. These open requisitions represented an ongoing
relationship between a DOD customer and a DPS plant; they were
customer specific rather than product specific. In replicating these
requisitions, the mix of products requested by the customer was more
important than the type of products. Thus, we needed DPS to nominate
products from the same or a similar customer based upon the mix and
volume of the customer's work. Because this process was customer
driven, the open requisitions could not be assigned to areas other
than those from which they were selected. We recognize that open
requisitions exist in other DPS areas. Under our assumption that
work in the Western Area was similar to work in the other areas, we
simulated the Western Area open requisitions in six other areas.
These open requisitions were the only requisitions that were, as GPO
states, "cloned."
17. We disagree, as explained in the Agency Comments and Our
Evaluation section of this report.
18. We disagree. In our post-analysis review of a subsample of items
in our database, information obtained from GPO's database did not
reveal any situations where GPO was allotted significantly less time
than required by our product parameters. Our "outlier" review
yielded similar results. In addition, it should be noted that our
comments cited by GPO in its review of our database were for the use
of the internal panel and were not a final determination by the panel
as to whether an actual problem existed. For example, GPO cites
sample NEA14-31-186 as having the comment "GPO received less than a
'fair share' time allotment." This meant that the number of days GPO
was allowed to perform the work (in this case, 10 days) was lower
than the number of days suggested on the fiscal year 1994 Job Request
form (again in this case, over 10). For this particular requisition
cited, GPO was allotted 10 days to produce 200 copies of a 1-page
document. We did not provide comments for the panel's use if the
number of days allowed for GPO to perform the work was greater than
the number suggested because we did not view this as a situation
where DPS was attempting to bias a selection.
19. See comment 10.
20. See comment 7.
21. DPS plant officials price their work using a standardized pricing
schedule. The schedule contains a detailed listing of the various
production processes--including items such as sundry labor and hand
assembly. Contrary to GPO's statement, we noticed that charges for
these processes were included in DPS sample prices. Regarding rush
charges, DPS financial records show that less than 0.5 percent of its
in-house costs are for rush work. Any omission of these charges in
the price estimates would therefore have had a negligible effect on
our results. Because the DPS' prices were derived using the normal
DPS pricing schedule, the prices would have been the actual prices
charged by DPS if it had produced the work.
We did not provide GPO access to DPS' prices until receipt of final
GPO invoice prices. This was done to prevent any potential GPO bias
in manipulating final prices. However, once the final GPO invoice
price was received, we provided GPO with all data upon request.
22. GPO's statement that we did not look at unreasonable price
differences until GPO reported them is not correct. When we
completed our data gathering, we began a post-analysis review to look
at "outliers"--requisitions that exhibited a large percentage
difference between DPS' and GPO's prices on both ends of the
spectrum--and some errors were found on both DPS' and GPO's prices.
Corrections were made to the data when appropriate.
23. See comment 21. In addition, GPO was provided all sample
requisitions for which we had obtained final GPO invoice prices.
Those sample jobs not provided to GPO did not have final GPO invoice
prices and therefore were not included in our final analysis.
24. As stated above, we did deny GPO access to DPS' prices until
receipt of final GPO invoice prices. This was done to prevent any
potential GPO bias in manipulating the final price. However, once
the final GPO invoice price was received, we provided GPO with all
data upon request. Moreover, GPO received timely access to all of
our workpapers as requested. For example, out of eleven items GPO
requested at the initial meeting following the release of our draft
report, we satisfied eight of them either that same day or the next,
including our entire database. All remaining items were provided
within a week.
25. We disagree. See our Agency Comments and Our Evaluation section.
26. This is addressed in the body of our report.
27. This is addressed in the body of our report.
28. Our effort to collect fiscal year 1994 sample requisitions from
DPS ceased on September 30, 1994. However, because of a time lag in
receiving final GPO invoice prices from contractors, our analysis
could not be completed. The report was drafted and sent to GPO and
DPS for comment only after we had received enough responses and
corresponding final GPO invoice prices to satisfy our standards.
29. Appropriate changes have been made in the report.
30. This issue is addressed in the body of the report.
MAJOR CONTRIBUTORS TO THIS REPORT
=========================================================== Appendix V
NATIONAL SECURITY AND
INTERNATIONAL AFFAIRS DIVISION,
WASHINGTON, D.C.
David R. Warren, Associate Director
John Klotz, Assistant Director
Waverly Sykes, Technical Director
James Reifsnyder, Evaluator-in-Charge
David Combs, Senior Evaluator
Dennis Fauber, Senior Evaluator
Nancy Lively, Senior Evaluator
Richard Meeks, Senior Evaluator
Yolanda Elserwy, Evaluator
Roberta Gaston, Evaluator
David Keefer, Evaluator
Paul Newton, Evaluator
Arthur James, Mathematical Statistician
Mae Jones, Communications Analyst