BNUMBER: B-276864
DATE: July 24, 1997
TITLE: J.A. Jones Management Services, Inc., B-276864, July 24, 1997
**********************************************************************
DOCUMENT FOR PUBLIC RELEASE
A protected decision was issued on the date below and was subject to a
GAO Protective Order. This version has been redacted or approved by
the parties involved for public release.
Matter of:J.A. Jones Management Services, Inc.
File: B-276864
Date:July 24, 1997
William A. Roberts III, Esq., Brian A. Darst, Esq., and Richard P.
Castiglia, Jr., Esq., Howrey & Simon, for the protester.
Jim Robertson, for Intersteel, Inc., an intervenor.
Robert W. Pessolano, Esq., Department of the Army, for the agency.
Andrew T. Pogany, Esq., and Michael R. Golden, Esq., Office of the
General Counsel, GAO, participated in the preparation of the decision.
DIGEST
Agency's contracting officer had inadequate basis for the award of a
contract where the award decision was based on inaccurate and
inadequately documented evaluation team report and recommendation
concerning the relative technical merit of competing proposals.
DECISION
J.A. Jones Management Services, Inc., the incumbent contractor,
protests the award of a contract to Intersteel, Inc. under request for
proposals (RFP) No. DACA27-97-R-0013, issued by the U.S. Army Corps of
Engineers, Louisville District, Kentucky, for base-wide repair and
construction work at Lexington Blue Grass Army Depot. The protester
contends that the agency's technical evaluation was unreasonable and
inaccurate; that the agency failed to adequately document and justify
its source selection decision or to rationally identify the best value
offeror during its evaluation; and that the agency otherwise
misevaluated proposals.
We sustain the protest.
BACKGROUND
The RFP, issued on December 20, 1996, contemplated the award of an
indefinite quantity contract, with individual fixed-price delivery
orders to be issued for specified work, for an 18-month base period
and 2 option years. The RFP stated that award would be made to that
offeror whose offer, conforming to the solicitation, was most
advantageous to the government, price and other factors considered.
The RFP cautioned offerors to submit their best terms from a cost or
price and technical standpoint in their initial proposals since the
agency intended to evaluate proposals and award the contract without
discussions unless the agency subsequently determined that discussions
were necessary. As amended, the RFP contained the following proposal
evaluation criteria, listed in descending order of importance:[1] (1)
management ability (33 points); (2) subcontracting support capability
(25 points); (3) related experience (20 points); (4) coefficient;[2]
(5) technical staff capability (12 points); and (6) financial ability
(8 points). Concerning the evaluation methodology, the RFP stated as
follows:
Proposals will be technically reviewed by qualified evaluators to
initially determine basic conformance with the RFP. Proposals
will be given a quality rating based on the evaluation factors.
Points will be assigned to any proposal in the proportion to the
extent the proposal exceeds minimum evaluation requirements.
The RFP also stated that proposals would be independently and
objectively evaluated as to merit with "[s]ubjective comparisons
[limited] to those areas where it is not feasible to quantify
criteria[, but] final determination of contract award will be based on
the best composite offer, all factors considered." The RFP stated
that evaluation would be inclusive of all options. The minimum
contract value was $250,000 for the base and for each of the three
options; the RFP also identified a maximum value of $4 million for
each of these contract periods.
The agency received 11 proposals by March 6, 1997, the closing date
for receipt of initial proposals. Under the agency's source selection
plan (SSP), the proposals were referred to an evaluation board which
made recommendations to the contracting officer, who was the source
selection authority (SSA).[3] The evaluation board prepared consensus
score sheets containing point scores and cursory narrative
annotations. The evaluation board also prepared for the contracting
officer a very brief summary of its consensus evaluation (consisting
of a total of five or six sentences for each offeror) stating the
results of its evaluation.[4] The evaluation's board technical point
scores and evaluated prices were as follows:[5]
Offeror Technical Score
(Maximum 98 points) Price
Intersteel [deleted] $601,875
Jones [deleted] [deleted]
Offeror A [deleted] [deleted]
Offeror B [deleted] [deleted]
Offeror C [deleted] [deleted]
The evaluation board recommended award to Intersteel. The SSA simply
noted subsequently in a memorandum dated April 16, 1997, that the
"evaluation resulted in selection of Intersteel, Inc.," and selected
that firm for award. This protest followed a debriefing; contract
performance has been stayed.
CONTENTIONS BY THE PROTESTER
Jones challenges the majority of technical point ratings assigned to
itself and Intersteel by the agency evaluators as inaccurate,
unsupported or otherwise improperly based upon the application of
undisclosed evaluation criteria. Jones contends that the agency
report and its supplemental report show that the only explanatory
information or narrative offered by the agency to support the raw
point scores that were assigned is cursory, missing, or "squarely at
odds with information contained in the technical proposals submitted"
by Jones and Intersteel. In other instances, Jones argues that the
weaknesses and strengths assigned to offerors by agency evaluators
bear no relationship to the stated evaluation criteria and that the
point scores are not supported even by the evaluators' own cursory
narratives. Jones also points out that neither in the report nor in
the supplementary comments submitted to our Office does the agency
provide a reference to a single paragraph in either the Jones or
Intersteel technical proposal to justify the scores given to each
company's proposal. Jones notes that, instead, the agency attempts to
defend its flawed evaluation conclusions by simply arguing that the
general procedures used by the evaluators were unobjectionable.
ANALYSIS
In considering protests against an agency's evaluation of proposals,
we will not reevaluate the proposals in order to make our own
determination as to their acceptability or relative merits. Technical
Servs. Corp., B-216408.2, June 5, 1985, 85-1 CPD para. 640 at 6. However,
we will examine the record to determine whether the evaluation was
fair, reasonable and consistent with the evaluation criteria. Bendix
Field Eng'g Corp., B-219406, Oct. 31, 1985, 85-2 CPD para. 496 at 3. We
will also review the documentation supporting the source selection
decision to determine whether that decision was adequately supported
and rationally related to the evaluation factors. Universal Shipping
Co., Inc., B-223905.2, Apr. 20, 1987, 87-1 CPD para. 424, at 10. Implicit
in the foregoing is that the evaluation must be documented in
sufficient detail to show that it was not arbitrary, Adelaide
Blomfield Management Co., B-253128.2, Sept. 27, 1993, 93-2 CPD para. 197
at 4, and a selection decision may not be made on point scores alone
where the agency selection official has inadequate documentation on
which to base a reasoned decision. See Universal Shipping Co., Inc.,
supra, at 10.
For the reasons stated below, we agree with Jones that the agency's
technical evaluation was both inaccurate and unsupported by adequate
documentation, narrative, or information (other than unexplained and
seemingly inaccurate raw point scores) with which the SSA could have
made a rational decision as to which proposal was most advantageous to
the government, from a technical and price standpoint.
As stated above, the three most important technical evaluation
criteria contained in the RFP, management ability, subcontracting
support capability, and related experience represented approximately
80 percent of the total technical score. We will here address, by way
of example, major evaluation flaws evident in the record concerning
the agency's evaluation of numerous subfactors of these major
factors.[6]
Regarding the management ability factor, the offerors were instructed
(subfactor 1B) to provide a "[m]anagement plan for on-site staff to
include a list of proposed management staff, their backgrounds, and
their respective positions with regard to this contract." Jones
presented a management plan which [deleted]. Jones received
[deleted]. The only two "explanations" for [deleted] was the agency
assertion that the protester's [deleted] and that its [deleted].
The protester states and the agency does not dispute that [deleted].
The protester states that the agency evaluators [deleted]. The record
confirms the protester's position. [Deleted]. In its submissions,
the agency has presented no substantive defense to these allegations
of a flawed evaluation by the protester; rather, the agency's
evaluation findings and narratives concerning Jones's management plan
under this subfactor has been shown by the protester to be simply
factually erroneous.
Concerning subfactor 2B of the subcontracting support capability
evaluation factor, the RFP required offerors to provide a "[l]ist of
contemplated subcontractors and their specialty, and rationale for
their selection. [The] list should include both technical and trade
subcontractors where applicable." The record shows that the
protester's technical proposal included [deleted]. Jones also
included [deleted].
The agency evaluators gave Jones [deleted]. The record shows that the
agency's findings were again factually erroneous.
Jones proposed [deleted]. Moreover, the protester correctly points
out that proposing [deleted] was not a stated evaluation factor, which
would generally mean that it should not have been considered by the
agency. The record shows that Intersteel [deleted]. The protester's
proposed [deleted] were apparently overlooked by the agency. We find
therefore that the agency not only misapplied this evaluation factor
but also ignored other specific information presented by Jones in its
proposal, which also contained a rationale and procedures for
selecting subcontractors and its sources for selection of these
subcontractors. The agency again does not present any substantive
defense to these evaluation errors and inaccuracies in its submissions
to our Office, despite having had the opportunity to do so.
Concerning the related experience evaluation factor, the RFP
(subfactor 3A) required offerors to demonstrate their "ability to
manage/construct multiple small to medium scale construction and
repair projects in different locations simultaneously." In its
technical proposal, Jones included [deleted]. One such project listed
by Jones was its current contract for this requirement. In its
technical proposal, Intersteel [deleted].
The agency evaluators again gave Jones [deleted]. A simple
comparison of the two project lists, however, shows that Jones
[deleted]. There is no substantive explanation in the evaluators'
consensus report to justify why the evaluators believed [deleted].
The evaluators [deleted].
Similarly, under this same related experience factor, the RFP
(subfactor 3B) required offerors to provide a "[l]isting of completed
projects within the last 3 years to include duration dates, owners,
and contact points on individual projects." The Jones proposal
included [deleted]. Intersteel presented [deleted]. The agency
recognized that Jones [deleted]. The agency also recognized that
Intersteel [deleted]. A simple comparison of the two listings shows
that [deleted]. Nevertheless, the agency evaluators [deleted]. The
only distinguishing comment in the agency's narrative under this
subfactor is a reference to [deleted]. However, the agency evaluators
apparently overlooked the fact that [deleted]. The agency again has
provided no substantive explanation for this scoring either
contemporaneously or as part of this protest proceeding.
The protester, in its comments on the initial agency report and its
comments on the supplemental report filed by the agency, has
identified other alleged evaluation errors concerning response time
between the prime contractor and its subcontractors (subfactor 2A),
administration of payroll and labor related functions (subfactor 1E),
technical staff capability and qualifications of its project manager
(subfactor 5A), and home office support/financial ability (subfactor
6A). In its submission, the agency fails to provide reference to a
single paragraph in either the Jones or Intersteel technical proposal
to justify the respective scores given to each company under the
numerous factors and subfactors whose scoring the protester alleges
was unreasonable and inaccurate. Rather than providing such
substantive responses to the protester's specific substantive
challenges, the agency merely attempts to defend the overall process
by which technical proposals were evaluated and relies on the general
discretion afforded procuring agencies in evaluating proposals. For
example, the agency asserts that each proposal is reviewed
independently during the evaluation process and is "not compared
and/or contrasted with other competing proposals, but [is] compared
and contrasted only with [the] evaluation subfactor. The resulting
scores are an independent assessment representing the consensus of the
evaluation panel of the qualitative and quantitative merit of that
individual proposal with respect to the individual subfactor being
evaluated at that point in time."[7]
While an agency's initial review of proposals may be done
independently, it is necessary for the agency at some point to make a
rational comparison of the relative merits of directly competing
proposals. The agency's SSA cannot make a rational comparison where
the evaluators' point scores do not accurately reflect the relative
strengths and weaknesses of the competing proposals and the record
lacks adequate documentation in support of such technical point
scores. Here, the dearth of evaluation narratives or other
explanations in this evaluation record and the SSA's summary reliance
on the point scores, which have been sufficiently shown by the
protester to be inaccurate in numerous areas, lead us to the
conclusion that the SSA could not and did not make a reasonable
selection decision. See Adelaide Blomfield Management Co., supra, at
4-6. In sum, on the evaluation record furnished to our
Office--consisting of point scores and cursory narratives--we find the
absence of a consistent, reasonable and accurate evaluation scoring as
well as the absence of supporting narrative documentation; moreover,
the agency has offered no substantive response to the protester's
concerns in response to the protest, and our review shows none. Here,
we simply are unable to assess the reasonableness of the agency's
selection decision. Accordingly, we sustain the protest.
We recommend that the agency reevaluate all proposals consistent with
the RFP criteria, and provide adequate, rational documentation
supporting the scores awarded. If the result of the reevaluation is
that Intersteel's proposal is not found to be the most advantageous
offer, the agency should terminate the contract with Intersteel and
award the contract to the firm whose proposal is selected.
Additionally, we recommend that the protester be reimbursed its costs
of filing and pursuing the protest, including reasonable attorneys'
fees. Bid Protest Regulations, 4 C.F.R. sec. 21.8(d)(1) (1997). The
protester should submit its certified claim for costs to the
contracting agency within 60 days of receiving this decision. Bid
Protest Regulations, 4 C.F.R. sec. 21.8(f)(1).
The protest is sustained.
Comptroller General
of the United States
1. The specific point values assigned each criterion were not revealed
in the RFP. The maximum technical score attainable was 98 points.
2. Concerning price, the RFP required each offeror to include a
coefficient (percentage factor) to be applied to specified fixed rates
for various tasks contained in a unit price book, provided by the
agency in the RFP. Upon the issuance of individual delivery orders,
the rates set forth in the unit price book for the various tasks and
services to be performed would be multiplied by the successful
contractor's coefficient, which would become the price for the
particular item or items set forth in the delivery order.
3. The SSP contained the following provisions:
Source selection decisions shall not be made on the basis
of a score alone. The decision shall be made on the basis
of an assessment of the evaluation results as a whole
[consistent with the evaluation criteria]. Scores shall
be established as a result of a consensus of the
evaluators. . . . The judgment of the board shall be
reduced to writing so that a reader may understand the
rationale for selection from one document to another. . .
. Evaluation results will be stated as a narrative
discussion of the value of each offer in terms of its
advantages and disadvantages, its deficiencies and the
effect of these elements should the offer be accepted.
Terms such as 'shortcomings' or 'weaknesses' shall not be
used. The supporting documentation shall include the
basis and reasons for the selection decision.
4. These two documents (the consensus score sheets and the brief
summary of the evaluation results) are the extent of the agency's
technical evaluation record.
5. We list the results for only 5 of the 11 offerors, and henceforth
limit our discussion to the evaluation of the protester's and
awardee's proposals.
6. Despite being given an opportunity to do so, the agency has not
substantively disputed the flaws in the evaluation alleged by the
protester as having occurred. We also agree with the protester that
the agency's evaluation record was extremely cursory in terms of
narrative explanation for point scores awarded.
7. The agency also contends that the narratives in the consensus score
sheets were only "impression[s] of the proposal in the evaluation
subfactor as expressed as a bullet command [and] do not solely
represent the justification for the rating given or score given in
that particular area." The protester characterizes this argument as a
"post-hoc repudiation [by the agency] of its own evaluation
narratives." We merely note that without these "bullet narratives"
the agency had no documentation whatsoever to support the scoring or
the selection decision.