BNUMBER:  B-276864 
DATE:  July 24, 1997
TITLE: J.A. Jones Management Services, Inc., B-276864, July 24, 1997
**********************************************************************

DOCUMENT FOR PUBLIC RELEASE
A protected decision was issued on the date below and was subject to a 
GAO Protective Order.  This version has been redacted or approved by 
the parties involved for public release.
Matter of:J.A. Jones Management Services, Inc.

File:     B-276864

Date:July 24, 1997

William A. Roberts III, Esq., Brian A. Darst, Esq., and Richard P. 
Castiglia, Jr., Esq., Howrey & Simon, for the protester.
Jim Robertson, for Intersteel, Inc., an intervenor.
Robert W. Pessolano, Esq., Department of the Army, for the agency.
Andrew T. Pogany, Esq., and Michael R. Golden, Esq., Office of the 
General Counsel, GAO, participated in the preparation of the decision.

DIGEST

Agency's contracting officer had inadequate basis for the award of a 
contract where the award decision was based on inaccurate and 
inadequately documented evaluation team report and recommendation 
concerning the relative technical merit of competing proposals.

DECISION

J.A. Jones Management Services, Inc., the incumbent contractor, 
protests the award of a contract to Intersteel, Inc. under request for 
proposals (RFP) No. DACA27-97-R-0013, issued by the U.S. Army Corps of 
Engineers, Louisville District, Kentucky, for base-wide repair and 
construction work at Lexington Blue Grass Army Depot.  The protester 
contends that the agency's technical evaluation was unreasonable and 
inaccurate; that the agency failed to adequately document and justify 
its source selection decision or to rationally identify the best value 
offeror during its evaluation; and that the agency otherwise 
misevaluated proposals.

We sustain the protest.

BACKGROUND

The RFP, issued on December 20, 1996, contemplated the award of an 
indefinite quantity contract, with individual fixed-price delivery 
orders to be issued for specified work, for an 18-month base period 
and 2 option years.  The RFP stated that award would be made to that 
offeror whose offer, conforming to the solicitation, was most 
advantageous to the government, price and other factors considered.  
The RFP cautioned offerors to submit their best terms from a cost or 
price and technical standpoint in their initial proposals since the 
agency intended to evaluate proposals and award the contract without 
discussions unless the agency subsequently determined that discussions 
were necessary.  As amended, the RFP contained the following proposal 
evaluation criteria, listed in descending order of importance:[1]  (1) 
management ability (33 points); (2) subcontracting support capability 
(25 points); (3) related experience (20 points); (4) coefficient;[2]             
(5) technical staff capability (12 points); and (6) financial ability 
(8 points).  Concerning the evaluation methodology, the RFP stated as 
follows:

     Proposals will be technically reviewed by qualified evaluators to 
     initially determine basic conformance with the RFP.  Proposals 
     will be given a quality rating based on the evaluation factors.  
     Points will be assigned to any proposal in the proportion to the 
     extent the proposal exceeds minimum evaluation requirements.

The RFP also stated that proposals would be independently and 
objectively evaluated as to merit with "[s]ubjective comparisons 
[limited] to those areas where it is not feasible to quantify 
criteria[, but] final determination of contract award will be based on 
the best composite offer, all factors considered."  The RFP stated 
that evaluation would be inclusive of all options.  The minimum 
contract value was $250,000 for the base and for each of the three 
options; the RFP also identified a maximum value of $4 million for 
each of these contract periods.

The agency received 11 proposals by March 6, 1997, the closing date 
for receipt of initial proposals.  Under the agency's source selection 
plan (SSP), the proposals were referred to an evaluation board which 
made recommendations to the contracting officer, who was the source 
selection authority (SSA).[3]  The evaluation board prepared consensus 
score sheets containing point scores and cursory narrative 
annotations.  The evaluation board also prepared for the contracting 
officer a very brief summary of its consensus evaluation (consisting 
of a total of     five or six sentences for each offeror) stating the 
results of its evaluation.[4]  The evaluation's board technical point 
scores and evaluated prices were as follows:[5]

       Offeror       Technical Score
                   (Maximum 98 points)      Price

     Intersteel         [deleted]         $601,875

        Jones           [deleted]         [deleted]

      Offeror A         [deleted]         [deleted]

      Offeror B         [deleted]         [deleted]

      Offeror C         [deleted]         [deleted]

The evaluation board recommended award to Intersteel.  The SSA simply 
noted subsequently in a memorandum dated April 16, 1997, that the 
"evaluation resulted in selection of Intersteel, Inc.," and selected 
that firm for award.  This protest followed a debriefing; contract 
performance has been stayed.

CONTENTIONS BY THE PROTESTER

Jones challenges the majority of technical point ratings assigned to 
itself and Intersteel by the agency evaluators as inaccurate, 
unsupported or otherwise improperly based upon the application of 
undisclosed evaluation criteria.  Jones contends that the agency 
report and its supplemental report show that the only explanatory 
information or narrative offered by the agency to support the raw 
point scores that were assigned is cursory, missing, or "squarely at 
odds with information contained in the technical proposals submitted" 
by Jones and Intersteel.  In other instances, Jones argues that the 
weaknesses and strengths assigned to offerors by agency evaluators 
bear no relationship to the stated evaluation criteria and that the 
point scores are not supported even by the evaluators' own cursory 
narratives.  Jones also points out that neither in the report nor in 
the supplementary comments submitted to our Office does the agency 
provide a reference to a single paragraph in either the Jones or 
Intersteel technical proposal to justify the scores given to each 
company's proposal.  Jones notes that, instead, the agency attempts to 
defend its flawed evaluation conclusions by simply arguing that the 
general procedures used by the evaluators were unobjectionable.

ANALYSIS

In considering protests against an agency's evaluation of proposals, 
we will not reevaluate the proposals in order to make our own 
determination as to their acceptability or relative merits.  Technical 
Servs. Corp., B-216408.2, June 5, 1985, 85-1 CPD  para.  640 at 6.  However, 
we will examine the record to determine whether the evaluation was 
fair, reasonable and consistent with the evaluation criteria.  Bendix 
Field Eng'g Corp., B-219406, Oct. 31, 1985, 85-2 CPD  para.  496 at 3.  We 
will also review the documentation supporting the source selection 
decision to determine whether that decision was adequately supported 
and rationally related to the evaluation factors.  Universal Shipping 
Co., Inc., B-223905.2, Apr. 20, 1987, 87-1 CPD  para.  424, at 10.  Implicit 
in the foregoing is that the evaluation must be documented in 
sufficient detail to show that it was not arbitrary, Adelaide 
Blomfield Management Co., B-253128.2, Sept. 27, 1993, 93-2 CPD  para.  197 
at 4, and a selection decision may not be made on point scores alone 
where the agency selection official has inadequate documentation on 
which to base a reasoned decision.  See Universal Shipping Co., Inc., 
supra, at 10.

For the reasons stated below, we agree with Jones that the agency's 
technical evaluation was both inaccurate and unsupported by adequate 
documentation, narrative, or information (other than unexplained and 
seemingly inaccurate raw point scores) with which the SSA could have 
made a rational decision as to which proposal was most advantageous to 
the government, from a technical and price standpoint.

As stated above, the three most important technical evaluation 
criteria contained in the RFP, management ability, subcontracting 
support capability, and related experience represented approximately 
80 percent of the total technical score.  We will here address, by way 
of example, major evaluation flaws evident in the record concerning 
the agency's evaluation of numerous subfactors of these major 
factors.[6]  
Regarding the management ability factor, the offerors were instructed 
(subfactor 1B) to provide a "[m]anagement plan for on-site staff to 
include a list of proposed management staff, their backgrounds, and 
their respective positions with regard to this contract."  Jones 
presented a management plan which [deleted].  Jones received 
[deleted].  The only two "explanations" for [deleted] was the agency 
assertion that the protester's [deleted] and that its [deleted].

The protester states and the agency does not dispute that [deleted].  
The protester states that the agency evaluators [deleted].  The record 
confirms the protester's position.  [Deleted].  In its submissions, 
the agency has presented no substantive defense to these allegations 
of a flawed evaluation by the protester; rather, the agency's 
evaluation findings and narratives concerning Jones's management plan 
under this subfactor has been shown by the protester to be simply 
factually erroneous.

Concerning subfactor 2B of the subcontracting support capability 
evaluation factor, the RFP required offerors to provide a "[l]ist of 
contemplated subcontractors and their specialty, and rationale for 
their selection.  [The] list should include both technical and trade 
subcontractors where applicable."  The record shows that the 
protester's technical proposal included [deleted].  Jones also 
included [deleted].

The agency evaluators gave Jones [deleted].  The record shows that the 
agency's findings were again factually erroneous.

Jones proposed [deleted].  Moreover, the protester correctly points 
out that proposing [deleted] was not a stated evaluation factor, which 
would generally mean that it should not have been considered by the 
agency.  The record shows that Intersteel [deleted].  The protester's 
proposed [deleted] were apparently overlooked by the agency.  We find 
therefore that the agency not only misapplied this evaluation factor 
but also ignored other specific information presented by Jones in its 
proposal, which also contained a rationale and procedures for 
selecting subcontractors and its sources for selection of these 
subcontractors.  The agency again does not present any substantive 
defense to these evaluation errors and inaccuracies in its submissions 
to our Office, despite having had the opportunity to do so.

Concerning the related experience evaluation factor, the RFP 
(subfactor 3A) required offerors to demonstrate their "ability to 
manage/construct multiple small to medium scale construction and 
repair projects in different locations simultaneously."  In its 
technical proposal, Jones included [deleted].  One such project listed 
by Jones was its current contract for this requirement.  In its 
technical proposal, Intersteel [deleted].

The agency evaluators again gave Jones [deleted].   A simple 
comparison of the two project lists, however, shows that Jones 
[deleted].  There is no substantive explanation in the evaluators' 
consensus report to justify why the evaluators believed [deleted].  
The evaluators [deleted].

Similarly, under this same related experience factor, the RFP 
(subfactor 3B) required offerors to provide a "[l]isting of completed 
projects within the last 3 years to include duration dates, owners, 
and contact points on individual projects."  The Jones proposal 
included [deleted].  Intersteel presented [deleted].  The agency 
recognized that Jones [deleted].  The agency also recognized that 
Intersteel [deleted].  A simple comparison of the two listings shows 
that [deleted]. Nevertheless, the agency evaluators [deleted].  The 
only distinguishing comment in the agency's narrative under this 
subfactor is a reference to [deleted].  However, the agency evaluators 
apparently overlooked the fact that [deleted].  The agency again has 
provided no substantive explanation for this scoring either 
contemporaneously or as part of this protest proceeding.

The protester, in its comments on the initial agency report and its 
comments on the supplemental report filed by the agency, has 
identified other alleged evaluation errors concerning response time 
between the prime contractor and its subcontractors (subfactor 2A), 
administration of payroll and labor related functions (subfactor 1E), 
technical staff capability and qualifications of its project manager 
(subfactor 5A), and home office support/financial ability (subfactor 
6A).  In its submission, the agency fails to provide reference to a 
single paragraph in either the Jones or Intersteel technical proposal 
to justify the respective scores given to each company under the 
numerous factors and subfactors whose scoring the protester alleges 
was unreasonable and inaccurate.  Rather than providing such 
substantive responses to the protester's specific substantive 
challenges, the agency merely attempts to defend the overall process 
by which technical proposals were evaluated and relies on the general 
discretion afforded procuring agencies in evaluating proposals.  For 
example, the agency asserts that each proposal is reviewed 
independently during the evaluation process and is "not compared 
and/or contrasted with other competing proposals, but [is] compared 
and contrasted only with [the] evaluation subfactor.  The resulting 
scores are an independent assessment representing the consensus of the 
evaluation panel of the qualitative and quantitative merit of that 
individual proposal with respect to the individual subfactor being 
evaluated at that point in time."[7]

While an agency's initial review of proposals may be done 
independently, it is necessary for the agency at some point to make a 
rational comparison of the relative merits of directly competing 
proposals.  The agency's SSA cannot make a rational comparison where 
the evaluators' point scores do not accurately reflect the relative 
strengths and weaknesses of the competing proposals and the record 
lacks adequate documentation in support of such technical point 
scores.  Here, the dearth of evaluation narratives or other 
explanations in this evaluation record and the SSA's summary reliance 
on the point scores, which have been sufficiently shown by the 
protester to be inaccurate in numerous areas, lead us to the 
conclusion that the SSA could not and did not make a reasonable 
selection decision.  See Adelaide Blomfield Management Co., supra, at 
4-6.  In sum, on the evaluation record furnished to our 
Office--consisting of point scores and cursory narratives--we find the 
absence of a consistent, reasonable and accurate evaluation scoring as 
well as the absence of supporting narrative documentation; moreover, 
the agency has offered no substantive response to the protester's 
concerns in response to the protest, and our review shows none.  Here, 
we simply are unable to assess the reasonableness of the agency's 
selection decision.  Accordingly, we sustain the protest.

We recommend that the agency reevaluate all proposals consistent with 
the RFP criteria, and provide adequate, rational documentation 
supporting the scores awarded.  If the result of the reevaluation is 
that Intersteel's proposal is not found to be the most advantageous 
offer, the agency should terminate the contract with Intersteel and 
award the contract to the firm whose proposal is selected.  
Additionally, we recommend that the protester be reimbursed its costs 
of filing and pursuing the protest, including reasonable attorneys' 
fees.  Bid Protest Regulations, 4 C.F.R.  sec.  21.8(d)(1) (1997).  The 
protester should submit its certified claim for costs to the 
contracting agency within 60 days of receiving this decision.  Bid 
Protest Regulations, 4 C.F.R.  sec.  21.8(f)(1).

The protest is sustained.

Comptroller General
of the United States

1. The specific point values assigned each criterion were not revealed 
in the RFP.  The maximum technical score attainable was 98 points.

2. Concerning price, the RFP required each offeror to include a 
coefficient (percentage factor) to be applied to specified fixed rates 
for various tasks contained in a unit price book, provided by the 
agency in the RFP.  Upon the issuance of individual delivery orders, 
the rates set forth in the unit price book for the various tasks and 
services to be performed would be multiplied by the successful 
contractor's coefficient, which would become the price for the 
particular item or items set forth in the delivery order.

3. The SSP contained the following provisions:

            Source selection decisions shall not be made on the basis 
            of a score alone.  The decision shall be made on the basis 
            of an assessment of the evaluation results as a whole 
            [consistent with the evaluation criteria].  Scores shall 
            be established as a result of a consensus of the 
            evaluators. . . .  The judgment of the board shall be 
            reduced to writing so that a reader may understand the 
            rationale for selection from one document to another. . . 
            .  Evaluation results will be stated as a narrative 
            discussion of the value of each offer in terms of its 
            advantages and disadvantages, its deficiencies and the 
            effect of these elements should the offer be accepted.  
            Terms such as 'shortcomings' or 'weaknesses' shall not be 
            used.  The supporting documentation shall include the 
            basis and reasons for the selection decision.

4. These two documents (the consensus score sheets and the brief 
summary of the evaluation results) are the extent of the agency's 
technical evaluation record.

5. We list the results for only 5 of the 11 offerors, and henceforth 
limit our discussion to the evaluation of the protester's and 
awardee's proposals.

6. Despite being given an opportunity to do so, the agency has not 
substantively disputed the flaws in the evaluation alleged by the 
protester as having occurred.  We also agree with the protester that 
the agency's evaluation record was extremely cursory in terms of 
narrative explanation for point scores awarded.

7. The agency also contends that the narratives in the consensus score 
sheets were only "impression[s] of the proposal in the evaluation 
subfactor as expressed as a bullet command [and] do not solely 
represent the justification for the rating given or score given in 
that particular area."  The protester characterizes this argument as a 
"post-hoc repudiation [by the agency] of its own evaluation 
narratives."  We merely note that without these "bullet narratives" 
the agency had no documentation whatsoever to support the scoring or 
the selection decision.