BNUMBER:  B-275209
DATE:  January 30, 1997
TITLE:  JW Associates Inc.

**********************************************************************

DOCUMENT FOR PUBLIC RELEASE
A protected decision was issued on the date below and was subject to a 
GAO Protective Order.  This version has been redacted or approved by 
the parties involved for public release.
Matter of:JW Associates Inc.

File:     B-275209

Date:January 30, 1997

Brad Piehl for the protester.
Allen W. Smith, Department of Agriculture, Forest Service, for the 
agency.
Christina Sklarew, Esq., and Michael R. Golden, Esq., Office of the 
General Counsel, GAO, participated in the preparation of the decision.

DIGEST

Source selection cannot be determined to be reasonable where 
evaluation record contains no meaningful evaluation of offers, but 
only conclusory statements which do not permit an understanding of the 
technical differences between proposals, and agency fails to rebut 
protester's specific allegations that its proposal and that of the 
awardee were misevaluated.

DECISION

JW Associates Inc. protests the award of a firm, fixed-price contract 
to Natural Resources Management Corporation (NRMC) under request for 
proposals (RFP)
No. RM-96-37, issued by the Forest Service for services to complete an 
environmental impact statement for the Cold Springs Analysis Area on 
the Medicine Bow National Forest in Wyoming.  JW alleges that the 
Forest Service's evaluation of proposals was inconsistent with the 
terms established in the RFP, and that the awardee's proposal did not 
conform with the RFP's requirements.

We sustain the protest.

The RFP was issued on August 6, 1996, as a total small business 
set-aside procurement.  Offerors were instructed to submit separate 
technical and business proposals, to allow technical merit and cost to 
be considered separately.  The RFP at M3 listed the following five 
evaluation criteria in descending order of importance:  qualifications 
of the firm; qualifications of the personnel assigned to the project; 
past experience of the firm and its employees assigned to the project; 
geographic location; and price.  The RFP characterized the first three 
criteria as "very important" and the remaining two as "important."  
The RFP also listed the type of information that should be provided 
under each criterion.  The RFP at M4 stated that award would be made 
to "that offeror (1) whose proposal is technically acceptable and (2) 
whose technical/cost relationship is the most advantageous to the 
Government."  The RFP further confirmed that it would perform a 
cost/technical trade-off to determine the most advantageous offer.  

The Forest Service received proposals from [deleted] firms, including 
JW and NRMC.  An evaluation board, made up of three Forest Service 
specialists, evaluated the technical proposals.  Each member assigned 
a technical score to each proposal, which were then averaged; 
proposals were ranked on the basis of the average scores.  Two 
proposals were determined to be technically unacceptable.  Of the 
remaining [deleted] proposals, NRMC's received the [deleted] averaged 
technical score with [deleted].  JW's proposal received the [deleted] 
highest score with an average of [deleted] points. 

Business proposals were evaluated based on the total price offered for 
the base and option items, as provided in the RFP.  NRMC's price was 
lowest among the proposals that were rated technically acceptable, and 
JW's was sixth low.  

The board prepared a "Memorandum for Record" that consisted of a 
one-paragraph narrative summary of each proposal's evaluation.  The 
contracting officer concluded that NRMC's proposal was the one that 
was most advantageous to the government, and awarded the contract to 
this firm on October 7.  When JW received notice of the award on 
October 9, it requested a debriefing.  JW received a debriefing letter 
and a copy of the awardee's technical proposal on October 17.[1]  This 
protest followed on October 22.  Because our Office notified the 
Forest Service of the protest within 5 days of the 
statutorily-required debriefing, the agency was required, under 31 
U.S.C.  sec.  3553 (1994), to stay the performance of the contract pending 
resolution of the protest.  Although the agency initially did stay 
performance, on November 6, the head of the contracting activity 
approved a request by the forest supervisor to override the stay and 
proceed with performance.[2]  

JW argues that the Forest Service's evaluation of proposals and its 
resulting source selection decision were improper and inconsistent 
with the evaluation scheme that was established in the RFP.  Comparing 
its own proposal to the awardee's, JW concludes that it should have 
received a significantly higher score under those criteria, if 
properly applied.  For example, under the first (and therefore most 
heavily weighted) criterion, "qualifications of the firm," JW points 
out that it has recently completed two timber sale environmental 
impact statements and is currently performing a third one for the 
Forest Service in roadless areas in National Forests in Idaho.  JW 
notes a number of factors that are very similar to the requirement at 
issue here, including the location in roadless areas, extensive areas 
of older lodgepole pine, the presence of risk/damage from mountain 
pine beetles, extensive intermixed land ownership, and the importance 
of water quality and biodiversity to the site.  In addition, JW points 
out its direct experience with ecosystem management under those 
contracts, and that it offers extensive experience with the National 
Environmental Policy Act of 1969 (NEPA), 42 U.S.C.  sec.  4321 et seq. 
(1994), the legislation that prescribes environmental impact studies.  
JW asserts that its current experience with the Forest Service 
includes project management, technical management, public involvement, 
technical writing, and hydrology expertise, all skills essential to 
performing this contract.  On the other hand, JW points out that the 
awardee's proposal reflects no experience in completing an 
environmental impact statement.  The protester notes that NRMC does 
not mention in its proposal experience with NEPA, project management, 
technical management, public involvement, technical writing or 
ecosystem management; it lists only forestry experience.  JW concludes 
that its own score for this criterion should have been higher than 
NRMC's.  JW provides a similar comparison of its proposal with the 
awardee's on the remaining technical factors which supports its view 
that its technical proposal should have been evaluated significantly 
superior to the awardee's.

In reviewing protests against allegedly improper evaluations, it is 
not our role to reevaluate proposals.  Rather, our Office examines the 
record to determine whether the agency's judgment was reasonable and 
in accord with the RFP's stated evaluation criteria.  Abt Assocs., 
Inc., B-237060.2, Feb. 26, 1990, 90-1 CPD  para.  223.  In order for us to 
review an agency's selection determination, an agency must have 
adequate documentation to support its selection decision.  Arco 
Management of Washington, D.C., Inc., B-248653, Sept. 11, 1992, 92-2 
CPD  para.  173.  While both adjectival ratings and point scores are useful 
as guides to decision-making, they generally are not controlling, but 
rather, must be supported by documentation of the relative differences 
between proposals, their weaknesses and risks, and the basis and 
reasons for the selection decision.  Federal Acquisition Regulation 
(FAR)  sec.  15.608(a)(3) and 15.612(d)(2); Engineering and Computation, 
Inc., B-261658, 
Oct. 16, 1995, 95-2 CPD  para.  176.

After reviewing all of the supporting documentation submitted by the 
Forest Service, we conclude that the technical evaluation is not 
adequately supported.  We also conclude that without adequate support 
for the technical evaluation, a proper award determination could not 
be made.  Engineering and Computation, Inc., supra;  see Redstone 
Technical Servs.; Dynamic Science, Inc., B-259222 et al., Mar. 17, 
1995, 95-1 CPD  para.  181.

The evaluation record consists of a chart which lists the offerors, 
the three evaluators' total scores for each offeror, the average score 
of the evaluators, and each firm's price. This chart ranks firms 
according to their price.  The record also contains a memorandum for 
the record.  The memorandum consists of a one-paragraph evaluation 
summary of each offeror.  For NRMC, the memorandum states:

     "Natural Resources Management Corporation presented a proposal 
     which offered to do the most for the money, or in other words, 
     providing the most acceptable proposal whose technical/cost 
     relationship was the most advantageous to the Government.  The 
     firm was considered to be highly qualified having lots of 
     experience in the land management field and forestry.  The firm 
     proposed establishing a field office in Wyoming for the duration 
     of this project which would be of benefit to the Forest.  The 
     only concern of the Board's was that the firm did not have any 
     roadless experience.  But this lack of experience was not enough 
     to eliminate the firm from being placed in the competitive 
     range."

Regarding JW's proposal, the memorandum states:

     "JW Associates, Inc., was ranked [deleted] making the most 
     advantageous offer to the Government.  The proposed project 
     schedule was considered to be good by the Board.  The firm 
     proposed using primarily two employees and using a lot of 
     subcontractors.  It appeared that the subcontractors had good 
     experience working together.  The primary reason for this firm's 
     ranking was due to its cost."

Further, in response to the protest, the Forest Service prepared a 
document entitled "Agency Reply to Protest," in which it generally 
states that the evaluation was fairly performed in accordance with the 
solicitation and responds to each of the protester's allegations with 
conclusory statements.  For example, for the protest basis at issue, 
that technical proposals were misevaluated, the agency reply repeats 
the evaluation criteria listed in the RFP and states that: 

     "Each Board member evaluated each technical proposal 
     independently and rated each proposal based upon what was 
     submitted.  Once all of the technical proposals were 
     independently evaluated, the Board members then discussed amongst 
     themselves the technical proposals.  As a result of this 
     discussion, technical proposals were determined to be either 
     acceptable or not acceptable to the Government.

     The Contracting Officer directed the Board members to evaluate 
     each proposal using only the evaluation criteria listed in the 
     solicitation.  An evaluation form was used by the Board to assist 
     with the evaluations.  The form listed each of the evaluation 
     criteria and their relative value.  No other criteria were used 
     during the evaluation.  A Memorandum for Record [from which we 
     quoted in pertinent part, above] was prepared providing reasons 
     for the rankings."

The report does not include any individual technical evaluation sheets 
or other contemporaneous documentation of the evaluation of the 
various evaluation criteria, but provided only the chart with the 
total score that each evaluator assigned to each proposal.  In 
contrast to the very specific and detailed allegations that the 
protester has raised, pointing out specific areas in its proposal that 
allegedly should have been evaluated more highly, such as under 
"qualifications of the firms,"  discussed above, the Forest Service 
has provided no specific rebuttal, nor has it provided any support or 
explanation for its conclusions.  Other examples of unrebutted, 
specific allegations include the protester's assertion that it should 
have been evaluated higher than the awardee because its own proposed 
personnel are more highly qualified than the awardee's, an allegation 
that it supports by comparing the education, background and experience 
of its own personnel and the awardee's proposed personnel; its 
assertion that the level of effort that NRMC proposes is inadequate to 
complete the work, pointing out, for example, that the awardee 
proposes to spend a total of two-thirds fewer hours on project 
management compared to JW's proposed hours, and the assertion that 
NRMC's proposal fails to include a detailed work plan or to disclose 
the firm's technical approach in sufficient detail, as required by the 
RFP.  None of these issues is addressed in the one-paragraph summary, 
or otherwise explained in the record.  In fact, there is nothing in 
the record furnished by the Forest Service which shows how each 
proposal was evaluated under each technical evaluation factor.  Thus, 
we have no basis on this record to assess the reasonableness of either 
JW's or NRMC's evaluation.   

In addition to the Forest Service's failure to support its technical 
evaluation in any meaningful way, the record suggests that the source 
selection decision was based primarily on price--the least important 
factor listed in the evaluation scheme.  With the exception of the 
paragraph in the Memorandum concerning NRMC's proposal, quoted above, 
every paragraph in the Memorandum summarizing the agency's basis[3] 
for its ranking of the individual proposals concludes that "[t]he 
primary reason for this firm's ranking was due to its cost."  Indeed, 
as stated above, the ranking of the proposals [deleted].  The 
contracting officer's debriefing letter confirms that price was the 
basis for the agency's award decision.   

While price, where lower in importance, can become the determining 
factor between proposals that have been evaluated as technically equal 
in a "best value" award decision, Ogilvy, Adams & Rinehart, 
B-246172.2, Apr. 1, 1992, 92-1 CPD  para.  332, the record in this case does 
not contain sufficient information and analysis to establish the 
equality of these two proposals; moreover, there is no evidence in the 
record that any such conclusion was reached.  To the contrary, as 
discussed above, the protester has provided an analysis of its 
proposal and the awardee's which reasonably shows that its proposal is 
superior to the awardee's in the three most heavily weighted technical 
areas--qualifications of the firm, personnel, and past experience.  
Further, the protester points to the RFP evaluation language which 
ranked technical merit as significantly more important than price.  JW 
states that its higher price was based on its proposal of high quality 
personnel and a level of effort commensurate with the agency's needs 
as announced in the RFP.

FAR  sec.  15.612(d)(2) requires that the documentation supporting 
selection decisions show the relative differences among proposals; 
their strengths, weaknesses and risks; and the basis and reasons for 
the decisions.  This required explanation provides protesters and this 
Office a basis upon which to judge the reasonableness of the agency's 
decision and, ultimately, its compliance with the procurement statutes 
and regulations.  TFA, Inc., B-243875, Sept. 11, 1991, 91-2 CPD  para.  239.  
Where, as here, the agency has not adequately documented its 
evaluation process and has failed to rebut the protester's very 
specific arguments that its own proposal should have received a higher 
technical score than the awardee's, we have no basis to conclude that 
the selection decision was reasonable.

The protest is sustained.

By letter of today to the Secretary of Agriculture, we are 
recommending that the agency, in accordance with the FAR, reevaluate 
the proposals, document its evaluation, and make a new selection 
decision.[4]  If after reevaluation, the Forest Service believes 
discussions with offerors are warranted, it may open discussions and 
request best and final offers.  If the agency selects an offeror other 
than NRMC for award, it should terminate the contract to NRMC and make 
award to that other offeror.  We also recommend that the protester be 
reimbursed its costs of filing and pursuing its protest.  Bid Protest 
Regulations,  sec.  21.8(d)(1), 61 Fed. Reg. 39039, 39046 (1996) (to be 
codified at 4 C.F.R.  sec.  21.8).  The protester should submit its 
certified claim for costs to the contracting agency within 60 days of 
receiving this decision.  Bid Protest Regulations,  sec.  21.8(f)(1), 
supra.

The protest is sustained.

Comptroller General
of the United States 

1. For reasons that are not clear, the Forest Service provided JW with 
a copy of the awardee's technical proposal along with its debriefing 
letter.

2. The statute permits an override of the stay on the basis of either 
urgency or best interests of the government.  The override 
determination does not identify the basis for the override.

3. The RFP creates confusion concerning the evaluation scheme.  While 
M3 and M4(2) envision a best value approach, M4(1) introduces the 
concept of technical acceptablility which is a scheme that results in 
award to the low cost offeror.  It appears from the record that the 
Forest Service may have inappropriately followed the latter approach.

4. Although JW raises a number of challenges to the evaluation and 
source selection decisions in its protest, we have not discussed each 
protest basis.  However, we recommend that the Forest Service, during 
its reevaluation of proposals, take into consideration JW's 
allegations that NRMC's proposal does not meet the requirements of the 
RFP because it fails to provide certain information.