BNUMBER: B-275209
DATE: January 30, 1997
TITLE: JW Associates Inc.
**********************************************************************
DOCUMENT FOR PUBLIC RELEASE
A protected decision was issued on the date below and was subject to a
GAO Protective Order. This version has been redacted or approved by
the parties involved for public release.
Matter of:JW Associates Inc.
File: B-275209
Date:January 30, 1997
Brad Piehl for the protester.
Allen W. Smith, Department of Agriculture, Forest Service, for the
agency.
Christina Sklarew, Esq., and Michael R. Golden, Esq., Office of the
General Counsel, GAO, participated in the preparation of the decision.
DIGEST
Source selection cannot be determined to be reasonable where
evaluation record contains no meaningful evaluation of offers, but
only conclusory statements which do not permit an understanding of the
technical differences between proposals, and agency fails to rebut
protester's specific allegations that its proposal and that of the
awardee were misevaluated.
DECISION
JW Associates Inc. protests the award of a firm, fixed-price contract
to Natural Resources Management Corporation (NRMC) under request for
proposals (RFP)
No. RM-96-37, issued by the Forest Service for services to complete an
environmental impact statement for the Cold Springs Analysis Area on
the Medicine Bow National Forest in Wyoming. JW alleges that the
Forest Service's evaluation of proposals was inconsistent with the
terms established in the RFP, and that the awardee's proposal did not
conform with the RFP's requirements.
We sustain the protest.
The RFP was issued on August 6, 1996, as a total small business
set-aside procurement. Offerors were instructed to submit separate
technical and business proposals, to allow technical merit and cost to
be considered separately. The RFP at M3 listed the following five
evaluation criteria in descending order of importance: qualifications
of the firm; qualifications of the personnel assigned to the project;
past experience of the firm and its employees assigned to the project;
geographic location; and price. The RFP characterized the first three
criteria as "very important" and the remaining two as "important."
The RFP also listed the type of information that should be provided
under each criterion. The RFP at M4 stated that award would be made
to "that offeror (1) whose proposal is technically acceptable and (2)
whose technical/cost relationship is the most advantageous to the
Government." The RFP further confirmed that it would perform a
cost/technical trade-off to determine the most advantageous offer.
The Forest Service received proposals from [deleted] firms, including
JW and NRMC. An evaluation board, made up of three Forest Service
specialists, evaluated the technical proposals. Each member assigned
a technical score to each proposal, which were then averaged;
proposals were ranked on the basis of the average scores. Two
proposals were determined to be technically unacceptable. Of the
remaining [deleted] proposals, NRMC's received the [deleted] averaged
technical score with [deleted]. JW's proposal received the [deleted]
highest score with an average of [deleted] points.
Business proposals were evaluated based on the total price offered for
the base and option items, as provided in the RFP. NRMC's price was
lowest among the proposals that were rated technically acceptable, and
JW's was sixth low.
The board prepared a "Memorandum for Record" that consisted of a
one-paragraph narrative summary of each proposal's evaluation. The
contracting officer concluded that NRMC's proposal was the one that
was most advantageous to the government, and awarded the contract to
this firm on October 7. When JW received notice of the award on
October 9, it requested a debriefing. JW received a debriefing letter
and a copy of the awardee's technical proposal on October 17.[1] This
protest followed on October 22. Because our Office notified the
Forest Service of the protest within 5 days of the
statutorily-required debriefing, the agency was required, under 31
U.S.C. sec. 3553 (1994), to stay the performance of the contract pending
resolution of the protest. Although the agency initially did stay
performance, on November 6, the head of the contracting activity
approved a request by the forest supervisor to override the stay and
proceed with performance.[2]
JW argues that the Forest Service's evaluation of proposals and its
resulting source selection decision were improper and inconsistent
with the evaluation scheme that was established in the RFP. Comparing
its own proposal to the awardee's, JW concludes that it should have
received a significantly higher score under those criteria, if
properly applied. For example, under the first (and therefore most
heavily weighted) criterion, "qualifications of the firm," JW points
out that it has recently completed two timber sale environmental
impact statements and is currently performing a third one for the
Forest Service in roadless areas in National Forests in Idaho. JW
notes a number of factors that are very similar to the requirement at
issue here, including the location in roadless areas, extensive areas
of older lodgepole pine, the presence of risk/damage from mountain
pine beetles, extensive intermixed land ownership, and the importance
of water quality and biodiversity to the site. In addition, JW points
out its direct experience with ecosystem management under those
contracts, and that it offers extensive experience with the National
Environmental Policy Act of 1969 (NEPA), 42 U.S.C. sec. 4321 et seq.
(1994), the legislation that prescribes environmental impact studies.
JW asserts that its current experience with the Forest Service
includes project management, technical management, public involvement,
technical writing, and hydrology expertise, all skills essential to
performing this contract. On the other hand, JW points out that the
awardee's proposal reflects no experience in completing an
environmental impact statement. The protester notes that NRMC does
not mention in its proposal experience with NEPA, project management,
technical management, public involvement, technical writing or
ecosystem management; it lists only forestry experience. JW concludes
that its own score for this criterion should have been higher than
NRMC's. JW provides a similar comparison of its proposal with the
awardee's on the remaining technical factors which supports its view
that its technical proposal should have been evaluated significantly
superior to the awardee's.
In reviewing protests against allegedly improper evaluations, it is
not our role to reevaluate proposals. Rather, our Office examines the
record to determine whether the agency's judgment was reasonable and
in accord with the RFP's stated evaluation criteria. Abt Assocs.,
Inc., B-237060.2, Feb. 26, 1990, 90-1 CPD para. 223. In order for us to
review an agency's selection determination, an agency must have
adequate documentation to support its selection decision. Arco
Management of Washington, D.C., Inc., B-248653, Sept. 11, 1992, 92-2
CPD para. 173. While both adjectival ratings and point scores are useful
as guides to decision-making, they generally are not controlling, but
rather, must be supported by documentation of the relative differences
between proposals, their weaknesses and risks, and the basis and
reasons for the selection decision. Federal Acquisition Regulation
(FAR) sec. 15.608(a)(3) and 15.612(d)(2); Engineering and Computation,
Inc., B-261658,
Oct. 16, 1995, 95-2 CPD para. 176.
After reviewing all of the supporting documentation submitted by the
Forest Service, we conclude that the technical evaluation is not
adequately supported. We also conclude that without adequate support
for the technical evaluation, a proper award determination could not
be made. Engineering and Computation, Inc., supra; see Redstone
Technical Servs.; Dynamic Science, Inc., B-259222 et al., Mar. 17,
1995, 95-1 CPD para. 181.
The evaluation record consists of a chart which lists the offerors,
the three evaluators' total scores for each offeror, the average score
of the evaluators, and each firm's price. This chart ranks firms
according to their price. The record also contains a memorandum for
the record. The memorandum consists of a one-paragraph evaluation
summary of each offeror. For NRMC, the memorandum states:
"Natural Resources Management Corporation presented a proposal
which offered to do the most for the money, or in other words,
providing the most acceptable proposal whose technical/cost
relationship was the most advantageous to the Government. The
firm was considered to be highly qualified having lots of
experience in the land management field and forestry. The firm
proposed establishing a field office in Wyoming for the duration
of this project which would be of benefit to the Forest. The
only concern of the Board's was that the firm did not have any
roadless experience. But this lack of experience was not enough
to eliminate the firm from being placed in the competitive
range."
Regarding JW's proposal, the memorandum states:
"JW Associates, Inc., was ranked [deleted] making the most
advantageous offer to the Government. The proposed project
schedule was considered to be good by the Board. The firm
proposed using primarily two employees and using a lot of
subcontractors. It appeared that the subcontractors had good
experience working together. The primary reason for this firm's
ranking was due to its cost."
Further, in response to the protest, the Forest Service prepared a
document entitled "Agency Reply to Protest," in which it generally
states that the evaluation was fairly performed in accordance with the
solicitation and responds to each of the protester's allegations with
conclusory statements. For example, for the protest basis at issue,
that technical proposals were misevaluated, the agency reply repeats
the evaluation criteria listed in the RFP and states that:
"Each Board member evaluated each technical proposal
independently and rated each proposal based upon what was
submitted. Once all of the technical proposals were
independently evaluated, the Board members then discussed amongst
themselves the technical proposals. As a result of this
discussion, technical proposals were determined to be either
acceptable or not acceptable to the Government.
The Contracting Officer directed the Board members to evaluate
each proposal using only the evaluation criteria listed in the
solicitation. An evaluation form was used by the Board to assist
with the evaluations. The form listed each of the evaluation
criteria and their relative value. No other criteria were used
during the evaluation. A Memorandum for Record [from which we
quoted in pertinent part, above] was prepared providing reasons
for the rankings."
The report does not include any individual technical evaluation sheets
or other contemporaneous documentation of the evaluation of the
various evaluation criteria, but provided only the chart with the
total score that each evaluator assigned to each proposal. In
contrast to the very specific and detailed allegations that the
protester has raised, pointing out specific areas in its proposal that
allegedly should have been evaluated more highly, such as under
"qualifications of the firms," discussed above, the Forest Service
has provided no specific rebuttal, nor has it provided any support or
explanation for its conclusions. Other examples of unrebutted,
specific allegations include the protester's assertion that it should
have been evaluated higher than the awardee because its own proposed
personnel are more highly qualified than the awardee's, an allegation
that it supports by comparing the education, background and experience
of its own personnel and the awardee's proposed personnel; its
assertion that the level of effort that NRMC proposes is inadequate to
complete the work, pointing out, for example, that the awardee
proposes to spend a total of two-thirds fewer hours on project
management compared to JW's proposed hours, and the assertion that
NRMC's proposal fails to include a detailed work plan or to disclose
the firm's technical approach in sufficient detail, as required by the
RFP. None of these issues is addressed in the one-paragraph summary,
or otherwise explained in the record. In fact, there is nothing in
the record furnished by the Forest Service which shows how each
proposal was evaluated under each technical evaluation factor. Thus,
we have no basis on this record to assess the reasonableness of either
JW's or NRMC's evaluation.
In addition to the Forest Service's failure to support its technical
evaluation in any meaningful way, the record suggests that the source
selection decision was based primarily on price--the least important
factor listed in the evaluation scheme. With the exception of the
paragraph in the Memorandum concerning NRMC's proposal, quoted above,
every paragraph in the Memorandum summarizing the agency's basis[3]
for its ranking of the individual proposals concludes that "[t]he
primary reason for this firm's ranking was due to its cost." Indeed,
as stated above, the ranking of the proposals [deleted]. The
contracting officer's debriefing letter confirms that price was the
basis for the agency's award decision.
While price, where lower in importance, can become the determining
factor between proposals that have been evaluated as technically equal
in a "best value" award decision, Ogilvy, Adams & Rinehart,
B-246172.2, Apr. 1, 1992, 92-1 CPD para. 332, the record in this case does
not contain sufficient information and analysis to establish the
equality of these two proposals; moreover, there is no evidence in the
record that any such conclusion was reached. To the contrary, as
discussed above, the protester has provided an analysis of its
proposal and the awardee's which reasonably shows that its proposal is
superior to the awardee's in the three most heavily weighted technical
areas--qualifications of the firm, personnel, and past experience.
Further, the protester points to the RFP evaluation language which
ranked technical merit as significantly more important than price. JW
states that its higher price was based on its proposal of high quality
personnel and a level of effort commensurate with the agency's needs
as announced in the RFP.
FAR sec. 15.612(d)(2) requires that the documentation supporting
selection decisions show the relative differences among proposals;
their strengths, weaknesses and risks; and the basis and reasons for
the decisions. This required explanation provides protesters and this
Office a basis upon which to judge the reasonableness of the agency's
decision and, ultimately, its compliance with the procurement statutes
and regulations. TFA, Inc., B-243875, Sept. 11, 1991, 91-2 CPD para. 239.
Where, as here, the agency has not adequately documented its
evaluation process and has failed to rebut the protester's very
specific arguments that its own proposal should have received a higher
technical score than the awardee's, we have no basis to conclude that
the selection decision was reasonable.
The protest is sustained.
By letter of today to the Secretary of Agriculture, we are
recommending that the agency, in accordance with the FAR, reevaluate
the proposals, document its evaluation, and make a new selection
decision.[4] If after reevaluation, the Forest Service believes
discussions with offerors are warranted, it may open discussions and
request best and final offers. If the agency selects an offeror other
than NRMC for award, it should terminate the contract to NRMC and make
award to that other offeror. We also recommend that the protester be
reimbursed its costs of filing and pursuing its protest. Bid Protest
Regulations, sec. 21.8(d)(1), 61 Fed. Reg. 39039, 39046 (1996) (to be
codified at 4 C.F.R. sec. 21.8). The protester should submit its
certified claim for costs to the contracting agency within 60 days of
receiving this decision. Bid Protest Regulations, sec. 21.8(f)(1),
supra.
The protest is sustained.
Comptroller General
of the United States
1. For reasons that are not clear, the Forest Service provided JW with
a copy of the awardee's technical proposal along with its debriefing
letter.
2. The statute permits an override of the stay on the basis of either
urgency or best interests of the government. The override
determination does not identify the basis for the override.
3. The RFP creates confusion concerning the evaluation scheme. While
M3 and M4(2) envision a best value approach, M4(1) introduces the
concept of technical acceptablility which is a scheme that results in
award to the low cost offeror. It appears from the record that the
Forest Service may have inappropriately followed the latter approach.
4. Although JW raises a number of challenges to the evaluation and
source selection decisions in its protest, we have not discussed each
protest basis. However, we recommend that the Forest Service, during
its reevaluation of proposals, take into consideration JW's
allegations that NRMC's proposal does not meet the requirements of the
RFP because it fails to provide certain information.