BNUMBER: B-270774.3
DATE: June 17, 1996
TITLE: Research for Better Schools, Inc.
**********************************************************************
DOCUMENT FOR PUBLIC RELEASE
A protected decision was issued on the date below and was subject to a
GAO Protective Order. This version has been redacted or approved by
the parties involved for public release.
Matter of:Research for Better Schools, Inc.
File: B-270774.3
Date:June 17, 1996
Joel R. Feidelman, Esq., Catherine E. Pollack, Esq., and Nancy R.
Wagner, Esq., Fried, Frank, Harris, Shriver & Jacobson, for the
protester.
Marcia G. Madsen, Esq., John A. Burkholder, Esq., Jessica C. Abrahams,
Esq., Morgan, Lewis & Bockius, for Temple University, an intervenor.
Jeffrey Morhardt, Esq., Department of Education, for the agency.
Scott H. Riback, Esq., and John M. Melody, Esq., Office of the General
Counsel, GAO, participated in the preparation of the decision.
DIGEST
1. Protest that awardee's proposal does not comply with applicable
statute relating to the composition and authority of proposed
governing board is denied where record shows awardee proposed board
meeting all statutory requirements.
2. Allegations relating to awardee's compliance with material
informational requirements in solicitation are denied where record
shows that, in fact, awardee provided all necessary information.
3. Protest that agency source selection official relied on documents
that failed to accurately embody the opinions of the agency's
technical evaluators is denied where record shows that documents
presented to source selection official contained the consensus
conclusions of the technical evaluators.
4. Agency properly did not advise protester during discussions that
aspects of its proposal were not viewed as favorably as same aspects
of awardee's proposal; such relative weaknesses are not matters for
discussion, as they do not relate to deficiencies found to protester's
proposal, but rather to fundamental differences in the offerors'
proposed solutions.
DECISION
Research for Better Schools, Inc. (RBS) protests the award of a
contract to Temple University under request for proposals (RFP) No.
95-040, issued by the Department of Education (DOE) for the operation
of a mid-Atlantic regional educational laboratory. RBS contends that
the agency improperly evaluated proposals, failed to engage in
adequate discussions and made an irrational award decision.
We deny the protest.
The RFP contemplated the award of a 5-year cost reimbursement contract
that requires the contractor to perform research and implement
broad-based, systemic school improvement strategies.[1] Award was to
be made to the firm submitting the proposal deemed to offer the best
overall value to the government, considering price and numerous
technical factors. The technical factors (and relative weights based
on a possible 125 point evaluation scale) were as follows: quality of
technical approach (35 points); quality of management plan (25
points); quality of personnel (20 points); corporate performance and
capability (20 points); and quality of the offeror's proposed
specialty area (25 points). (Within each evaluation area, the points
were further divided among various subfactors.)
DOE received three initial proposals including RBS' and Temple's (the
third was excluded from the competitive range and is not relevant
here). The agency conducted discussions, solicited technical proposal
revisions, obtained best and final offers (BAFO) and then evaluated
the BAFOs. RBS' BAFO received an average technical score of 94 points
and Temple's a score of 93.5 points; RBS' evaluated price was
$21,197,005 and Temple's $21,000,000. On the basis of these
evaluation results, the agency made award to Temple, finding that its
proposal represented the best overall value to the government. In
this regard, the agency found that, despite its slightly lower average
technical point score, Temple's proposal, in fact, was technically
superior to RBS'. The agency also concluded that Temple's proposal
offered the better value from a cost standpoint because its evaluated
cost was lower than RBS' and because it offered more professional
hours over the life of the contract.
COMPLIANCE WITH 20 U.S.C. sec. 6041
RBS maintains that Temple's regional laboratory will not comply with
the terms of 20 U.S.C. sec. 6041(h)(6) (1994), which requires each
laboratory to establish a governing board comprised of representatives
from various constituent organizations and groups (such as teachers,
state educational representatives and commissioners, and educational
researchers). The statute further requires that the governing board
be "the sole entity" responsible for guiding and directing the
regional laboratory, determining the regional agenda, and performing
various oversight functions. RBS maintains that Temple's proposed
laboratory will not be compliant with the statute because the
laboratory is to be located within the organizational structure of the
university, and will utilize university facilities and personnel. RBS
concludes that the laboratory therefore ultimately will be subject to
the authority of the Temple University Board of Trustees.
This contention is without merit. Temple's proposal takes no
exception to the governing board requirement. On the contrary, the
firm's proposal describes in detail its plan for the establishment of
the board and the selection of its members, as well as the governance
and operations of the board on an ongoing basis. Temple's proposal
also specifically states that:
"The Governing Board of Directors . . . will have sole overall
responsibility for establishing the programmatic goals of the
Laboratory, and for directing and overseeing the work of the
[laboratory] in fulfilling the terms and conditions of the award.
. . . The Board will also oversee and evaluate the performance of
the Laboratory's Executive Director, Executive directorate, and
key laboratory staff on an annual basis."
The proposal goes on to describe the responsibilities of the board,
including evaluation of the laboratory's overall programmatic
direction and approval, oversight, and direction of the laboratory's
specific programs and projects.
The essence of RBS' argument is that Temple's laboratory cannot be
governed "solely" by the board because the resources of the laboratory
are located within or are owned in part by Temple University.
However, there is nothing in the statute which precludes an entity
that is located within, or is a part of, a university from meeting the
statutory requirement of having an independent board, see The Regional
Laboratory for Educational Improvement of the Northeast and Islands,
Inc. B-270774 et al., Apr. 22, 1996, 96-1 CPD para. 204, and the record
does not show that there is any legal impediment to an agreement
between the laboratory and the university that vests the necessary
authority in the proposed governing board. Since Temple's proposal
does not take exception to the requirements of the statute, and in
fact goes to some length to explain the establishment and functioning
of its outside board, we have no basis to find its proposed laboratory
noncompliant with the statute.
FAILURE TO MEET MATERIAL RFP REQUIREMENTS
RBS contends that Temple's proposal failed to meet numerous material
informational requirements.[2] We have reviewed all of RBS'
allegations in this regard and find them to be without merit. We
discuss two of RBS' arguments below.
50-Percent Time Commitment
RBS maintains that Temple failed to include information in its
proposal showing that all of its key employees were committed to work
on the contract for at least 50 percent of their time, as required by
the RFP. In support of its contention, RBS notes that Temple's
proposal lists some 40 professionals as "key" employees, but fails to
include information relating to the time commitments of some 26 of
these individuals. RBS contends that, without this information,
Temple's proposal is technically unacceptable, and thus ineligible for
award.
RBS' allegation is based on an unreasonable reading of Temple's
proposal. The RFP required firms to propose key staff who had
committed at least 50 percent of their time to the contract, but went
on to describe key staff as ". . .the executive director and all those
who will direct major laboratory programs or functions, e.g. director
of planning, etc." Temple's proposal contains a manloading chart that
is captioned "time commitments of key personnel," but this chart lists
not only those individuals who will be responsible for directing major
laboratory programs or functions, but also all of Temple's other
professional and support personnel, including, for example, clerical
staff that are not specifically identified. The chart does contain a
separate section entitled "laboratory management unit" and under this
subheading are listed the individuals who will be responsible for
directing the laboratory's major programs and functions. The
laboratory management unit includes four individuals who are clearly
identified as the personnel responsible for managing the laboratory's
operations--the executive director, two associate directors, and one
assistant director responsible for administration and management
services functions. The chart goes on to specify the time commitments
of these individuals;[3] each employee is committed to work on the
contract in excess of 50 percent of his or her time. While Temple's
chart does suggest that all individuals listed are key employees, it
is clear from the solicitation's definition of key employee that only
those individuals involved in the direction or management of the
laboratory are to be considered key. Since Temple's offer clearly
shows that its managing employees are in fact committed to work at
least 50 percent of their time in performing the contract, Temple's
offer meets the 50-percent requirement.
Cap on Subcontractors' Overhead Rates
RBS contends that the agency unreasonably concluded that Temple's
subcontractors' overhead rates were capped at [deleted] on the basis
that Temple proposed to cap its own rate at [deleted], since the
proposal did not include separate documentation showing that the
subcontractors had agreed to the cap. According to RBS, the agency
should have performed a cost realism evaluation and adjusted Temple's
evaluated cost upward to account for the difference between Temple's
subcontractors' ordinary overhead rates and the [deleted] rate that
the agency used for establishing Temple's evaluated cost.
This argument is without merit. The agency, in requesting Temple's
BAFO, specifically asked it to clarify whether its subcontractors'
overhead rates would be capped at [deleted] for the contract's period
of performance. In response Temple stated "[y]es, the overhead rate
for all subcontractors will be CAPPED at [deleted] throughout the five
years of the contract. All subcontractors have agreed to the
[deleted] cap." Based on this language, the agency reasonably
concluded that both Temple's and its subcontractors' overhead rates
would be capped at [deleted].
TECHNICAL EVALUATION PROCESS AND RESULTS
RBS argues that the source selection was based on misleading
information--various memoranda prepared by an individual designated as
the agency's technical evaluation facilitator--that allegedly does not
accurately embody the evaluation results found in the individual
evaluation worksheets.[4] More specifically, RBS disagrees with the
emphasis given to the respective strengths and weaknesses of the two
proposals, as initially identified by the individual evaluators. For
example, RBS maintains that the facilitator's memoranda fail to
adequately convey the deficiencies found with Temple's proposed
management strategy. (Some of the evaluators downgraded Temple's
proposal for what they perceived as a weak management strategy
associated with its technical approach.)[5]
There is nothing in the record to suggest that the facilitator's
memoranda inaccurately report the consensus views of the evaluators.
The focus of those memoranda is the debate among the evaluators
regarding the comparative benefits of the proposals--two evaluators
originally preferred the RBS offer while two preferred Temple's--and,
ultimately, the basis for the entire group's view that the Temple
proposal offered the greatest technical merit. While recognizing that
Temple's proposal contained some weaknesses (and RBS' proposal some
strengths), the evaluators concluded that Temple's basic technical
approach of forming a coalition of educational institutions and
individual researchers was preferable to RBS' essentially
"self-contained" laboratory approach. The shift in emphasis between
the views expressed by the individual evaluators and those found in
the facilitator's memoranda reflects no more than the process of
forming a consensus opinion from competing views; as the process of
achieving consensus necessarily involves compromise, there is nothing
inherently improper in there being less than complete correlation
between each individual evaluation worksheet and the consensus
materials. See MVM, Inc.; Burns Int'l Sec. Servs., 73 Comp. Gen. 124
(1994), 94-1 CPD para. 279; DUAL, Inc., B-252593.3, Aug. 31, 1993, 93-2
CPD para. 190. Since there is nothing in the record to suggest that the
facilitator's memoranda inaccurately embody the consensus reached by
the evaluators, we find no basis to object to the source selection
official's basing the award decision on those memoranda.
RBS also maintains that the facilitator's memoranda improperly
recommended award to Temple based on factors outside the RFP or,
alternatively, accorded too much weight to one of the subfactors. The
evaluators ultimately preferred Temple's proposal because it reflected
"vision" in its method and strategy for conducting research and
development activities. The evaluators commented favorably on
Temple's proposal to [deleted]. (In contrast, the evaluators found
that RBS' proposal lacked "vision" because, as the incumbent, RBS had
proposed to build on its existing programs and had offered essentially
a "self-contained" laboratory rather than one involving partnering
arrangements with other institutions throughout the region.) RBS
maintains that "vision" was not one of the stated evaluation criteria
and thus should not have been the basis for preferring Temple's
proposal. Alternatively, RBS asserts, to the extent "vision" is
encompassed by the quality of technical approach factor, the
facilitator improperly relied on Temple's advantage under the first of
the three stated subfactors, which was worth only 15 points, rather
than on RBS' advantage under the other two subfactors, which were
worth 20 points combined.
We have no basis to object to the agency's reliance on "vision" in
discriminating between the proposed approaches. While the protester
is correct that the RFP did not specify "vision" as a discrete
evaluation criterion, we think this consideration is reasonably
encompassed in the first subfactor under the quality of technical
approach criterion. That subfactor required the agency to consider:
"The extent to which the proposed work as a whole constitutes a
coherent, sustained program of development and applied research
in the field using a well-conceptualized and theoretically sound
framework."
While the evaluators apparently used the term "vision" for descriptive
purposes, it is clear from a reading of the evaluation materials that
they were referring to Temple's proposed program of applied research
and development as it would be implemented throughout the region. The
panel members agreed that Temple offered "approaches . . . for
research and development [that were] novel, challenging, and replete
with fruitful strategies grounded in basic research." The evaluators
also agreed that Temple offered to implement (or 'scale up') numerous
well documented and validated programs throughout the region and
"[deleted]." Agencies properly may consider unstated evaluation
elements that are reasonably related to or encompassed by the stated
criteria. ORI Servs. Corp., B-261225, July 28, 1995, 95-2 CPD para. 55.
In addition, the fact that the other two subfactors under the quality
of technical approach criterion together were weighted more heavily
than the first subfactor is immaterial. There is no requirement that
the key award discriminator between offers be the most heavily
weighted factors. Corvac, Inc., B-254222, Dec. 2, 1993, 93-2 CPD para.
294.
DISCUSSIONS
RBS maintains that the agency failed to conduct meaningful discussions
with it. In this respect, RBS focuses on the criticisms made by the
agency evaluators in connection with its proposal, including its lack
of "vision," its status quo approach which built upon its earlier
work, its failure to propose partnering arrangements with outside
institutions which made its approach relatively insular and
conservative, and its failure to optimally address the needs of the
region, especially in urban areas. RBS maintains that if the agency
had discussed these matters, it would have changed its proposed
approach and emphasis.
The noted concerns in no way reflected a finding of deficiencies in
the RBS proposals. Rather, the criticisms noted were identified
primarily in connection with the evaluators' comparative assessment of
the two proposals for purposes of choosing between the firms'
respective technical approaches. Thus, for example, the observation
that RBS' proposal lacked "vision" did not reflect a finding that RBS'
proposal was somehow deficient or unacceptable; it merely represented
a finding that Temple's proposal showed greater promise for contract
performance. Agencies are not required to--and in fact ordinarily
should not--discuss elements of a competitor's technical approach that
render it comparatively superior; such matters are not properly for
discussion. Bioqual, Inc., B-259732.2; B-259732.3, May 15, 1995, 95-1
CPD para. 243.
SOURCE SELECTION
RBS challenges the award decision on grounds that the agency
incorrectly found that Temple's proposal was lower in cost, that it
had offered approximately 62,000 more professional hours than RBS
offered, and that there was no basis for finding Temple's proposal
technically superior in light of RBS' slightly higher technical point
score.
In making source selections, agencies enjoy broad discretion to make
cost/technical tradeoffs, and such judgments are subject only to the
test of rationality and consistency with the solicitation's stated
evaluation and award criteria. Information Sys. Networks, Inc.,
B-254384.3, Jan. 21, 1994, 94-1 CPD para. 27. Evaluation point scores
are no more than guidelines for intelligent decision making; they do
not automatically mandate the selection of a particular proposal.
Porter/Novelli, B-258831, Feb. 21, 1995, 95-1 CPD para. 101. The
question, ultimately, is whether the record supports the agency's
conclusions regarding the relative merits of the proposals.
RBS' argument regarding the agency's conclusion that Temple's proposal
was lower in cost is based on RBS' position that Temple's
subcontractors did not agree to cap their overhead rates at [deleted].
As discussed above, however, we find that Temple's proposal did
effectively cap its subcontractors' overhead rates. There is thus no
basis for questioning the agency's reliance on Temple's lower
evaluated cost in its source selection.
With respect to the conclusion that Temple's proposal was technically
superior, the record shows that the evaluators assigned virtually
equal scores to the two proposals, with only one-half point separating
the two. The record further reflects that, during the conference
discussion between the evaluators, they reached consensus that
Temple's proposal was technically superior notwithstanding its
slightly lower point score. As discussed, the evaluators preferred
the Temple proposal primarily because of the potential for substantial
improvement in student achievement for the region; this conclusion was
based primarily on Temple's coalition approach, and its proposed
research and development initiatives that, in the evaluators' opinion,
represented a positive departure from the status quo nature of RBS'
proposal. The evaluators also concluded that, with close agency
monitoring, the central deficiency in Temple's proposal--its weak
management strategy--could be overcome; they therefore concluded that
the Temple proposal was superior to RBS'. Since numeric point scores
are merely guides to intelligent decision making, they do not
necessarily establish that a particular firm is technically superior
or mandate selection of a particular proposal for award. Contract
Servs., Inc., B-251761.4, July 20, 1993, 93-2 CPD para. 40. RBS has not
shown that the evaluators' central conclusions regarding the relative
strengths and weaknesses of the proposals are not supported by the
record. In addition, as discussed above, the evaluators' conclusions
were consistent with the solicitation's evaluation criteria Thus,
there is no basis for questioning the agency's conclusion that
Temple's offer was slightly superior from a technical standpoint.
Finally, we need not decide the validity of RBS' contention that, had
the agency properly calculated professional hours, it would have found
that Temple did not offer more hours. Even if RBS were correct, this
would eliminate only one of Temple's evaluation advantages. Given our
findings above--that Temple's offer was
properly found technically superior and lower in cost--there is no
basis to conclude that the agency's source selection decision would
have been different without the professional hours advantage.
The protest is denied.
Comptroller General
of the United States
1. This contract is one of 10 awarded to run regional laboratories
throughout the United States. DOE issued a single RFP for all 10
contracts, but essentially conducted separate acquisitions for each
region. This decision concerns only the agency's acquisition in the
mid-Atlantic region.
2. RBS contends that Temple's proposal failed to offer key personnel
meeting the RFP requirement that such employees commit at least 50
percent of their time to contract performance; did not identify the
other time commitments of its key personnel; did not include letters
of commitment from all of its key employees; did not include
information necessary for a proper past performance evaluation; and
did not include evidence of its subcontractors' legal commitment to
cap their overhead cost rates at [deleted].
3. Not all of the key employees' time is designated as committed to
laboratory management; their time is divided between management
functions and research and development activities.
4. The record shows that, consistent with its source selection plan,
the agency provided copies of the initial proposals to the members of
the technical evaluation team; the evaluators read the proposals and
prepared detailed individual evaluation worksheets for each of the
evaluation criteria. After reviewing the initial proposals, the
evaluators forwarded questions to the agency's technical evaluation
facilitator, who in turn prepared discussion questions for each
offeror. The technical evaluators then were provided with the
offerors' responses to the discussion questions; after reviewing these
materials, the evaluators prepared revised individual evaluation
sheets that were forwarded to the technical evaluation facilitator.
The facilitator then convened a telephone conference among the
evaluators for the purpose of their arriving at a consensus rating for
each proposal, and also to provide the evaluators an opportunity to
comparatively discuss the relative merits of the two offers. The
results of this conference call are embodied in the memoranda relied
on by the source selection official.
5. RBS takes issue with numerous specific conclusions in the
facilitator's memoranda. For the reasons discussed, we find that
these conclusions are not inconsistent with the views of the
evaluators overall.