TITLE: Kalman & Company, Inc., B-287442.2, March 21, 2002
BNUMBER: B-287442.2
DATE: March 21, 2002
**********************************************************************
Decision
Matter of: Kalman & Company, Inc.
File: B-287442.2
Date: March 21, 2002
Daniel R. Weckstein, Esq., Walter T. Camp, Esq., and Michael L. Sterling,
Esq., Vandeventer Black, for the protester.
Bernard J. Roan, Esq., and David A. Samuels, Esq., National Aeronautics and
Space Administration, for the agency.
Tania Calhoun, Esq., and Christine S. Melody, Esq., Office of the General
Counsel, GAO, participated in the preparation of the decision.
DIGEST
Protest that contracting agency's evaluation of technical, past performance,
and cost proposals was unreasonable is denied where the record shows that
the evaluation was reasonable and consistent with the solicitation's stated
evaluation criteria.
DECISION
Kalman & Company, Inc. protests the award of a contract to Analytical
Services & Materials, Inc. (AS&M) under request for proposals (RFP) No.
RFP4-00017, issued by the National Aeronautics and Space Administration
(NASA) to obtain engineering and technical services (ETS) in support of
flight research and development programs for NASA's Dryden Flight Research
Center, located at Edwards Air Force Base, California. Kalman argues that
the agency improperly evaluated technical, past performance, and cost
proposals.
We deny the protest.
The ETS contract is the primary vehicle by which the Dryden research,
projects, management operations, chief engineer's office, and public
affairs/education/
commercialization organizations obtain support for flight research projects,
internal research and development, and various technical support functions.
RFP Statement of Work (SOW) at 3. As a result, the SOW divides the technical
requirements into four sections: research engineering support; management
system support; chief engineer's office support; and public affairs,
commercialization, education outreach (PACE) and history office support. Id.
This acquisition combines the current ETS contract with two other Dryden
contracts. The solicitation, issued as a small business set-aside,
contemplated award of an indefinite-delivery/indefinite-quantity contract
with cost-plus-award-fee and cost-reimbursable-no-fee contract line items.
Work was to be performed pursuant to specific task orders over a base period
of 2 years, with one 3-year option period.
Award was to be made to the firm whose proposal was most advantageous to the
government, considering mission suitability, past performance, and cost
factors. The combined importance of the mission suitability and past
performance factors was significantly more important than the cost factor.
As individual factors, the mission suitability and past performance factors
were approximately equal in importance, and both were individually
significantly more important than cost. RFP sect. M.2.
The mission suitability factor, which was to be point-scored and
adjectivally rated, was comprised of two subfactors. The first subfactor,
"understanding the requirements" (UR), was worth 600 of the 1,000 points
available for the mission suitability factor and was comprised of two
sub-subfactors, "overall understanding and technical approach" and
"representative task orders" (RTO). The second subfactor, "management and
staffing" (MS), was worth 400 of the 1,000 points available and was
comprised of six sub-subfactors. RFP sect. M.3.
The past performance evaluation was to assess the extent of the offeror's
relevant experience associated with each area of the SOW, as evidenced
within the contract(s) referenced in the proposal or otherwise known to the
government. In addition to providing for the specific consideration of
technical performance, schedule performance, and cost performance, NASA
planned to evaluate the complexity and magnitude of the offeror's previous
and current work and its relation to the work required for this procurement,
with the quality and quantity of specific technical and management
experience directly applicable to this requirement being especially
important. Current and relevant experience was to have greater impact than
less recent and less relevant experience. The term "relevant" was defined as
meaning "contracts having comparable size (in excess of $4 [million]
annually) or scope, and significant technical or cost similarity,
association or other relationship with the effort and/or contract type
described in this RFP." RFP sect. M.4(c). Citing section 15.305(a)(2) of the
Federal Acquisition Regulation (FAR), the RFP added that "[o]fferors without
a record of relevant past performance or for whom information on past
performance is not available, will not be evaluated favorably or unfavorably
on past performance, but will receive a 'Neutral' rating." RFP sect. M.4.
The agency planned to conduct a cost realism analysis of each cost proposal
to determine the probable cost of performance for each offeror. This
analysis would determine whether the estimated proposed cost elements were
realistic for the work to be performed; reflected a clear understanding of
the requirements; and were consistent with the unique methods of performance
and materials described in the offeror's technical proposal. The probable
cost would be used for purposes of evaluation. RFP sect. M.5.
NASA received proposals from six offerors. The source evaluation committee
(SEC) presented its initial evaluation results to the source selection
authority (SSA), who decided that discussions were necessary. A list of
discussion items and a request for a final proposal revision (FPR) were sent
to each of the four offerors in the competitive range, including Kalman and
AS&M, the incumbent ETS contractor. The SEC evaluated the FPRs and arrived
at the following final evaluation results:
Mission Suitability Past Cost
Performance (in Millions)
AS&M 850 Very Good Excellent $43.12
Proposed
- UR 520/Very Good/Low Low Risk
Risk $43.26
Probable
- MS 330/Very Good/Low
Risk
Offeror 745 Good-Very Good Good $41.95
A Proposed
- UR 420/Good/Moderate Moderate
Risk Risk $42.30
Probable
- MS 325/Good-Very
Good/Low Risk
Offeror 605 Good Marginal $48.14
B Proposed
- UR 390/Good/Low Risk Moderate
Risk $46.86
- MS 215/Good/Moderate Probable
Risk
Kalman 590 Good Neutral $46.17
Proposed
- UR 405/Good/Moderate High Risk
Risk $48.23
Probable
- MS 185/Fair/Moderate
Risk
The SSA determined that AS&M's proposal represented the best value to the
government. AS&M received the highest score under the mission suitability
factor due, in part, to having the most significant strengths, including a
demonstrated understanding of the SOW requirements and of the contractor's
role in performing the work. The SSA concluded that when these strengths
were combined with AS&M's other major strengths, the higher rating was
justified. While other offerors received significant strengths in specific
areas, none demonstrated as deep or comprehensive an understanding of all
functional areas as AS&M. Under the past performance factor, the SSA stated
that AS&M's excellent rating reflected mostly excellent responses in its
past performance questionnaires, and its past performance also reflected
relevant experience on the SOW requirements. Finally, with respect to cost,
the SSA noted that Kalman and Offeror B had significantly higher proposed
and probable costs than AS&M and Offeror A. While AS&M had a higher probable
cost than Offeror A, the SSA stated that the difference was outweighed by
the significant strengths in AS&M's proposal and its excellent past
performance. On December 17, the unsuccessful offerors in the competitive
range were notified that NASA planned to make award to AS&M. Kalman filed
this protest after its debriefing, raising various challenges to the
reasonableness of the agency's evaluation of its mission suitability and
past performance proposals, as well as to the reasonableness of the agency's
evaluation of both AS&M's and its own cost proposals. Award has been
suspended pending the outcome of the protest.
Kalman's allegation that NASA unreasonably evaluated its proposal under the
mission suitability factor is focused on the two significant weaknesses NASA
found with respect to the "understanding the requirement" subfactor, and two
of the eight weaknesses NASA found with respect to the "management and
staffing" subfactor.
The evaluation of technical proposals is primarily the responsibility of the
contracting agency, and our Office does not make an independent evaluation
of their merits. Rather, we examine the agency's evaluation to ensure that
it is reasonable and consistent with the stated evaluation criteria. The
protester bears the burden of showing that the evaluation is unreasonable,
and the fact that it disagrees with the agency does not render the
evaluation unreasonable. ESCO, Inc., B-225565, Apr. 29, 1987, 87-1 CPD para. 450
at 7. Kalman has not met that burden.
In reviewing proposals under the "understanding the requirement" subfactor's
"overall understanding and technical approach" sub-subfactor, NASA was to
evaluate the offeror's understanding of the work to be performed as related
to the functions in the SOW; the approach described by the offeror to
performing the work; the methodology used to identify, report, and resolve
problems encountered in the performance of the requirements; how the offeror
identified and mitigated the risks to fulfilling the requirements; and how
the narrative reflected the offeror's understanding of the role of the
contractor in the Dryden environment.
RFP sect. M.3(a)(1).
Kalman's proposal was evaluated as having a significant weakness because
NASA believed it demonstrated a lack of understanding of the complexity and
organizational structure of the PACE office and associated tasks. The caucus
summary report on Kalman's initial proposal shows that the SEC had a number
of detailed reasons to support this finding: (1) the proposal referred to
"PACE and History Office Support" but the History Office is in the PACE
organization and not a separate entity; (2) the response to SOW para. 4.2, "PACE
Office," simply stated the requirements and identified problems that were
not representative of typical issues under this task; (3) the description of
the firm's understanding under SOW para. 4.2 did not provide a comprehensive
response to the task; (4) the response to SOW para. 4.3, "Public Affairs,"
included little response to the public affairs tasks but, instead, a
reiteration of current policies and practices which did not describe how the
firm would perform the task; (5) the response to SOW para. 4.4, "Communicate
Knowledge," contained no information about how the firm would interface with
the PACE organization and did not recognize established priorities within
this task element;
(6) the response to SOW para. 4.5, "Education," did not state how the firm would
perform the tasks but basically restated the SOW requirements; and (7) the
response to SOW para. 4.7, "History," did not state how the firm was going to
perform the tasks, listed problems that were not within the scope of the
task, and did not mention the level of expertise and skill required for the
task.
Kalman's FPR included a response to the SEC's request to revisit its
discussion of this section of the SOW. The SEC evaluated this response and
concluded that it was inadequate because the only significant information
presented was the listing of the Kalman team that would support the effort.
The SEC also remarked that there was still a lack of understanding of the
PACE Office since the revised response contained references to the "PACE and
History Offices," in the plural, indicating Kalman was not aware that the
History Office was part of the PACE organization and not a separate entity.
The SEC's final caucus report did not delete any of the specific bases for
evaluating this aspect of the proposal as having a significant weakness.
Kalman challenges only one of the specific bases underpinning the SEC's
finding, the comment about the firm's references to the PACE and History
Offices, in the plural. Kalman argues that NASA has extrapolated that the
use of a particular title demonstrates a lack of understanding and that NASA
ignored all of the specifics in its proposal. Kalman's argument is plainly
contradicted by the detailed findings made by the SEC which show that NASA
did note the specifics of Kalman's proposal but found them lacking. We also
agree with NASA that the references to the PACE and History Offices, in the
plural, give rise to the reasonable concern whether Kalman was aware that
they were one organization. Kalman further contends that NASA's finding is
unreasonable because the parts of the proposal pertaining to the PACE office
were prepared by a former Dryden employee with intimate knowledge of the
office, and because Kalman had contingent hire letters from numerous current
key staff members and prior employees. This argument does not address the
specific problems found by NASA during its evaluation of Kalman's proposal;
moreover, this subfactor was not designed to evaluate the abilities of the
firm's proposed key personnel [1] but, rather, whether the firm could
demonstrate a full understanding of the nature of the work to be performed.
Kalman has given us no reason to question NASA's conclusion that the firm
demonstrated a lack of understanding of the PACE office and its associated
tasks or that this failure represented a significant weakness.
In reviewing proposals under the "understanding the requirement" subfactor's
"RTO" sub-subfactor, NASA was to evaluate the depth of discussion of how the
task would be performed, including technical approach and schedule; the
realism of the proposed staffing plan, including skill mix and level of
staffing needed to accomplish the objectives; the realism of additional
resources necessary to successfully complete the task; and the completeness
and accuracy in the identification of critical issues, including risk
identification and mitigation. RFP sect. M.3(a)(2). Kalman's proposal was
evaluated as having a significant weakness because NASA found it reflected a
significant lack of understanding of the requirements associated with the
Controlled Impact Demonstration Anniversary Symposium RTO. The caucus
summary report on Kalman's initial proposal shows NASA reached this
conclusion because Kalman's proposal deleted the requirement for paper
copies, stating that there would be no sensitive material; did not include a
risk assessment or mitigation; did not include resources and staffing
estimates; and because the major portion of Kalman's response was a detailed
schedule of events and week-of activities.
Kalman's FPR included a response to the SEC's request to revisit its
discussion of this RTO. The SEC evaluated the response and found it
inadequate because, among other things, it did not discuss attendees'
classification and processing; did not discuss how symposium papers would be
distributed if they contained sensitive information or information that
might not be readily available to the public; and because Kalman relied on
web/electronic distribution of symposium material and did not provide an
approach for supporting the print and distribution requirements. The SEC's
final caucus report did not delete any of the above specific bases for
evaluating this aspect of Kalman's proposal as having a significant
weakness.
Kalman argues that the SEC's criticism that the major portion of its
response was a detailed schedule of events and activities is in error since
the SEC was required to evaluate "[the] [d]epth of [the] discussion of how
the task would be performed, including technical approach and schedule." RFP
sect. M.3(a)(2). Kalman has misread the record. The SEC's concern was not that
Kalman included a detailed schedule of events and activities, but that this
information constituted the major portion of its response at the expense of
other information to be evaluated. Our review of the record gives us no
basis to question the SEC's criticism.
Kalman also objects that there was no requirement for paper copies and that
it should have been given extra credit for its plans to use electronic
publishing methods. Kalman is incorrect. One subtask to be performed in this
RTO was to publish and distribute all symposium papers in one document. A
Dryden publication governing how technical reports and papers are to be
published was posted as part of the solicitation on the agency's web site.
That publication states that the Dryden technical publications office will
print the document and references a publications manual detailing the
procedures. The final step in these procedures is to print the report. The
contracting officer states that Kalman's proposal was downgraded because it
did not even recognize the documented procedures in place for publication;
had it done so in addition to proposing electronic publication as an
alternative, it might have received additional credit. We cannot find the
agency's evaluation unreasonable.
Kalman next contests two of the eight weaknesses the SEC identified in its
proposal under the "management and staffing" subfactor.
The first weakness stems from the SEC's finding that the benefit packages
among Kalman's team members were not uniform and differed substantially. The
SEC was concerned that these disparities--such as widely varying numbers of
paid holidays and sick days--could affect employee morale. The SEC was also
concerned that the proposed salaries varied greatly between the team members
for the same position, which could potentially harm employee recruitment and
morale. During discussions, Kalman was asked to explain how it planned to
mitigate the differences that appeared in the various compensation plans
such that employee recruitment, retention and morale would be maintained. In
response, Kalman stated that, "Ideally, [it] would like each of its team
members to offer the same competitive benefit package" and that each team
member was revisiting its compensation plan and would be encouraged to share
information to ensure that benefits would be comparable. Kalman FPR at 28.
The SEC found this response to be generic, noted that the FPR had greater
salary discrepancies than did the initial proposal, and did not change its
view that this was a weakness. Kalman's contention that "it is not for the
government to argue for consistency if it leads to mediocrity," Protester's
Comments, Feb. 21, 2002, at 4, both fails to address the agency's concerns
about employee morale and is inconsistent with its FPR statement that it
wanted to encourage its team members to offer the same competitive benefit
package. [2]
The second weakness results from the SEC's concern that Kalman's proposal
failed to disclose the duration of time its instrumentation engineering key
person would be available to work on the effort. Kalman asserts that the
same problem existed in AS&M's proposal but that it, unlike Kalman, had an
opportunity to correct the problem during discussions. Kalman's assertion
that the agency treated the offerors disparately again reflects a misreading
of the record.
The SEC evaluated Kalman's initial proposal as having a weakness because it
failed to identify the key person in instrumentation engineering. During
discussions, Kalman was asked to identify this key person and to provide his
resume and letter of intent. Kalman's FPR did so, and the SEC deleted the
original weakness it had assigned, but added a new weakness because neither
the newly-identified person's resume nor his letter of intent made it clear
how long this person would be available to work on the effort. There was no
disparate treatment because both AS&M and Kalman were given the opportunity
to address their original weaknesses. The weakness at issue here was first
introduced by Kalman in its FPR, when it first submitted this person's
resume and letter of intent, and the agency was not required to reopen
discussions to permit the firm to address this weakness. The Communities
Group, B-283147, Oct. 12, 1999, 99-2 CPD para. 101 at 4-5.
Kalman has thus given us no basis to find that NASA unreasonably evaluated
its proposal under the mission suitability factor. Moreover, despite having
access to the entire evaluation record and AS&M's proposal, Kalman has
failed to make any specific argument in support of its protest allegation
that, on information and belief, NASA's evaluation of AS&M's proposal relied
not squarely on its proposal, but on its awareness of the firm's
capabilities based on its incumbency. As a result, this allegation lacks a
valid basis. Siebe Envtl. Controls, B-275999.2, Feb. 12, 1997, 97-1 CPD para. 70
at 2.
Kalman next alleges that NASA's evaluation of its past performance as
neutral--a rating required for offerors without a record of relevant past
performance--was unreasonable and inconsistent with NASA's finding that the
Kalman team as a whole had relevant experience for the majority of SOW
requirements. Our review of the record shows that the apparent inconsistency
merely reflects two different meanings of the term "relevant."
The solicitation provided for a specific evaluation of an offeror's
technical performance, schedule performance, and cost performance. The
solicitation also provided for a more general assessment of the extent of an
offeror's relevant experience associated with each of the SOW areas, as
evidenced within contracts referenced in the proposal, or otherwise known to
the government. As noted earlier, current and relevant experience was to
have greater impact in the government's assessment than less recent and less
relevant experience. The RFP defined the term "relevant" as meaning
"contracts having comparable size (in excess of $4 [million] annually) or
scope, and significant technical or cost similarity, association or
relationship with the effort and/or contract type described in this RFP."
RFP sect. M.4(c). Citing FAR sect. 15.305(a)(2), the RFP stated that offerors
without a record of relevant past performance or for whom information on
past performance was not available would not be evaluated favorably or
unfavorably on past performance, but would receive a neutral rating. RFP sect.
M.4(b).
Kalman's proposal did not identify any "relevant" contracts, i.e., any
"contracts having comparable size (in excess of $4 [million] annually) or
scope, and significant technical or cost similarity, association or other
relationship with the effort and/or contract type described in this RFP."
RFP sect. M.4(c). Since Kalman had no record of "relevant" past performance as
defined by the solicitation, NASA assigned its proposal a neutral rating in
accordance with the solicitation's terms. See also
FAR sect. 15.305(a)(2). NASA's finding that the Kalman team as a whole had
"relevant technical experience" refers to the fact that the team's technical
performance experience was relevant to the SOW tasks, not that the proposal
met the solicitation's definition of the term "relevant." Under the
circumstances, and given the express language of the solicitation, we cannot
find the evaluation unreasonable. [3] See Boland Well Sys., Inc., B-287030,
Mar. 7, 2001, 2001 CPD para. 51 at 3.
In its protest, Kalman also argued that NASA's evaluation was unreasonable
because NASA claimed not to have received, and evaluated, past performance
questionnaires on one of its subcontractors. Kalman asserted that the
questionnaires were, in fact, submitted by the respondents. In its report,
NASA denied that it had received these questionnaires but explained that
Kalman was not prejudiced thereby because the subcontractor represented such
a small percentage of the total proposed workforce here. Kalman's comments
on the agency report do not rebut NASA's claim that it did not receive the
questionnaires, and we have no basis to find otherwise. In any event, there
is no legal requirement that all past performance references be included in
a valid review of past performance. Advanced Data Concepts, Inc.,
B-277801.4, June 1, 1998, 98-1 CPD para. 145 at 10. For our Office to sustain a
protest challenging the failure to obtain a reference's assessment of past
performance, a protester must show unusual factual circumstances that
convert the failure to a significant inequity for the protester. MCS of
Tampa, Inc., B-288271.5, Feb. 8, 2002, 2002 CPD para. _ at 5. Kalman has made no
such showing and, given the basis for the neutral rating, we find none.
Kalman finally argues that NASA improperly failed to upwardly adjust AS&M's
proposed direct labor rates in conducting its cost realism analysis. Kalman
asserts that, in various cases, AS&M's proposed direct labor rates are below
the rates found in some market surveys and that NASA failed to note the
substantial performance risk presented by these allegedly unreasonably low
rates.
Where an agency evaluates proposals for award of a cost-reimbursement
contract, an offeror's proposed estimated costs of contract performance and
proposed fees are not considered controlling, since the offeror's estimated
costs may not provide valid indications of the final actual costs that the
government is required, within certain limits, to pay. Advanced
Communication Sys., Inc., B-283650 et al., Dec. 16, 1999, 99-2 CPD para. 3 at 5.
Accordingly, a cost realism analysis must be performed when a
cost-reimbursement contact is contemplated. FAR sect. 15.404-1(d)(2). A cost
realism analysis is the process of independently reviewing and evaluating
specific elements of each offeror's proposed cost estimate to determine
whether the estimated proposed cost elements are realistic for the work to
be performed, reflect a clear understanding of the requirements, and are
consistent with the unique methods of performance and materials described in
the offeror's technical proposal. FAR sect. 15.404-1(d)(1). Because the agency
is in the best position to make this cost realism determination, our review
is limited to determining whether its cost evaluation was reasonably based
and not arbitrary. NV Servs., B-284119.2, Feb. 25, 2000, 2000 CPD para. 64 at 7.
The RFP advised offerors that, in establishing the probable cost, the SEC
might make use of any available source, including data provided by the
offerors, other government agencies, commercial sources, in-house sources,
and any other sources,
as appropriate. RFP sect. M.5. Section M.3(b)(4) of the RFP stated that proposed
compensation would be reviewed for
a clear understanding of the work to be performed and an indication of the
capability of the proposed compensation structure to obtain and keep
suitably qualified personnel to meet mission objectives. . . . [P]roposals
envisioning compensation levels lower than those of predecessor contractors
for the same work will be evaluated on the basis of maintaining program
continuity, uninterrupted high quality of work, and availability of required
competent professional service employees.
In response to Kalman's complaint that several of the direct labor rates
proposed by AS&M are below the rates found in some market surveys, NASA
explains that the RFP did not emphasize survey data to the exclusion of all
other data, and that its evaluation was not so limited. Specifically, the
agency reviewed both the survey data provided in AS&M's proposal and
historical data in the form of the wage data for the current ETS and other
NASA contracts to evaluate the realism of AS&M's proposed direct labor
rates. AS&M is the incumbent ETS contractor, and based its proposed direct
labor rates on those it was paying under the current contract. Since AS&M
was currently and successfully performing the contract, NASA states that it
had no basis to find that the firm's wages were not realistic or consistent,
no basis to find a negative impact on recruiting and retention, and no basis
to adjust AS&M's proposed costs upward to accommodate higher wage rates than
those proposed. Notwithstanding its conclusion that the rates were
realistic, NASA did downgrade AS&M's proposal as Kalman suggests by
assigning it a weakness under the management and staffing subfactor because
certain of its proposed direct labor rates were at the low end of the scale.
Under the circumstances, we cannot conclude that NASA's evaluation was
unreasonable.
In any event, we need not give further consideration to Kalman's allegations
about AS&M's cost proposal or to its allegation that NASA unreasonably
adjusted its own cost proposal. Under the Competition in Contracting Act of
1984, 31 U.S.C. sect. 3553(a) (1994), and our Bid Protest Regulations, 4 C.F.R.
sect. 21.1(a) (2001), a protester must be an "interested party" before we will
consider its protest. A protester is not an interested party if it would not
be in line for award if its protest were sustained. Avondale Tech. Servs.,
Inc., B-243330, July 18, 1991, 91-2 CPD para. 72 at 2.
We have already found Kalman's allegations regarding the evaluation of its
own proposal under the non-cost factors to be without basis. In addition,
there is at least one offeror aside from AS&M who received higher non-cost
ratings and whose proposed and probable costs are lower than those of
Kalman; Kalman does not challenge the evaluation results with respect to any
offeror save AS&M. Thus, even if we found that AS&M's proposed costs should
have been upwardly adjusted to a substantial degree and found to represent
high performance risk, and even if we found that NASA should not have made
any upward adjustments at all to Kalman's
proposed costs, at least one offeror, not Kalman, would be next in line for
award. Accordingly, Kalman lacks the requisite direct and substantial
interest with regard to
the award to be considered an interested party to protest the evaluation.
RS Information Sys., Inc., B-287185.2, B-287185.3, May 16, 2001, 2001 CPD para.
98 at 5.
The protest is denied.
Anthony H. Gamboa
General Counsel
Notes
1. The record shows that Kalman was given credit for the strength of its key
personnel under the management and staffing subfactor.
2. Our review of the record shows that Kalman's contention that AS&M's team
also proposed significantly disparate benefit packages is without basis.
See, e.g., AS&M FPR at 5.
3. Kalman does not challenge the reasonableness of the underlying past
performance evaluation, which identified one significant weakness, three
weaknesses, and various reasons for the firm's high risk rating.