TITLE:  SDS International, Inc., B-291183; B-291183.2, December 2, 2002
BNUMBER:  B-291183; B-291183.2
DATE:  December 2, 2002
**********************************************************************
SDS International, Inc., B-291183; B-291183.2, December 2, 2002

   DOCUMENT FOR PUBLIC RELEASE                                                
The decision issued on the date below was subject to a GAO Protective      
Order.  This redacted version has been approved for public release.        

   Decision
    
Matter of:   SDS International, Inc.
    
File:            B-291183; B-291183.2
    
Date:              December 2, 2002
    
Barbara S. Kinosky, Esq., and James S. Phillips, Esq., The Centre Law
Group,
for the protester.
Michael A. Gordon, Esq., Holmes, Schwartz & Gordon, for CBD Training,
Inc.,
an intervenor.
John D. Inazu, Esq., Department of the Air Force, for the agency.
Charles W. Morrow, Esq., and James A. Spangenberg, Esq., Office of the
General Counsel, GAO, participated in the preparation of the decision.
DIGEST
    
Selection of higher-priced, higher-rated proposal, instead of
lower‑priced, lower‑rated one, was unreasonable where it was
based in material part on an unsupported reported strength in the
awardee's proposal.
DECISION
    
SDS International, Inc. protests the award of a contract to CBD Training,
Inc. under request for proposals (RFP) No. F44650-02-R-0009, issued by the
Department of the Air Force, Air Combat Command Contracting Squadron,
Langley Air Force Base Virginia, for aircrew training and courseware
development. 
    

   We sustain the protests.
    
The RFP, issued April 3, 2002 as a small business set-aside, was for
aircrew training and courseware development at the 57th Wing United States
Air Force Weapons School (USAFWS), Nellis Air Force Base, Nevada, under a
fixed-price contract for a 30-day phase-in period and a base year with 4
option years.   In addition to the training and courseware development
requirements, the statement of work (SOW) required the contractor to
maintain the USAFWS's academic database and make it accessible from the
Internet.  The SOW explained that *[t]he database shall provide an
[I]nternet[-]based Learning Management System (LMS)/Courseware Management
System [(CMS)] that identifies and tracks student deficiencies, provides
graduate critiques, and service as a 'one stop shopping' site for all
instructor and student academic/courseware needs.*  The SOW further stated
that *all unclassified material shall be accessible from the weapons
school unclassified LAN [local area network], while all classified and
unclassified material shall be located on a classified server that can be
accessed via a SIPRNET (Secret Internet Protocol Router Network) web page
with password controls,* and required the task be accomplished *with no
additional workload* on the USAFWS LAN and staff.  See RFP, SOW P:P:
1.7.7, 1.7.7.1, 1.7.7.2.
    
The RFP provided for award on a *best value* basis considering as
evaluation factors past performance, mission capability, risk, and
price.   Past performance and mission capability were of equal importance
in the evaluation scheme, followed by risk, with price being of least
importance.  RFP at 45.  Under the mission capability factor, the RFP
listed three subfactors in descending order of importance:  (1) training
and workload management, (2) courseware development (CWD) approach and
instructional systems development (ISD) management plan, and (3)
phase-in. 
RFP at 48.  
    
The RFP stated *[p]ast performance shall be evaluated as a measure of the
Government's confidence in the offeror's ability to successfully perform
based on previous and current contract efforts,* and *will be evaluated by
examining four (4) references determined by the Government to be the most
recent and relevant in relation to this requirement,* although *more or
less may be evaluated as appropriate.*[1]  The RFP further explained that
for this purpose *[r]ecency . . . is defined as performance occurring in
the last five (5) years*; that *[r]elevancy . . . is defined as courseware
development and aircrew training related to Factor 2, Mission Capability,
experiences, especially those provided to the military at multiple
locations*; and that *[e]xperience with training contracts, which utilize
the Air Force's model of [ISD] . . .will also be viewed favorably.* 
Completed Contractor Performance Assessment Reports (CPAR) were said to be
the primary source for evaluating past performance.  RFP at 40, 46.
    
The Air Force received four proposals, including SDS's and CBD's, by the
May 23 closing date.  A source selection evaluation team (SSET) rated
proposals utilizing an adjectival scale for rating past performance, a
color-coded scale for rating mission capability, and a graduated risk
scale for rating risk.[2]  Following the initial evaluation, all proposals
were included in the competitive range. 
    
The Air Force conducted discussions with the offerors.  In the discussions
with SDS and CBD the agency expressed concern that their proposals did not
adequately address how they would provide Web access capability in
accordance with
SOW P: 1.7.7.  Both offerors responded to these concerns.  SDS provided
further details about how it would furnish this capability.  CBD, which
did not address this requirement in its initial proposal, responded with
details about how this capability would be provided, and by proposing a
*[DELETED].*
    
Final proposal revisions were submitted by May 30.  SDS and CBD submitted
the most highly rated final proposal revisions as follows:
    

   +------------------------------------------------------------------------+
|   |Past Performance          |Mission Capability/Risk      |Price      |
|---|                          |-----------------------------+-----------|
|   |                          |Subfactor|Subfactor|Subfactor|           |
|   |                          |No.  1   |No.  2   |No.  3   |           |
|---+--------------------------+---------+---------+---------+-----------|
|SDS|Very Good/Significant     |Blue     |Green    |Blue     |$15,964,852|
|   |Confidence                |---------+---------+---------|           |
|   |                          |Low      |Low      |Low      |           |
|---+--------------------------+---------+---------+---------+-----------|
|CBD|Exceptional/High          |Blue     |Green    |Green    |$17,916,218|
|   |Confidence                |---------+---------+---------|           |
|   |                          |Low      |Low      |Low      |           |
+------------------------------------------------------------------------+

    
Agency Report, Tab 14, Source Selection Decision, at 2. 
    
The source selection authority (SSA), after discussions with *the SSET
Chair, contracting officer, and various Source Selection Advisors,*
decided to increase CBD's rating to blue for the second subfactor of the
mission capability factor, determining that *[CBD's] proposal [in this
area] exceed[ed] the stated requirements in a beneficial way.*  Id. at
2-3. 
    
In his source selection decision, the SSA concluded that the technical
strengths associated with CBD's more highly-rated proposal were worth the
higher price. 
For example, the SSA noted that CBD received an exceptional rating for
past performance because it had four relevant contracts with CPARs, in
which the majority of the performance ratings were *exceptional* and
included a statement that the rater *definitely would award* CBD another
contract.  On the other hand, the SSA noted that SDS, as the incumbent
contractor, received very good ratings based on two CPARs it received for
the current contract that reflected a majority
of very good ratings and a statement that the rater *probably would award*
SDS another contract.  Id. at 3-4.  Although the record is not entirely
clear on this point,
it appears that the other seven contracts that SDS had offered as evidence
of its experience were determined either not relevant or recent enough,
and were not considered in the very good rating.   
    
The source selection decision also referenced as *extremely beneficial*
CBD's proposal of [DELETED] and a [DELETED], both of which were identified
strengths under the first subfactor of the mission capability factor.  The
decision acknowledged that SDS's proposal also has strengths under this
subfactor
(as indicated by its blue rating), which ensure that the *school is well
managed,* but mitigated these strengths by observing that SDS's approach
and staffing *does little more than otherwise maintain the status quo in
terms of overall training approach.*  Id. at 4, 6. 
    
Finally, the source selection decision references CBD's *visionary
approach of using the [DELETED]* as a particular strength to justify the
selection of the higher-priced proposal.  Id. at 6. 
    
The award was made to CBD on August 13.  This protest followed.  SDS
contends that the source selection decision was irrational because the SSA
unjustifiably increased CBD's rating under the second subfactor of the
mission capability factor
in justifying the award to the significantly higher-priced offeror, even
though CBD's proposal assertedly did not provide any additional benefit
with regard to the provision of Internet and Web services as compared to
CBD's proposal. 
    
Where an agency chooses between a higher-priced, higher-rated proposal,
and a lower-priced, lower-rated one, we will review the agency's source
selection decision to determine whether it is consistent with the stated
evaluation criteria, and whether the agency's price/technical tradeoff is
reasonable.  A source selection decision based on inconsistent or
inaccurate information concerning the technical evaluation or the relative
merits and contents of the offerors' technical proposals is not
reasonable.  New Breed Leasing Corp., B-259328, Mar. 24, 1995, 96-2 CPD P:
84 at 4.
    
As indicated above, before making the source selection decision, the SSET
chair and others briefed the SSA.  The SSET rated CBD's proposal green
under the second subfactor of the mission capability factor.  The briefing
charts identified three strengths in CBD's proposal under this
subfactor:[3]
    
[DELETED]
Agency Report, Tab 13, SSA Briefing Slides, at 24.  In justifying raising
CBD's rating from green to blue under this subfactor, the SSA recognized
and discussed all three of these strengths.  Based on our review, it
appears that the strengths relating to CBD's improved [DELETED] were
reasonably based.[4]  With regard to the third strength of an [DELETED],
the SSA found:  
    
CBD . . . also proposed an [DELETED] to determine the appropriate USAFWS
[LMS/CMS].  The last few years we have seen technology advance at an
incredible pace, and too few of those benefits have been applied to
government processes.  The approach to provide an [DELETED] that would use
Weapons School Instructor inputs to determine a custom approach is viewed
to be of significant value. . . . The personnel proposed were deemed
appropriate and highly desirable for the level of effort required.  This
approach was viewed positively and definitely exceeded the minimum
performance requirements in a way beneficial to the Air Force.
Id. at 4-5. 
    
As detailed below, the SSA's discussion of the [DELETED] strength does not
comport to the contents of CBD's proposal.  This is important here because
of the three strengths discussed by the SSA to justify raising CBD's
rating to blue under this subfactor, the [DELETED] strength was the only
one that was *viewed to be of significant value.*  Thus, CBD's blue rating
for this subfactor and the source selection decision identifying this as a
*visionary approach* in justifying the award are not, on this record,
reasonably based.
    
The change in this rating was based on CBD's response to the agency's
discussion concern about CBD's proposal's failure to discuss the migration
of courseware and curriculum to a Web environment as required by SOW P:
1.7.7.  In its response, CBD stated in pertinent part:
    
We had assumed that all [Weapons School] Database requirements in the
current contract had been fully met and that the current [Weapons School]
LAN was web‑accessible.  Discussions revealed that our assumption
was incorrect.  To correct this shortcoming, following contract award we
will institute an aggressive approach to fully meet the contract
requirements as shown below.  In addition to providing the hardware and
software outlined in our proposal, we [DELETED].  Lacking specific
knowledge of the current state of the [Weapons School] Academic Database,
we anticipate that the web-based system will require considerable effort
to establish and to provide ongoing maintenance/update support to meet all
USAFWS requirements without burdening USAF personnel.
.     .     .     .     .
    
CBD will maintain the USAFWS Academic Database to provide unclassified
material over the [Weapons School] LAN and both classified and
unclassified material on a database that can be accessed over the
Internet.  A [LMS/CMS] provides student tracking, graduate critiques, and
a site for all instructor and student academic/courseware needs will be
provided by modifying the existing [Microsoft] Access‑based
database.  We will also provide web interface via a SIPRNET web page using
Active Server Pages and file download capabilities that are SCORM
compliant. We will use [DELETED] approved by the USAFWS LAN Manager.  The
system will also record pertinent tracking information, such as web
log-on, user actions, and log-off data, while fully meeting government
security standards.  . . .
At contract start, CBD will acquire the services of a [DELETED] and
provide the Web access.
Agency Report, Tab 10, CBD Discussions, at 1-2.
    
Contrary to the source selection decision, nowhere in CBD's proposal does
it offer to provide an [DELETED] to develop a custom approach to address
technological weaknesses in the current database to meet the current and
future requirements, nor does CBD agree that it would seek input from
USAFWS instructors for this purpose.  In our view, a fairer reading of
CBD's proposal to engage an [DELETED] is that CBD recognized that it had
no knowledge of the USAFWS's current academic database and LAN, and that
[DELETED] was only to assist the [DELETED] CBD to *meet the contract
requirements.*[5]  We find no support for the SSA's conclusion that CBD
offered a *custom approach to address technological weaknesses in the
current database.*[6]  In fact, CBD's approach (quoted above) for the most
part either acknowledges, restates, or generically responds to the SOW
requirements with no discussion or promise of a customized approach to
satisfying these requirements, except for some reference to employing
[DELETED] software ([DELETED]).[7] 
See Tr. at 80-88.           
    
At the hearing at which testimony from the SSA was elicited concerning
this issue, the SSA stated that he did not read the proposals before
making his source selection decision, but reviewed and relied upon
information provided by the SSET.[8]  Tr. at 31‑32, 57, 64, 98.  The
SSA testified as to his understanding of CBD's approach to satisfying the
SOW requirements concerning migration of courseware and curriculum to a
Web environment.  He testified that he understood that CBD's proposal was
based on *leveraging [the] newest technologies* in order to make the
database Web accessible, and *to look at whatever the most current
Internet techniques or technologies are, as opposed to trying to modify an
existing database to meet that need.*  Tr. at 16-17, 20, 25, 95, 97, 99,
175-76.  However, he could not identify where in CBD's proposal these
alleged benefits are discussed, did not explain how these benefits would
necessarily accrue from the actual contents of CBD's proposals
(particularly given CBD's admission that it was not familiar with the
existing database), and was unable to detail the differences between CBD's
and SDS's approaches, other than stating his perception that CBD was
proposing to take a *fresh look* at the agency's database requirements. 
See, e.g., Tr. at 86-88, 91-98.
    
Based on the foregoing, we find that the source selection decision
contained statements and information that were not supported by the record
yet were material to the decision.
    
The Air Force argues that even accepting that the blue rating for CBD's
proposal was unjustified, the award should be upheld because of the other
strengths identified by the SSA in making the cost/technical tradeoff
decision.  The Air Force argues that these documented strengths
demonstrate that SDS was not prejudiced by any misevaluation under the
second subfactor.  Agency Post-Hearing Comments at 7‑10.  We are
unpersuaded by this argument in the circumstances of this case.  
    
While we consider the entire record, including the parties' later
explanations and arguments, we accord greater weight to contemporaneous
evaluation and source selection material than to arguments and
documentation prepared in response to protest contentions.  Boeing
Sikorsky Aircraft Support, B‑277263.2, B-277263.3, Sept. 29, 1997,
97-2 CPD P: 91 at 15. 
    
Here, the record does not evidence that the SSA's misstatement concerning
the benefits attributable to CBD's proposal was immaterial and would not
have affected the source selection decision.  As noted, the decision
erroneously described and highlighted the perceived contribution and
benefits of CBD's proposed [DELETED], and, based on this description,
characterized this perceived benefit to be of *significant value* and a
*visionary approach* in the cost/technical tradeoff. 
Thus, a key part of the source selection decision justifying the award to
CBD, notwithstanding SDS's significantly lower price, was based on
information not supported by the record.[9]  Under these circumstances, we
find no basis to find the cost/technical tradeoff reasonable, and sustain
SDS's protest on this ground. 
See OneSource Energy Servs., Inc., B-283445, Nov. 19, 1999, 2000 CPD P:
109 at 10 (source selection decision based upon erroneous portrayal of
protester's past performance was unreasonable); New Breed Leasing Corp.,
supra (source selection decision was unreasonable where it was based upon
uncorrected errors in protester's ratings and an inaccurate portrayal of
the protester's past performance, and it included conclusions about the
proposals' technical differences that were inconsistent with the
evaluation record and proposals).
    
SDS also challenges the past performance evaluation.  Based on our review
of the record, including the hearing testimony, we find no basis to find
that the Air Force's ratings of CBD's past performance as exceptional and
SDS's as very good were unreasonable.  For example, SDS's claim that its
performance should have been considered exceptional on the incumbent
contract, apparently the sole contract considered in its past performance
evaluation, has no merit, considering that the CPARs for this contract
contain a preponderance of very good ratings for SDS. 
    
SDS complains that its other contracts submitted in its past performance
proposal should also have been considered and that if they had been SDS
would have received an exceptional rating.  Where a solicitation
contemplates the evaluation of an offeror's past performance, the agency
has the discretion to determine the scope
of the offeror's performance history to be considered, provided all
proposals are evaluated on the same basis and the evaluation is consistent
with the terms of the RFP.  Systems Mgmt., Inc.; Qualimetrics, Inc.,
B-287032.5; B-287032.6, Nov. 19, 2001, 2002 CPD P: 29 at 4-5.  To the
extent that these contracts were not considered, it was because the agency
for the most part reasonably determined that they did not involve aircrew
training or because SDS had been performing under these contracts for only
a short period of time.  Nevertheless, even if these contracts (most of
which did not have CPARs) had been considered, we find no basis to believe
that this would have elevated SDS's past performance rating to an
exceptional level, given its very good rating based on the CPARs on the
most relevant incumbent contract.[10]
    
We recommend that the Air Force reevaluate the proposals, reopen
discussions and obtain revised final proposals if appropriate, and make a
new source selection.  If CBD is not the successful offeror, its contract
should be terminated and a new award made.  We also recommend that the
agency reimburse SDS the reasonable costs of filing and pursuing the
protest, including attorney's fees.  4 C.F.R. S: 21.8(d)(1) (2002).
    
SDS's certified claim for costs, detailing the time spent and the costs
incurred, must be submitted to the agency within 60 days of receiving this
decision.  4 C.F.R. S: 21.8(f)(1).
    
The protests are sustained.
    
Anthony H. Gamboa
General Counsel
    
    

   ------------------------

   [1] Contrary to the protester's argument, in our view, this provision does
not require the agency to consider four contract references where it does
not consider them relevant.
[2] The possible past performance ratings were exceptional/high
confidence, very good/significant confidence, satisfactory/confidence,
neutral/unknown confidence, marginal/little confidence, and
unsatisfactory/no confidence.  The possible color ratings were
blue/exceptional, green/acceptable, yellow/marginal, and
red/unacceptable.  The possible risk ratings were high, moderate, and
low.  RFP at 47, 49.
[3] One strength was identified for SDS's proposal for this subfactor.
[4] In contrast, SDS did not propose a [DELETED], but proposed that its
[DELETED] devote far less than half of his time to LMS/CMS management, in
addition to his primary function of updating the database.  Agency Report,
Tab 10, SDS's Response to Discussions, at 5; Tab 18, SDS's Proposal,
at 35‑37.
[5] CBD only proposed to engage an [DELETED].  There may have been no need
for SDS to propose [DELETED] for this purpose because the record reflects
that SDS *built and maintains the USAFWS academic database using the
SDS[-]established Academix* database* and because its expansion to the
Internet would be based on this *robust database,* as it was doing under
another contract.  See Agency Report, Tab 10, SDS's Discussions, at 5; Tab
18, SDS's Proposal, at 36; Hearing Transcript (Tr.) at 90-96. 
[6] Neither the identity nor qualifications of the proposed [DELETED] were
identified.  Tr. at 60, 81-82.
[7] Ironically, the benefits attributed to, but not stated in, CBD's
proposal concerning its proposed [DELETED] were actually discussed in
SDS's proposal, as indicated by the following SDS response to the agency's
discussions:
SDS has a solid background in ADL [Advance Distributed Learning] design
and SCORM standards.  Under a contract with the US Army FORSCOM, we are
currently migrating ISD-based courseware toward interactive learning
modules using Academixa: as the base engine. 
We will draw on this expertise and experience should [Weapons School]
courseware needs change in the future.  In our approach to the [Weapons
School], we will work with the [Weapons School] to identify courseware for
interactive Web applications.  When identified, we will determine the
level of interactivity expected and the requirements to configure the
content to be SCORM compliant.  Once the interactive learning modules are
completed, the courseware can be made available on the appropriate server,
and maintained concurrently with the [Weapons School] formal curricula.
Agency Report, Tab 10, SDS's Discussions, at 5.  Moreover, from our
review, it appears that SDS provided more specific details in its initial
proposal and discussion responses than did CBD about how it intends to
comply with the SOW P: 1.7.7 requirements.  While CBD suggests that SDS's
failure to offer an [DELETED] showed that firm's insensitivity to these
requirements, there is no evidence (and the agency does not argue) that
converting the Academix database for Web applications would require
[DELETED], given CBD's experience with this database and migrating it for
Web applications.
[8] Other than noting that CBD's proposal of an[DELETED], Agency Report,
Tab 13, SSA Briefing, at 24 and that the *[DELETED],* Agency Report, Tab
12, Final Proposal Analysis Report, at 28, the SSET's evaluation
documentation does not elaborate on what this [DELETED] would contribute
to CBD's proposal.  The agency did not call any SSET witness to testify on
this matter.
[9] In its Post-Hearing Comments (at 4), the Air Force essentially admits
that this was a material part of the source selection decision, since the
SSA could rely upon this perceived strength to justify the award to CBD,
notwithstanding the associated price premium.
[10] SDS raised a number of other protest issues, not specifically
discussed here, which we have carefully reviewed and find lack merit; for
example, that discussions should have been conducted with SDS regarding
its past performance, that the agency should not have considered the past
performance of CBD's large business subcontractor in the evaluation, that
CBD may violate the subcontracting limitation, and that there was not an
adequate price evaluation of CBD's proposal.