BNUMBER:  B-280595 
DATE:  October 23, 1998
TITLE: Keane Federal Systems, Inc., B-280595, October 23, 1998
**********************************************************************

DOCUMENT FOR PUBLIC RELEASE
The decision issued on the date below was subject to a GAO Protective 
Order.  This redacted version has been approved for public release.
     
Matter of:Keane Federal Systems, Inc.

File:     B-280595

Date:October 23, 1998

Jason I. Hewitt, Esq., Eric J. Marcotte, Esq., and Carl J. 
Peckinpaugh, Esq., Winston & Strawn, for the protester.
Jessica C. Abrahams, Esq., and Lynda Troutman O'Sullivan, Esq., Miller 
& Chevalier, and Helaine G. Elderkin, Esq., Computer Sciences 
Corporation, for Computer Sciences Corporation; Mark H. Neblett, Esq., 
Philip J. Davis, Esq., and Rand L. Allen, Esq., Wiley, Rein & 
Fielding, and Joseph T. Casey, Esq., Electronic Data Systems 
Corporation, for Electronic Data Systems Corporation; and Carla D. 
Craft, Esq., Paul N. Wengert, Esq., Lars E. Anderson, Esq., and Thomas 
J. Madden, Esq., Venable, Baetjer, Howard & Civiletti, and Bucky 
Mansuy, Esq., Lockheed Martin Corporation, for Lockheed Martin 
Technical Services, Inc., intervenors.
Robert J. Brown, Esq., and Michael K. Cameron, Esq., Department of 
Justice, for the agency.
Tania Calhoun, Esq., and Christine S. Melody, Esq., Office of the 
General Counsel, GAO, participated in the preparation of the decision.

DIGEST

1.  Protest that contracting agency improperly evaluated protester's 
proposal with respect to management and cost issues is denied where 
the record shows that the evaluation was reasonable and consistent 
with the evaluation criteria.

2.  Protest that contracting agency's best value analysis improperly 
distorted the solicitation's evaluation scheme and resulted in an 
irrational award decision is denied where, in accordance with the 
solicitation's terms, the agency's best value analysis consisted of 
the development of discriminators between competing proposals, the 
weighing of those discriminators between competing proposals, and the 
selection for award of those proposals based upon the quantity, 
significance, and applicability of the superior features proposed. 

DECISION

Keane Federal Systems, Inc. protests the award of contracts to 
Computer Sciences Corporation (CSC), Electronic Data Systems 
Corporation (EDS), and Lockheed Martin Technical Services, Inc. (LM) 
under request for proposals (RFP) No. HQ- 97-13, issued by the 
Department of Justice, Immigration and Naturalization Service (INS), 
in support of its Service Technology Alliance Resources (STARS) 
program, under which INS intends to meet all of its information 
technology needs into the next century.  Keane primarily contends that 
INS improperly evaluated its proposal with respect to management and 
cost and conducted an irrational and arbitrary best value analysis 
that improperly distorted the solicitation's evaluation scheme.

We deny the protest.

BACKGROUND

The STARS program is the result of INS's desire to form an alliance 
with information technology contractors to obtain a full range of 
technological solutions to support the agency in achieving its mission 
objectives and information technology needs into the 21st century.  
RFP  sec.  C.1.1.  This solicitation, issued on September 16, 1997, is the 
vehicle by which INS planned to obtain the services of its STARS 
contractors.

To fulfill the STARS program strategy INS intended to award one 
contract for system management and integration services; three hybrid, 
indefinite-delivery performance contracts[1] to three other firms for 
systems development, implementation, operations, and maintenance 
services; and one contract to a fifth firm for independent 
verification and validation services.  RFP  sec.  C.1.2.  All work to be 
performed under the STARS contracts will be assigned by task orders 
issued over 1 base year, with up to 4 option years available.  RFP  sec.  
B, C.1.2.  This protest concerns the award of the performance 
contracts.  

The performance contractors will be responsible for, among other 
things, ongoing systems engineering and integration to support 
implementation of a comprehensive information systems architecture; 
systems maintenance and sustaining engineering; systems management; 
designing, developing, installing, operating, and maintaining existing 
and planned communications and electronic systems; and assisting INS 
with system definition and providing development, installation, 
operation, maintenance of existing and planned end-user automated data 
processing support systems or services, user training, and video 
services.  RFP  sec.  C.4.     

Since software development is the heart of the STARS program, the RFP 
required offerors to possess a documented Software Engineering 
Institute (SEI) Software Capability Maturity Model (CMM) at level 2 or 
higher.[2]  Offerors not meeting this mandatory requirement were not 
eligible for evaluation or award--their proposals were to be rejected.  
Offerors with documented Software CMM levels higher than 2 were to be 
given credit in the overall best value analysis.[3]  RFP  sec.  L.7.2.1., 
M.2.

Awards were to be made to the offerors whose proposals represented the 
greatest overall value to the government, cost and other factors 
considered.  RFP  sec.  M.3.2.  The solicitation set forth five evaluation 
factors:  management, experience and past performance, technical, 
cost, and other business factors.  RFP  sec.  M.4.  

Within the management factor, the contract management subfactor was 
more important than the corporate management subfactor; the RFP also 
set forth numerous elements to be considered under each management 
subfactor.  RFP  sec.  M.4.1(a), M.4.3.1.  The experience and past 
performance factor consisted of two equally important subfactors, 
experience and past performance.  RFP  sec.  M.4.1(a).  Within the 
technical factor, each sample task response was considered to be 
equally important.  RFP  sec.  M.4.1(b).  All of these factors and their 
components were to be adjectivally rated.  RFP  sec.  M.4.2. 

The management and experience and past performance factors were 
considered to be equally important, and these two factors were 
significantly more important than the technical factor.  RFP  sec.  
M.4.1(a).  Collectively, the management, experience and past 
performance, and technical factors represented the performance 
capability factors, and the performance capability factors were 
significantly more important than the cost factor.  RFP  sec.  M.4, 
M.4.1(c). 

Costs and other business factors were not to be adjectivally rated.  
Information in the cost volumes was to be evaluated to determine the 
total proposed contract cost, cost realism, reasonableness, and cost 
risk.  RFP  sec.  M.4.4.  The cost factor was significantly more important 
than the other business factors.  RFP  sec.  M.4.1(d).  The other business 
factors were to be evaluated and considered and were also critical to 
INS's responsibility determination.  RFP  sec.  M.4.4.2.  

Offerors' management and experience and past performance volumes were 
to be evaluated by the management, experience & past performance 
evaluation committee (MEPPEC).  Offerors' technical volumes were to be 
evaluated by the technical evaluation committee (TEC).  Finally, 
offerors' cost and other business factors volumes were to be evaluated 
by the business evaluation committee (BEC).  At the conclusion of the 
evaluation process, each committee was to submit its findings to the 
source selection advisory council (SSAC), which was to review the 
committees' findings; conduct comparative analyses of the proposals; 
and present its recommendations to the source selection authority 
(SSA).  The SSA was to make the final source selection decision.  RFP  sec.  
M.3.1.

The determination of greatest value was to be made by comparing the 
differences in the value of the management, experience and past 
performance, and technical factors with differences in the costs 
proposed.  INS would not make an award at a significantly higher 
overall cost to achieve only slightly superior performance capability.  
INS planned to make its greatest value assessment through the 
development of trade-off analyses and other analytic studies that 
involve the assessment of benefits of superior performance capability 
features versus the added cost.  RFP  sec.  M.3.3.

INS received proposals from five offerors, each consisting of a prime 
contractor and numerous subcontractors.  The contracting officer 
included all five proposals in the competitive range based upon the 
initial evaluation.  INS conducted extensive written and oral 
discussions with each offeror and received best and final offers 
(BAFO) in April 1998.  Each evaluation committee reviewed the BAFOs 
and revised their final reports as appropriate.  The BEC listed both 
the offerors' proposed costs, corrected for any errors, and their most 
probable costs, inclusive of the BEC's cost realism adjustments.

The SSAC was given the committees' final reports, each of which 
consisted of detailed documentation of the strengths and weaknesses of 
each proposal in connection with each evaluation factor, subfactor, 
and element.  The SSAC voted unanimously to accept the findings of the 
committees.  SSAC Recommendation
at 27.  The SSAC decided to consider offerors' proposed corrected 
costs as their evaluated costs since it believed the BEC's cost 
realism adjustments reflected upper 

bounds on quantifiable risks rather than corrections for 
unrealistically low cost proposals.[4]  Id.   The final relevant 
evaluation results were as follows:[5]

                    CSC         EDS         LM         Keane

 Management     Outstanding Outstanding Outstanding    Good

 Contract ManagementOutstandingOutstandingOutstanding  Good

 Corporate ManagementOutstandingOutstandingOutstandingOutstanding

 Experience and Past PerformanceGoodGood   Good     Outstanding

 Experience     Outstanding Outstanding Outstanding Outstanding

 Past Performance  Good        Good        Good     Outstanding

 Technical      Outstanding    Good     Acceptable  Outstanding

 Proposed Costs (Millions)
                   $272
                
                               $296
                            
                                           $244
                                        
                                                       $283
                                                    
The SSAC used various methods to rank the offerors but achieved 
different results depending on the method used.  In any event, the 
SSAC decided that a further ranking of proposals was unnecessary since 
it required a detailed best value analysis.  Id. at 29.  The best 
value working group (BVWG) was appointed by the SSAC Chair to conduct 
this analysis.  As discussed in detail below, the BVWG began its task 
by identifying 53 potential discriminators based upon a review of the 
evaluation committee reports.  BVWG Report, Exhibit 1.  The BVWG next 
reviewed each proposal to ascertain which of the potential 
discriminators were actual discriminators--superior features and 
risks--and arrived at a list of 12 superior features and 5 risks.  
BVWG Report, Exhibit 2.   

After undertaking a detailed review of each proposal, the BVWG 
determined that CSC's proposal offered 11 superior features and 1 
risk; EDS's proposal offered
9 superior features and 1 risk; LM's proposal offered 6 superior 
features and 1 risk; and Keane's proposal offered 5 superior features 
and 2 risks.  SSAC Recommendation at 31.  The BVWG compared these 
results with the offerors' proposed costs and reached some 
conclusions.  

First, Keane's proposal did not present a sufficient number of 
features to warrant  paying a $39 million premium (the difference 
between Keane and LM's proposed costs) for slightly fewer features and 
added cost risk.  Second, LM's proposal offered a significant number 
of superior features, was the lowest cost offer, and had very little 
cost and performance risk.  Third, CSC was the highest-ranked offer 
based on the performance capability factors and its proposal, ranked 
third based on cost, offered a significantly greater number of 
superior features than any other offeror with little cost or 
performance risks.  Fourth, EDS's proposal offered superior features 
that were virtually identical to those proposed by CSC, its proposed 
costs were $24 million higher than CSC's, and the superior features 
found in EDS's proposal significantly exceeded those found in Keane's 
proposal with a cost that was slightly higher.  Id. at 32.  

The BVWG recommended that awards be made to CSC, EDS, and LM, and the 
SSAC adopted this recommendation.  Id. at 33-35.  The SSA decided to 
award the contracts to CSC at a maximum value of $817.9 million; to 
EDS at a maximum value of $890.2 million; and to LM at a maximum value 
of $734.1 million.[6]  In his Source Selection Decision, the SSA 
stated:

     The CSC proposal offers the most performance capability, third 
     lowest cost, and represents the greatest value to the Government 
     of all offerors.

     The EDS proposal offers high performance capability and a 
     significant number of superior features.  Although the EDS 
     proposal carries a cost premium over the unsuccessful offeror's 
     proposals, it represents the second greatest value to the 
     Government.

     The [LM] proposal offers high performance capability, a number of 
     superior features, and is the lowest cost proposal.  As such it 
     represents the third greatest value to the Government.

     In making this determination, I have considered the rankings, 
     ratings, and recommendations of the [SSAC] and evaluation 
     committees and have used them as the basis to arrive at my own 
     independent decision.

Keane filed this protest after it received its debriefing.  Keane 
primarily argues that INS improperly evaluated its management volume 
with respect to the contract management subfactor; improperly 
evaluated its cost information in assessing it several cost risks; and 
conducted an irrational and arbitrary best value analysis that 
improperly distorted the evaluation scheme. 

EVALUATION OF MANAGEMENT VOLUME

Evaluating the relative merits of competing proposals is a matter 
within the discretion of the contracting agency since the agency is 
responsible for defining its needs and the best method of 
accommodating them, and must bear the burden resulting from a 
defective evaluation.  Advanced Tech. and Research Corp.,
B-257451.2, Dec. 9, 1994, 94-2 CPD  para.  230 at 3.  Where an evaluation is 
challenged, we will examine the agency's evaluation to ensure that it 
was reasonable and consistent with the evaluation criteria.  
Professional Software Eng'g, Inc., B-272820, Oct. 30, 1996, 96-2 CPD  para.  
193 at 4.  A protester's disagreement with the agency's conclusion 
does not itself establish that the agency acted unreasonably.  Id.

Keane argues that if EDS's proposal was rated outstanding under the 
management factor its proposal should have received the same rating.  
Keane points to the fact that EDS's proposal was rated "outstanding" 
under the management factor even though it had two fewer major 
strengths and four fewer minor strengths than did Keane's proposal 
under the contract management subfactor, and the fact that Keane's 
proposal was rated "outstanding" under the corporate management 
subfactor.  

As an initial matter, as discussed below, the source selection here 
did not depend upon these adjectival ratings or the mere number of 
strengths or weaknesses found in given proposals.  Keane's narrow 
focus on these matters is meaningless because it fails to consider the 
underlying nature of the strengths found in each proposal and their 
relative importance to the evaluation as a whole.  See Innovative 
Logistics Techniques, Inc., B-275786.2, Apr. 2, 1997, 97-1 CPD  para.  144 
at 9.  Keane's only substantive challenge to the evaluation is its 
assertion that its proposal and EDS's proposal were treated unequally, 
citing several examples.  We address three of these.

Keane complains that INS evaluated EDS's proposal as having a major 
strength for "High Visibility of STARS Program within EDS" because its 
proposed STARS program is four organizational levels from the EDS 
chairman, while Keane's proposed STARS program, three organizational 
levels from its chairman, was not evaluated as a strength.  The record 
shows that EDS's evaluated strength was not attributable to the mere 
placement of the STARS program within its organization, but to many 
additional factors as well.  Final MEPPEC Report at II-41-42 and 
Appendix 3 at D-2.  As a result, we have no basis to question INS's 
evaluation.

Keane incorrectly contends that the MEPPEC's final report erroneously 
indicated that EDS's proposal was rated as having two major strengths 
for staffing when only one major strength appears in the backup 
documentation.  The record shows that, from the beginning, EDS's 
proposal was considered to have two major strengths for staffing.  
Final MEPPEC Report, Appendix 2 at D-18 and Appendix 3 at D-12.  

Keane contends that the MEPPEC improperly rated EDS's proposal as 
having a minor strength for its goal of awarding 16 percent of its 
annual revenue to small businesses and its commitment to a minimum 
set-aside of 10 percent, whereas Keane's proposal was rated as having 
the same minor strength for its proposal to allocate 28.1 percent of 
its work to small businesses and to a minimum set-aside of 12.5 
percent.  While there are differences between the two proposals, Keane 
has provided no reason for us to conclude that these differences 
merited a materially different assessment.  

EVALUATION OF COST VOLUME

The BEC's evaluation of Keane's proposal disclosed three significant 
cost risks associated with the status of its purchasing system, its 
proposed indirect rates, and its cost accounting standards (CAS) 
status.  Keane argues that each of these cost risks is unreasonable.  
We address each in turn.

One of the other business factors was "approval of business systems."  
Offerors were required to state whether all of their business systems 
which required government approval, such as their purchasing systems, 
were approved or, if not, to explain the status of their approval.  
RFP  sec.  L.7.7.6.6.  In this regard, contractors with approved purchasing 
systems generally need not obtain administrative contracting officer 
(ACO) consent for cost-reimbursement subcontracts, while contractors 
without approved purchasing systems generally do need to obtain such 
consent.  Federal Acquisition Regulation (FAR)  sec.  44.201-2.  In 
considering whether to grant consent, the ACO is to consider numerous 
factors associated with the way the prime contractor selected the 
subcontractor; "particularly careful and thorough consideration" is 
necessary when the prime contractor's purchasing system is inadequate 
or when subcontracts are proposed on a cost-reimbursement basis.
FAR  sec.  44.202-2.

To assist the BEC with its cost proposal evaluation, offerors were 
required to furnish a copy of each cost proposal and the analysis made 
thereof for each subcontract valued at $100,000 or more.  Offerors 
were required to have each of these subcontractors submit a full set 
of cost forms documenting various elements of their proposed costs.  
RFP  sec.  L.7.7.5.1.

Keane's initial proposal did not advise INS whether its purchasing 
system was approved and, in addition, the BEC found that it had not 
been given sufficient cost information to evaluate Keane's 
subcontractors.  During discussions, Keane was asked to address the 
status of its purchasing system and to provide sufficient information 
on its proposed subcontractors to allow for their evaluation.  Keane's 
answers did not specifically address the status of its purchasing 
system and provided only general information concerning its 
subcontractors.  Keane's Response to Deficiency Reports BEC-DR-346, 
BEC-DR-410.

Amendment No. 0005 revised RFP  sec.  L.7.7.5.3. to specifically instruct 
offerors that: 

     All prime contractors must show, or as a minimum explain what 
     analysis (cost analysis, price analysis, etc.) was performed on 
     subcontractor proposals to determine the reasonableness of the 
     subcontractor pricing.  If the prime contractor has an approved 
     purchasing system, the prime must affirmatively state whether 
     they did or did not adhere to their own purchasing system 
     requirements when evaluating the subcontractors proposals.

Keane's BAFO provided some additional explanation of its subcontractor 
agreements and arrangements, BAFO Cost Volume at V-B-14 through 
V-B-22, but the BEC concluded that these general and unsupported 
statements failed to meet the requirements of RFP  sec.  L.7.7.5.3.  Keane 
also advised that it did not have an approved purchasing system.  See 
BAFO Other Business Factors Volume at VI-17. 

The BEC found that Keane was the only offeror that did not provide 
required documentation of the process by which it selected 
subcontractors in accordance with FAR Part 44, and that its analysis 
was not sufficient to permit the contracting officer to authorize 
approval of subcontracting at the time of award.  Final BEC Report at 
2.  This lack of information, which might not have been necessary had 
Keane possessed an approved purchasing system, represented significant 
risk, particularly since Keane planned to subcontract out 40 percent 
of the contract effort.  The contracting officer would have to approve 
each subcontractor until Keane's purchasing system was reviewed and 
approved.  Id. at 18-19.  Keane's assertion that it identified each 
subcontractor for the government to evaluate, making its lack of an 
approved purchasing system irrelevant, is wholly unresponsive to INS's 
concerns, and we conclude that the BEC's assessment was reasonable.  

Next, in evaluating Keane's initial proposal the BEC was concerned 
that the firm's proposed indirect rates declined over the period of 
performance and were both significantly lower than historical rates 
and unsupported.  During discussions, Keane was asked to explain and 
support its proposed declining rates.  Keane generally asserted that 
it had experienced declining rates due to increased business volume 
and that these rates would continue to decline with the addition of 
STARS business.  Keane's Responses to Clarification Requests 
BEC-CR-623, BEC-CR-620.  One of Keane's subcontractors was also 
advised, during discussions, that its proposed indirect rates were 
unrealistically low and unsupported, and its response was similarly 
vague and unsupported.  Id., BEC-CR-167, BEC-CR-445, BEC-CR-166, 
BEC-CR-628, BEC-CR-146.  These explanations did not persuade INS that 
either firm's proposed indirect rates were reliable, and the BEC 
repeated its concerns during oral discussions.  

Keane's BAFO provided no substantive support for its proposed indirect 
rates but only generalized statements about its projected growth which 
Keane itself termed "optimistic."  BAFO Cost Volume at V-B-5 through 
V-B-7.  Accordingly, the BEC concluded that Keane's indirect rates, 
and those of its subcontractor, represented a significant cost risk.  
The issuance of any competitive task order on a cost-reimbursement 
basis would require that these indirect rates be routinely checked 
with the Defense Contract Audit Agency to enable the contracting 
specialist to monitor trends.  Final BEC Report at 18-20. 

Keane's argument that the BEC mechanically adjusted the firm's--and 
its subcontractor's--indirect rates without considering the positive 
impact that receiving the STARS contract would have on lowering their 
historical indirect rates is simply untrue and fails to address INS's 
concerns regarding the lack of support for these proposed indirect 
rates.  Given the unsupported discrepancy between Keane's actual 
indirect rates and its proposed indirect rates, we see nothing 
unreasonable about INS's evaluation.  See Booz-Allen & Hamilton, Inc., 
B-275934.2, May 29, 1997, 97-1 CPD  para.  222 at 5.

Finally, since the performance contracts are subject to CAS coverage, 
FAR Appendix  sec.  9901.306, the RFP asked offerors to submit their CAS 
Disclosure Statements--written descriptions of their cost accounting 
practices and procedures--  as part of their cost volumes.  RFP  sec.  
L.7.7.5.3., K.3.21.  In addition, approval of an offeror's accounting 
system was one of the business systems specified under the "approval 
of business systems" factor discussed above.  RFP  sec.  L.7.7.6.6., 
M.4.4.2.6.  In its initial proposal Keane advised that it had not 
submitted a CAS Disclosure Statement because it had not previously 
been required to do so, and that it would submit the required 
statement in 1998.  Initial Proposal Volume VI at VI-17.

Business units selected to receive a CAS-covered contract of $25 
million or more are required to submit a CAS Disclosure Statement 
before award.  FAR Appendix  sec.  9903.202-1(b)(1).  As a general matter, 
contracting officers are not to award a CAS-covered contract until the 
ACO has made a written determination that a required Disclosure 
Statement is adequate.  FAR  sec.  30.202-6(b).  Accordingly, during 
discussions, the BEC asked Keane about its CAS compliance status and 
was advised that the firm would submit its disclosure statement 
concurrent with award of this contract.  Keane's Response to 
Clarification Request BEC-CR-619.  The BEC repeated its inquiry during 
oral discussions, and expressed concern about the time it takes to 
review such statements.  

The BEC gave Keane's proposal a passing rating under the approval of 
business systems factor since it had not previously been required to 
submit a CAS Disclosure Statement.[7]  However, since the contracting 
officer may generally not award a
CAS-covered contract until the ACO has determined that the required 
Disclosure Statement is adequate, which it could not do here, and 
since the performance contracts contemplated the issuance of 
high-dollar value cost-reimbursement task orders, the BEC found the 
fact that Keane did not have approval of its cost accounting standards 
to be a significant risk because the government did not know if its 
cost accounting practices would meet government requirements.  Final 
BEC Report at 18-19.  Keane's argument that it is CAS-compliant does 
not minimize the agency's concerns about whether it actually is 
CAS-compliant, or its concern regarding the firm's ability to obtain a 
review of its CAS Disclosure Statement in a timely fashion.  Under the 
circumstances, we cannot conclude that INS's concerns here were 
unreasonable.  

While Keane is correct that the awardees' proposals evidence various 
CAS 
non-compliance issues, its implication that their proposals should 
have been found to present cost risks as well is unsupported.  Keane 
is incorrect when it argues that INS ignored these matters with 
respect to the awardees.  The record shows that the agency reviewed 
the issues, conducted discussions to gain more information, and 
ultimately concluded that the risks, if any, were not significant.  
Keane has given us no reason to think that the risks, if any, 
presented by these proposals were equivalent to the risk presented by 
its own.

As for Keane's assertion that these areas of risk were undisclosed 
evaluation criteria, the consideration of risk involved in an 
offeror's proposed approach is inherent in the evaluation of 
proposals.   DIGICON Corp., B-275060, B-275060.2, 
Jan. 21, 1997, 97-1 CPD  para.  64 at 4.  Moreover, the RFP here 
specifically informed offerors that the cost risks associated with 
their proposals would be evaluated, and that their CAS status, methods 
of selecting subcontractors, purchasing systems, and indirect rates 
would be reviewed.  Thus, the agency's consideration of risk here was 
not improper.  Id. at 5.  Finally, Keane's argument that INS failed to 
conduct meaningful discussions on these issues is without support.  
Aside from the fact that applicable regulations and RFP provisions put 
Keane on notice of most of these matters, as noted above, the record 
is replete with evidence that INS conducted meaningful discussions in 
each of these areas.[8]

BEST VALUE ANALYSIS AND SOURCE SELECTION DECISION

INS took particular care to inform offerors how their proposals were 
to be evaluated against the RFP's terms and how their proposals were 
to be selected for award.  The RFP put offerors on notice that a set 
of evaluation committees was to review, analyze, evaluate, and rate 
the various volumes of their proposals, gave them the evaluation 
roadmap that each committee planned to follow, and instructed them 
that the committees' findings were to be given to the SSAC.  The RFP 
put offerors on notice that the SSAC's role was to review the findings 
of the evaluation committees, conduct comparative analyses of the 
proposals, and present its recommendations to the SSA, who would make 
the final source selection decision.  

In this regard, offerors were told that awards were to be made to 
those offerors whose proposals represented the greatest overall value 
to the government, cost and other factors considered.  Section M.3.3. 
of the RFP told offerors precisely how the "greatest overall value" 
was to be determined:

     The determination of greatest value will be made by comparing the 
     differences in the value of the Management, Experience & Past 
     Performance, and Technical Factors with differences in the costs 
     proposed.  These evaluation factors, when combined, are 
     significantly more important than cost.  However, the Government 
     will not make an award at a significantly higher overall cost to 
     the Government to achieve only slightly superior performance 
     capability.  The Government will make this assessment through the 
     development of trade-off analyses and other analytic studies that 
     involve the assessment of benefits of superior performance 
     capability features (e.g., economic benefits clearly attributable 
     to increased INS productivity, probability of successful contract 
     performance, and/or unique and innovative approaches or 
     capabilities) versus the added cost.  Overall cost to the 
     Government may become the ultimate determining factor for award 
     of the contracts as proposals become more equal based on the 
     other factors.  The degree of equality between the Offerors' 
     proposals will be measured by the quantity, significance, and 
     applicability of the superior features proposed.

The record shows that INS followed the RFP's instructions.  

Each evaluation committee reviewed, analyzed, evaluated, and rated the 
proposals of each offeror.  The evidence of their labor is found in 
hundreds of pages of detailed documentation which contains 
factor-by-factor, subfactor-by-subfactor, and element-by-element 
analysis of how each proposal measured up to the solicitation's 
requirements.  Each evaluation report culminated in a set of 
adjectival ratings or adjusted costs, as applicable.  The SSAC 
reviewed these findings and convened the BVWG to assist with the 
comparative analysis of proposals.  The process followed for the best 
value analysis and source selection is set forth in the BVWG's Report 
and the SSAC's Recommendation, and was further illuminated by the 
MEPPEC Chair (a BVWG member), the SSAC Chair, and the SSA at a 
hearing. 

The BVWG's goal was to identify the differences--discriminators--betwe
en proposals that would be important to the INS mission and/or program 
objectives, and that would be expected to affect INS's operation under 
the contract.  SSAC Final Recommendation at 30.  To ascertain the 
universe of potential discriminators, the BVWG prepared a matrix which 
listed each evaluation factor, subfactor, and element as set forth in 
section M of the RFP, and contained a column for each offeror.  The 
BVWG reviewed each committee's final evaluation report, 
factor-by-factor, to identify superior features and risks--potential 
discriminators--found in the offerors' proposals.  The BVWG inserted 
each potential discriminator into the matrix under the appropriate 
evaluation factor, and inserted a plus (for a superior feature) or a 
minus (for a risk) in the column of each firm whose proposal contained 
the potential discriminator.  Id.; BVWG Exhibit 1; Hearing Transcript 
(Tr.) at 26-34, 70-71.  The BVWG identified 53 potential 
discriminators.

Next, the BVWG set out to determine which of these potential 
discriminators were actual discriminators--features that would have a 
significant impact on INS's mission productivity, effectiveness, and 
cost.  The BVWG reviewed each proposal, factor-by-factor, to ensure 
that it fully understood what was offered and why it did or did not 
add value.  The evidence of their labor is found in more than 100 
pages of documented analysis which links each discriminator to an RFP 
requirement, discusses the relevant proposal feature, and analyzes the 
value of that feature, if any.  BVWG Exhibit 2 Backup Documentation; 
Tr. at 34-35, 37-39.  To summarize its findings the BVWG prepared the 
following matrix which lists each actual discriminator and inserts 
pluses or minuses, as applicable, with respect to each proposal:

Qualitative  DiscriminatorsCSCEDS    LM    Keane

CMM Level               +                

Understanding/Embracing of STARS Program+++  +

Location/Facilities     +      +      +      +

Program Visibility      +      +         

Corporate Support for the STARS Performance Team+++

Merit of Proposed Subcontractors++       

Senior Program Manager  +      +      +  

Management Team Composition+   +         

Quality Assurance Role and Responsibility+++ +

Corporate Resources     +      +      +  

Degree of Corporate Commitment+          

Sample Tasks            +                    +

Performance Risk                         

Direct Labor Costs-Risk                  

Indirect Rates (Prime)-Risk                  -

Subcontractor Costs-Risk-      -             -

Distribution of Work Effort-Risk      -  

Net Totals:            10      8      5      3
Each SSAC member was given the BVWG's full report, and the SSAC 
received an extensive debriefing on the reasons various proposals did 
or did not offer the various discriminators.  SSAC Recommendation at 
30-33; Tr. at 42-43, 89, 125-30.  The SSAC concurred with the BVWG's 
recommendation and provided an extensive debriefing to the SSA.  As 
part of his debriefing the SSA was given and reviewed the SSAC's Final 
Recommendation, which contained the BVWG's report, as well as the 
BVWG's detailed analysis of proposals.   Tr. at 136, 143-46, 175-78.

Keane argues that this best value analysis improperly disregarded and 
distorted the evaluation scheme set forth in section M of the RFP and 
improperly ignored many of Keane's major strengths as found by the 
lower-level evaluation committees.  According to Keane, since the SSAC 
unanimously voted to accept the findings of the committees, it should 
have made award based on their adjectival ratings.

In a negotiated procurement with a best value evaluation plan, 
adjectival ratings are only guides to assist contracting agencies in 
evaluating proposals; they do not mandate automatic selection of 
particular proposals.  Chemical Demilitarization Assocs., B-277700, 
Nov. 13, 1997, 98-1 CPD  para.  171 at 6.  Source selection officials, which 
includes officials at an intermediate level, are not bound by the 
recommendations or evaluation judgments of lower-level evaluators, 
even though the working level evaluators may normally be expected to 
have the technical expertise required for such evaluations.  PRC, 
Inc., B-274698.2, B-274698.3, Jan. 23, 1997, 97-1 CPD  para.  115 at 7.  
Source selection officials have broad discretion in determining the 
manner and extent to which they will make use of the technical and 
cost evaluation results, and their judgments are governed only by the 
tests of rationality and consistency with the stated evaluation 
criteria.  Id.; Chemical Demilitarization Assocs., supra.

INS did not discard the lower-level evaluation results and, along with 
them, the RFP's evaluation scheme.  The record shows that INS analyzed 
those results and drew from them a list of discriminators with which 
to make its best value analysis, in accordance with the 
solicitation.[9] 

The solicitation explicitly advised offerors that INS planned to 
assess the benefits of superior features versus added cost.  RFP  sec.  
M.3.3.  Moreover, where detailed technical proposals are sought and 
detailed technical evaluation criteria are used to enable the agency 
to make comparative judgments about the relative merits of competing 
proposals, offerors are on notice that qualitative distinctions will 
be made.  Computer Sys. Dev. Corp., B-275356, Feb. 11, 1997, 97-1 CPD  para.  
91 at 6.

Information regarding specific proposal advantages or disadvantages is 
the type of information that agencies should make available to source 
selection officials to enable them to reasonably determine whether and 
to what extent adjectival evaluation ratings indicate meaningful 
differences in proposals and the resulting value of such differences.  
Israel Aircraft Indus., Ltd., MATA Helicopters Div., 
B-274389 et al., Dec. 6, 1996, 97-1 CPD  para.  41 at 7.  Such 
considerations are the essence of any best value source selection 
decision, F2M-WSCI, B-278281, Jan. 14, 1998, 98-1 CPD  para.  16 at 8, and 
the BVWG's designation of discriminators was simply and properly a 
tool to help assess which proposals represented the best value.  See 
Engineering and Professional Servs., Inc., B-262179, Dec. 6, 1995, 
95-2 CPD  para.  266
at 5.

Keane complains that the BVWG's selection of discriminators improperly 
distorted the evaluation scheme by "abandoning" the experience and 
past performance evaluation factor.  Keane points out that the 
management factor is represented by numerous discriminators.  In 
contrast, the experience and past performance factor, which was just 
as important as the management factor, is represented by only one 
discriminator, and that a negative one--performance risk.  According 
to Keane, the selection of discriminators gave disproportionate weight 
to the management factor and other, less important factors, and failed 
to give Keane credit for its superior past performance rating.

The record shows that INS did not "abandon" the experience and past 
performance factor, but that the BVWG simply did not consider 
favorable past performance to be a discriminator.  As a BVWG member 
stated at the hearing, past performance instills confidence that a 
company will probably do well but does not in and of itself guarantee 
that a company will do well, whereas poor past performance is a good 
indicator of performance risk.  Tr. at 92-96, 102-106.  

There is no requirement to have a discriminator for each evaluation 
factor, or to have an equivalent number of discriminators for equally 
important evaluation factors.  See Computer Sys. Dev. Corp., supra.  
Moreover, whenever equal factors are considered, the fact that one is 
chosen as more valuable does not mean that the relative weights of the 
evaluation factors have been changed or that one has been abandoned.  
It simply means that one has become the discriminator between 
competing proposals.  Calspan Corp., B-258441, Jan. 19, 1995, 95-1 CPD  para.  
28 at 14.  There is also no requirement that award discriminators be 
the most heavily weighted factors.  Research for Better Schools, Inc., 
B-270774.3, June 17, 1996,
96-2 CPD  para.  41 at 8.  So long as the less heavily weighted criteria 
have been disclosed to the offerors in the RFP, as they were here, 
there is nothing improper in their becoming the discriminator where 
competing proposals are evaluated as equal in the more heavily 
weighted ones.  Duke/Jones Hanford, Inc., B-249367.10,
July 13, 1993, 93-2 CPD  para.  26 at 10 n.8.

While Keane's proposal was rated "outstanding" under the past 
performance subfactor, to the other offerors' "good" ratings, the BVWG 
considered the offerors' past performance to be approximately the 
same.  Tr. at 100.  Indeed, our review of the MEPPEC's findings show 
virtually identical strengths for all offerors, and there is no basis 
for us to conclude that the minor concerns with respect to the 
awardees' past performance amounted to a material distinction.  As a 
result, we cannot conclude that the agency improperly omitted a 
positive discriminator for past performance here.       

Keane also asserts that INS improperly failed to credit the firm for 
offering two discriminators, one for a superior senior program 
manager, and one for possessing a CMM level higher than 2.  With 
respect to the first, Keane merely argues that it received a major 
strength for its senior program manager and, thus, should have been 
credited with this discriminator.  Since Keane does not address the 
BVWG's detailed rationale for crediting other offerors with this 
discriminator, and not itself, we have no basis to question the 
agency's assessment.  See BVWG Exhibit 2 Backup Documentation; Final 
MEPPEC Report, Appendix 3 at C-11; Appendix 2 at D-18; Appendix 3 at 
F-9.  With respect to the second, Keane's allegation is untimely.

On the day of its debriefing Keane was given redacted copies of BVWG 
Exhibits 1 and 2.  The first potential discriminator listed on exhibit 
1 was "CMM Level . . . Higher than Level 2, per RFP."  As noted above, 
the RFP specifically provided that offerors with documented Software 
CMM levels higher than 2 were to be given credit in the overall best 
value analysis.  RFP  sec.  M.2.  There was no plus in Keane's column for 
this discriminator.  This discriminator was abbreviated and carried 
over onto exhibit 1; again, there was no plus in Keane's column for 
this discriminator.

Despite Keane's knowledge on that day that offerors with CMM levels 
higher than 2 were to be given credit in the best value analysis; its 
belief that it offered a
CMM level 3; and its knowledge that it was not given credit for having 
a CMM level higher than 2 in the best value analysis as set forth in 
BVWG exhibits 1 and 2, Keane failed to raise this issue until its 
comments on the agency report.  In view of the detail available to 
Keane when it filed its protest, it was required to raise the issue 
then and its failure to do so renders the issue untimely.  Global 
Eng'g & Constr. Joint Venture, B-275999.4, B-275999.5, Oct. 6, 1997, 
97-2 CPD  para.  125 at 4.

Finally, Keane argues that the award decision was irrational and 
arbitrary because it reduced the entire source selection decision to a 
tradeoff between the net totals of superior features versus cost 
risks.  As discussed above, while the BVWG exhibits list net totals, 
and the SSAC's recommendation and source selection decision speak in 
terms of the numbers of features, the record is abundantly clear that 
the award decision was not based on a simple "scorecard" as Keane 
suggests.  Instead, the SSAC and SSA relied on these numbers as a 
shorthand for the well-documented, reasonable findings of the BVWG; 
the source selection decision was clearly based on the BVWG's findings 
as a whole.  See Israel Aircraft Indus., Ltd., MATA Helicopters Div., 
supra, at 7; Tr. at 111-17, 147-48, 178-80.  

The rationale for the source selection decision is clearly outlined in 
the full BVWG Report, the SSAC Recommendation, and the source 
selection decision.  As discussed above, the rationale is rational, 
supported by the evaluation record, and
consistent with the evaluation criteria.  As a result, there is no 
basis to object to INS's selection of CSC, EDS, and LM for contract 
award, and no basis to object to INS's decision not to award a 
contract to Keane.  Calspan Corp., supra, at 14.

The protest is denied.

Comptroller General
of the United States

1. INS intended to award three performance contracts but reserved the 
right to award more or fewer such contracts based on the evaluation 
results.  RFP  sec.  C.1.2.

2. The CMM is a tool for assessing and evaluating the maturity of an 
organization's software processes.  The model, which identifies five 
levels of maturity, provides for software process assessments--or 
self-assessments--that allow organizations to implement improvement 
programs and for software capability evaluations that allow evaluators 
to identify the risks of selecting among different contractors for 
award.  Research Analysis and Maintenance, Inc., B-272261, B-272261.2, 
Sept. 18, 1996, 96-2 CPD  para.  131 at 4 n.1.

3. Since INS evaluated Keane's proposal and considered it for award, 
and since we conclude that the evaluation and source selection 
decision were reasonable, we need not resolve the question, raised by 
INS during the pendency of the protest, whether Keane's proposal 
actually met this mandatory requirement.  As discussed further below, 
Keane's allegation that INS improperly failed to credit it with 
possessing a documented CMM level 3 in the best value analysis is 
untimely.

4. As a result, our consideration of INS's cost realism analysis need 
only address those areas of cost risk that became important in the 
best value analysis.

5. The ratings assigned to the adjectivally rated other business 
factors are not at issue here, and all offerors were rated as passing 
the remaining other business factors. 

6. These values are based upon the RFP's overall contract maximum 
value, which was computed by multiplying each offeror's total 
evaluated costs by a factor of three.  This formula assumes the 
possibility that each performance contractor might perform all of the 
work ordered under the performance contracts.  RFP  sec.  H.10.2.2.

7. As revised by amendment No. 0005, offerors were to be given passing 
ratings if none of their business systems that required approval had 
significant outstanding citations of non-approval that might affect 
their eligibility for a cost-type contract, and if the offeror had no 
significant CAS violations that might affect their eligibility for a 
cost-type contract.  RFP  sec.  M.4.4.2.6.

8. In its comments Keane pointed out that the BEC found six 
significant cost risks in EDS's proposal but the BVWG only considered 
one of these to be a discriminator.  In response to our request, INS 
provided a detailed explanation for the BVWG's decision, which Keane 
failed to address in its subsequent response.  As a result, we 
consider the issue to have been abandoned and will not consider it.  
IT Corp., 
B-258636 et al., Feb. 10, 1995, 95-1 CPD  para.  78 at 8 n.11. 

9. The 17 discriminators selected by the BVWG are not, as Keane 
insists, undisclosed evaluation criteria.  All are drawn directly, and 
many verbatim, from section M of the solicitation.