TITLE:  Systems Management, Inc.; Qualimetrics, Inc., B-287032.3; B-287032.4, April 16, 2001
BNUMBER:  B-287032.3; B-287032.4
DATE:  April 16, 2001
**********************************************************************
Systems Management, Inc.; Qualimetrics, Inc., B-287032.3; B-287032.4, April
16, 2001

Decision

Matter of: Systems Management, Inc.; Qualimetrics, Inc.

File: B-287032.3; B-287032.4

Date: April 16, 2001

Donald J. Walsh, Esq., Scaldara & Potler, for Systems Management, Inc., and
Qualimetrics, Inc., the protesters.

William A. Shook, Esq., Kelley P. Doran, Esq., and Gary J. Campbell, Esq.,
Preston, Gates, Ellis & Rouvelas, Meeds, for Coastal Environmental Systems,
the intervenor.

Gregory Petkoff, Esq., John D. Inazu, Esq., and Maj. Marc Fox, Department of
the
Air Force, for the agency.

Katherine I. Riback, Esq., and James Spangenberg, Esq., Office of the
General Counsel, GAO, participated in the preparation of the decision.

DIGEST

Protest is sustained where the procuring agency improperly relaxed for the
awardee a mandatory solicitation requirement that the weather observation
system be certified by the Federal Aviation Administration, yet did not
notify the offerors of its changed requirements, and the record evidences
that the protesters were prejudiced because they could have proposed
different systems if they had been apprised of the agency's actual
requirements.

DECISION

Systems Management, Inc. (SMI) and Qualimetrics, Inc. protest an award to
Coastal Environmental Systems under request for proposals (RFP) No.
F19628-00-R-0032, issued by the Department of the Air Force for a fixed-base
weather observation system for the 21st century (OS-21 FBS). The protesters
contend that Coastal's proposal should have been rejected as technically
unacceptable because it was not, as required by the solicitation, certified
by the Federal Aviation Administration (FAA) or a foreign agency, at the
time that proposals were due.

We sustain the protests.

The OS-21 FBS represents an integrated system of multiple weather sensors
and data automation components that is designed to continually measure the
conditions of the environment at military installations in support of flight
safety and asset protection. The RFP, issued August 24, 2000, anticipated
the award of an
indefinite-quantity contract. RFP at 21.

The evaluation factors listed in the RFP were mission capability
(technical), proposal risk, past performance and price. The RFP further
provided that, when combined, the technical and past performance factors
were more important than price. The agency warned that the contract may be
awarded to a higher-priced offer which offered technical superiority if the
price differential was determined to be worth the technical merit offered
over a lower-priced offer. RFP at 41-42.

Proposals were required to be compliant with specific sections of the
Technical Requirements Document (TRD) (RFP at 42), including the following:

The system, with the exception of Runway Visual Range (RVR) and video
monitoring capability, needs to already exist as an integrated system, and
needs to have been evaluated and certified by the FAA or similar foreign
agency prior to submission of proposal.

RFP attach. 3, TRD sect. 2.2.1 (emphasis added).

SMI's proposal in response to the RFP was based on its Automated Surface
Observing System (ASOS) platform. The FAA had previously certified this
system in accordance with FAA Circular 6000.15C. SMI included its
certification letter with its proposal. See Agency Report, Tab 8.1, SMI
Proposal, Executive Summary, attach. 2.

Qualimetrics' proposal was based on its Automated Weather Sensor System
(AWSS), which was based on the Automated Weather Observing System (AWOS) 900
series that had been previously certified by the FAA. Qualimetrics noted in
its proposal that its platform was FAA-certified and produced detailed
certification test reports for its proposed system. Agency Report, Tab 8.4,
Qualimetrics' Proposal, vol. 2, sect. i, at 1.

Coastal's proposal was based on the Stand Alone Weather Sensors (SAWS)
platform. This platform had not received any formal FAA certification at the
time of proposal submission. In fact, the only SAWS unit that had been
delivered was under a contract with the FAA, and was not yet operational and
was undergoing testing at Hanscom Air Force Base (AFB). Coastal's
Pre-Hearing Submission (March 9, 2001) at 4. Coastal's proposal noted that
"a single runway system (SAWS) for the FAA goes in next week at Hanscom
AFB." Agency Report, Tab 9.1, Coastal's Proposal, vol. 2, at 1.

Four proposals were received by the September 27 closing date. The agency
conducted written discussions with the offerors and final proposal revisions
were received by November 29. The proposals were evaluated using a
color/adjectival system with extra credit to be given to proposals that
exceed the RFP's threshold requirements. [1] Proposal risk was evaluated to
assess the risks associated with the offeror's proposed approach, as well as
the likelihood of unsuccessful contract performance. [2] RFP at 42. The
findings of the source evaluation board (SEB) are summarized as follows:

 Offeror        SMI           Offeror A    Qualimetrics   Coastal

 Mission        Color/        Color/       Color/         Color/
 Capability
 and Proposal   Risk          Risk         Risk           Risk
 Risk

 Subfactor 1.1  Green         Green        Green          Blue
 Technical
 Compliance     Low           Low          Low            Low

 Subfactor 1.2  Green         Green        Green          Green
 Interim
 Contractor     Low           Low          Low            Low
 Support

 Evaluated      [DELETED]     [DELETED]    [DELETED]      $60,038,016
 Price

Agency Report, Tab 7.3, Source Selection Decision, at 4.

The agency noted as a strength in Coastal's proposal its inclusion of a
graphical user interface. The agency viewed this feature as a strength
because it provides "[DELETED]." Agency Report, Tab 7.2, Proposal Analysis
Report, at 7. The source selection authority noted that Coastal's proposal
with the graphical user interface, as compared to the other proposals
submitted, "effectively leverage[d] the latest commercial information
technology." Agency Report, Tab 7.3, Source Selection Decision, at 3. The
source selection authority went on to note that "[DELETED]." Id. Another
feature that the agency favorably viewed in Coastal's proposal was the
[DELETED]. According to the agency, "[DELETED]." Agency Report, Tab 7.2,
Proposal Analysis Report, at 8. The source selection authority concluded
that Coastal's "significant technical merit" was worth the associated $5.1
million price premium. Agency Report, Tab 7.3, Source Selection Decision, at
4. The contract was awarded to Coastal on December 26. After receiving
debriefings, Qualimetrics and SMI filed these protests.

The protesters contend that Coastal's proposal should have been rejected as
technically unacceptable because Coastal's proposed system was not certified
by the FAA or any foreign agency prior to the submission of proposals and it
therefore did not comply with the solicitation requirement that the system
be "evaluated and certified by the FAA or similar foreign agency prior to
submission of proposal." [3] The protesters argue that the agency either
waived or relaxed this requirement, and that if they had known that FAA
certification was not required they would have proposed either systems that
were lower priced or systems that contained the very characteristics, such
as hardware and software flexibility, that the agency found so attractive in
Coastal's proposal. [4]

The Air Force responds that the protesters are reading this requirement in
an overly restrictive manner and contends that a certified system is "a
system that has been declared operational at a given location, maintenance
documentation and trained personnel are available, and logistics support in
place to support the fielded system." Contracting Officer's Statement at 5;
see also VT at 9:52. The agency states that the purpose of the certification
requirement was to obtain a vendor with experience, particularly in
obtaining certifications for its systems, and a proven technology. VT at
9:38, 10:15. With regard to the proposals received in response to the RFP,
the contracting officer stated:

The proposals detail that the offerors had sold several systems to the
[National Weather Service], FAA, and/or foreign governments that are
currently operational and, therefore, certified, thereby, satisfying the
certification requirement of the TRD. Coastal . . . has produced and fielded
such systems to both domestic and foreign governments in accordance with
numerous contracts (e.g. according to their proposal [Coastal] has had 2200
contracts over their 18[-]year history, all related to environmental
monitoring systems).

Contracting Officer's Statement at 5. The Air Force states that its
understanding of the phrase "evaluated and certified" is "deliberately
nonspecific," and that it determined that there were "multiple industry
understandings of the term ‘certification.'" Air Force Pre-Hearing
Submission (Mar. 9, 2001) at 1. In fact, in its reports, the Air Force uses
the phrase "implied certification," not to suggest an additional kind of
certification, but to contrast it to "formal FAA certification." Id. at 2
n.1. The agency concludes that because Coastal's SAWS incorporates its
[DELETED] software, which was an integral part of other systems provided by
Coastal, and because the SAWS successfully underwent final system review at
the FAA, Coastal's system based on SAWS satisfies this requirement.

We think that a reasonable reading of the requirement that the proposed
system "needs to already exist as an integrated system, and needs to have
been evaluated and certified by the FAA or similar foreign agency prior to
submission of proposal" is that a "certification" of the base system or
platform is required from an independent organization. [5] Contrary to the
agency's argument, the term "certification" does not appear to be a vague
term that connotes only that a system is "operational." [6] In fact, as
indicated by the protester, FAA Order 6000.15C (Aug. 11, 2000), which
established the Airway Facilities maintenance program and is applicable to
weather observation systems for airports, defines "certification" as
follows:

Determination, validation, and documentation that a system, subsystem, or
Service is providing or is capable of providing the advertised services to
the user. The certification includes independent determination about when a
system, subsystem or Service should be continued in, restore to, or removed
from service, and insertion of the prescribed entry in the maintenance log.

FAA Order 5000.15C, App. 1 Definitions sect. 8. Chapter 5 of this order
establishes the FAA requirements for certification of systems, subsystems,
and national airspace systems (NAS) infrastructure services provided to
users of the NAS. This chapter states the requirement for a certification to
include an "independent determination, which ascertains the quality of
advertised services, and a validation, which officially confirms and
documents the determination in the maintenance log." FAA Order 6000.15C sect.
501. The certification process is said to be applicable to systems,
subsystems, and services when they "provide essential meteorological
information for takeoff and landing aircraft at airports," such as the OS-21
FBS. Id. sect. 504. While the FAA order does not establish the certification
procedures applicable to particular systems and leaves "the choice of
methods used of certification determination . . . to the professional
judgment of the certifying System Specialist," there is no doubt that the
FAA certifies weather observation systems; indeed, the FAA-certified systems
are listed at www.faa.gov/ats/aaf/aop/300/6000.15/ at App. 3, [7] and the
protesters' ASOS and AWSS systems were among those listed when proposals
were received under this RFP. [8]

Moreover, the record shows that the agency was certainly aware that FAA
certifications were extant and relevant. Indeed, a number of the evaluators
mentioned FAA certifications as strengths in SMI's and Qualimetrics'
proposals. For example, the evaluators noted as a strength in SMI's proposal
that its ASOS system was certified by the FAA to replace the human observer.
[9] Agency Report, Tab 7.2, Proposal Analysis Report, at 6.

On the other hand, it is now clear that the agency does not want to impose
the "overly restrictive" requirement of a certification by the FAA or
similar foreign agency on the competition. [10] The Air Force program
manager stated in a hearing that was held in relation to this case that
"[o]ur system for the Air Force is not required to be certified by the FAA.
To have said flatly only an FAA-certified system would have been enforcing a
standard that's not justifiable in our minds and been overly restrictive to
competition." VT at 9:41. Later, she went on to state that "[w]e've tried to
keep this from being overly restrictive, but yet somewhat restrictive to
keep up the experience level and the commercial off-the-shelf [COTS] element
but it probably could have been worded a little bit better." VT at 9:47. In
this regard, the agency indicates the purpose of this language was to
procure a COTS system, which was not a developmental item, from an
experienced contractor who has gone through some "certification process"
with a government agency. [11] VT at 9:38. This system could have been
provided to the Air Force, National Weather Service, FAA, another arm of the
Department of Defense, or to a foreign agency. VT at 9:52, 10:14.

Thus, it is clear that the agency overstated its minimum needs in requiring
that systems be FAA-certified, and that it then either waived or relaxed
this requirement. It is a fundamental principle of government procurement
that competition must be conducted on an equal basis, that is, offerors must
be treated equally and be provided with a common basis for the preparation
of their proposals. W.D.C. Realty Corp., B-225468, Mar. 4, 1987, 87-1 CPD para.
248 at 5; Cylink Corp., B-242304, Apr. 18, 1991, 91-1 CPD para. 384 at 4. When,
either before or after receipt of proposals, the government changes or
relaxes its requirements, it must issue a written amendment to notify all
offerors of the changed requirements. Federal Acquisition Regulation
sect. 15.206(a). The same principle applies where a protester was misled into
believing that a solicitation required it to meet certain stated
requirements, whereas the agency evaluated competitors' proposals on the
basis of lesser requirements. Cylink Corp., supra; Fort Wainwright
Developers; Fairbanks Assocs., B-221374.4 et al., June 20, 1986, 86-1 CPD para.
573 at 9; Corbetta Constr. Co. of Illinois, Inc., B-182979, Sept. 12, 1975,
75-2 CPD para. 144 at 9, aff'd, B-182979, Apr. 9, 1976, 76-1 CPD para. 240. We will
sustain a protest where an agency, without issuing a written amendment,
relaxes an RFP specification to the protester's prejudice (e.g., where the
protester might have altered its proposal to its competitive advantage had
it been allowed to respond to the relaxed requirements). SRW Inc., B-284075,
B-284075.2, Feb. 16, 2000, 2000 CPD para. 43 at 3.

As detailed above, Qualimetrics and SMI have stated that had they known of
the agency's actual requirements, that FAA certification was not necessary,
they each would have submitted different systems that contain the very
characteristics that the agency found so attractive in Coastal's proposal or
which were lower priced. [12]
See SMI's Protest at 7; Qualimetrics' Protest at 4-5; Qualimetrics/SMI's
Pre-Hearing Submission (March 8, 2001) at 3-5. The Air Force does not
question the protesters' statements in this regard. [13] Under the
circumstances, the agency should have revised the solicitation to reflect
the relaxed requirements and permitted all potential offerors an opportunity
to compete on that basis, and we sustain the protest on this basis. Cylink
Corp., supra, at 5.

The agency and intervenor have raised certain alternate arguments in
response to this protest ground. Specifically, in its report on the
protests, the agency points out a letter from the project manager regarding
the past performance on Coastal's ongoing FAA contract in which he stated
that "SAWS used the latest technology in its design as a certified system
for the [FAA]." [14] Agency Report, Tab 9.3, Coastal's Past Performance
Documents. The agency claims that this is evidence that FAA has certified
Coastal's system.

However, there is no doubt from the record that Coastal's SAWS system had
not been certified by FAA prior to the submission of proposals or when this
past performance reference was submitted to the Air Force. Indeed, as noted
above, at the time of proposal submission the only SAWS unit that had been
produced was not yet operational and was still undergoing testing at Hanscom
AFB. Coastal's Pre-Hearing Submission (March 9, 2001) at 4. Moreover, when
this sentence is read in the context of the entire paragraph in which it
appears, it seems clear that no certification had been done on the SAWS
system, but the SAWS system had undergone technical review "prior to the
First Article and the Factory Acceptance Test," and it was designed to
become a certified system. [15]

The invervenor points to the "interim certification" that its SAWS system
received from the FAA on November 13, 2000, after the submission of initial
proposals, but before final proposals were due, and claims that this
satisfies any applicable FAA certification requirement imposed by the RFP.
Agency Report, Tab 9.6, FAA Interim Certification of SAWS System.

However, the Air Force has stated that the interim certification played no
part in the agency's evaluation of Coastal's proposal. [16] VT at 10:25.
Moreover, this certification was not in existence "prior to the submission
of the proposal," as required by the RFP. [17] We are also unsure whether
the interim FAA certification would satisfy the certification requirement in
any case; in this regard the protester submitted an affidavit from a former
FAA official who had determined, in his official capacity, that the SAWS
needed to be certified, and who stated the following regarding Coastal's
interim certification:

This interim certification was issued to ensure that field technicians had
the appropriate updates in their maintenance manuals for testing and
verifying that the deployed SAWS units were being properly maintained and
functioning. The interim was issued in advance of the system's deployment
and in advance of the system having been tested and accepted. The interim
certification was not validating that the system has passed its acceptance
testing. It was issued only to ensure that a step in the certification
process was eventually met. It was unusual for such an interim certification
to be issued because frequently the materials necessary to monitor and
validate deployed systems were delayed until after verification that the
system was operational.

The interim certification is not a statement that SAWS has met any of its
operational requirements, that it satisfied any of its testing criteria, or
that it performed its advertised services. It was done strictly to ensure
filed maintenance documentation would be available.

Qualimetrics/SMI's Final Submission (Mar. 23, 2001), attach. 2, Affidavit of
Former FAA Official.

Thus, we find the RFP overstated the agency's minimum needs to the prejudice
of the protesters. We recommend that the Air Force amend the RFP to
represent its actual needs concerning certification and resolicit. [18] If a
firm other than Coastal is the successful offeror, the agency should make
award to that firm and terminate Coastal's contract for the convenience of
the government. We also recommend that SMI and Qualimetrics be reimbursed
their protest costs, including reasonable attorney's fees. 4 C.F.R. sect.
21.8(d) (2000). The protesters should submit their certified claims for
costs, detailing the time expended and the costs incurred, directly to the
agency within 60 days of receipt of this decision. 4 C.F.R. sect. 21.6(f)(1).

The protests are sustained.

Anthony H. Gamboa

General Counsel

Notes

1. The agency rated proposals with color/adjectival ratings of
blue/exceptional, green/acceptable, yellow/marginal, and red/unacceptable.

2. The performance risk ratings assigned were high, moderate, or low.

3. As indicated, a weather observation system includes a set of sensors that
feed information into a data acquisition unit or platform and provides some
type of output. Videotape Transcript (VT) at 9:49. The sensors may be
tailored or vary depending on the particular site at which the system
operates. Air Force's Post-Hearing Comments (Mar. 19, 2001) at 4. Thus,
according to the protester, the only reasonable reading of TRD sect. 2.2.1 is
that the system to be certified under TRD sect. 2.2.1 is the underlying platform
of the proposed system. Qualimetrics/SMI's Comments (Feb. 26, 2001) at 2
n.1.

4. SMI has stated that but for the Air Force's insistence that offerors use
a FAA-certified platform, it would have proposed another of its systems
known as the Portable Automated Surface Observing System (PASOS). This
system is a derivative system of its ASOS system and does not carry the FAA
certification and approvals. SMI states that the "PASOS has many of the
quality features of SMI's ASOS system and is easily adapted to meet the [Air
Force]'s needs under OS-21." SMI points out that the PASOS costs
approximately 15 percent less than its ASOS system. Qualimetrics/SMI
Pre-Hearing Submission (March 8, 2001) at 3.

Qualimetrics, the lowest-priced offeror, has stated that if it had not been
required to provide an FAA-certified platform, it would have proposed its
standard AWOS system and included a Windows operating system and graphical
user interface at no additional cost. Qualimetrics goes on to state that
"[t]his would also have allowed Qualimetrics to offer many options including
those for bringing in data from other systems such as Radar plots, [High
Resolution Photograph Transmission] satellite pictures that Qualimetrics
commonly uses on its International AWOS, greater flexibility and
expandability, as well as the advanced security options available with
Windows." The FAA had previously declined to certify Qualimetrics system if
it used a Windows-based system or one containing a graphical user interface
as part of its certified AWSS system because FAA regarded such applications
as not compliant with the FAA AWSS TRD and because of security concerns.
Qualimetrics/SMI's Pre-Hearing Submission (March 8, 2001) at 4-5, attach.,
Affidavit of Qualimetrics' Vice President.

5. According to the agency and the intervenor, the Air Force's intent to
obtain an experienced vendor with an established technology, rather than
some formal certification, was communicated by the agency to prospective
offerors in the answers that it gave regarding this RFP posted on the
Internet. For example, Coastal cites the following exchange:

2) Is it correct to assume that automated weather systems previously
validated by the Government are still considered validated?

If by validated, you mean certified, the answer is yes, with respect to the
site and use of the system. The certification is provided for the site,
based on the site's operational functionality. When the system is installed,
the site is evaluated with the new system, and certified. Information
relating to the certification of proposed system should therefore reference
the site or sites where the system was installed, the procuring agency, and
a description of the level of service provided by the system.

Agency Report, Tab 11.3, Industry Day, at 1. Rather than showing that the
Air Force was not interested in certifications, in our opinion, this
response indicated that the agency expected each offeror to supply
information about the "certifications" related to its proposed system.

6. In addition, as the protester points out (Qualimetrics/SMI's Final
Submission (Mar. 23, 2001) at 4), the fact that the only SAWS unit that was
produced was not operational at the time of the submission of proposals
indicates that Coastal's SAWS system did not even meet the Air Force's
definition of "a system that has been declared operational at a given
location." Contracting Officer's Statement at 5.

7. Paragraph 3.a of the introduction to FAA Order 6000.15C Appendix 3 states
that each listed system requires certification in accordance with the
associated maintenance technical handbook.

8. We note that the weather observation/integration items certified and
listed by FAA in Appendix 3 are data acquisition units or platforms.

9. The agency and intervenor seek to draw a distinction between formal FAA
certifications and "certified by the FAA" as used in the RFP, which we find
to be unsupported and unpersuasive.

10. The Air Force submitted an affidavit from an FAA official in which he
states his understanding of the Air Force's needs, that is, that the Air
Force was not requiring "offerors to demonstrate compliance with a specific
formal FAA commissioning/certification process." We note that this
individual never states that an FAA certification was not required by the
RFP or not applicable to the systems being procured here; rather, he states
that the certification process varies from system to system based on certain
factors. Air Force's Pre-Hearing Submission
(Mar. 9, 2001), attach. 1, Affidavit of FAA Official.

11. The intervenor has made the argument that none of the systems provided
in response to this solicitation are FAA certified because each offeror had
to adjust the sensors on their proposed platforms to meet the specific
requirements of this RFP. Coastal Comments (Feb. 26, 2001) at 6-7. However,
the record before us indicates that platforms for weather observation
systems are subject to certification by FAA and does not indicate that the
use of different sensors invalidates these basic certifications. We also
note that the Air Force program manager testified, in the context of
explaining what needs to be certified, "that swapping a sensor does not make
it [the proposed system] a non-COTS product." VT  at 9:51.

12. Since the RFP informed prospective offerors that FAA certification was a
mandatory requirement, it may well have been that other prospective offerors
were deterred from competing because of this requirement. See MTS Sys.
Corp., B-238137, Apr. 27, 1990, 90-1 CPD para. 434 at 5.

13. While the intervenor questions whether either of these offerors'
proposals would have been selected if they had changed their proposals as
they claim, the fact is that these offerors did not have the opportunity to
do so and we see no basis to speculate about how such different proposals
would have been evaluated vis ï¿½ vis Coastal's proposal.

14. We note that this memorandum was not referenced in the contemporaneous
notes of the evaluators, the summary evaluator scores or the source
selection authority's memorandum in support of the award. Indeed, the Air
Force program manager testified that this letter did not form the basis for
determining that Coastal's proposed system satisfied the certification
requirement, but when it came to the evaluators' attention, it only
confirmed their view that Coastal's system had been certified. VT at 10:28.

15. The paragraph reads as follows:

During the Final System Review (FSR) conducted on May 31, 2000 Coastal
demonstrated its outstanding engineering and technical ability. The FSR was
a major technical review of the final system design prior to the First
Article and Factory Acceptance Test (FA/FAT). SAWS used the latest
technology in its design as a certified system for the Federal Aviation
Administration.

Agency Report, Tab 9.3, Coastal's Past Performance Documents.

16. The agency received this document on January 18, 2001. VT at 10:23.

17. As explained by the agency, the certification requirement was to make
sure that it acquired a system that was certified and established, something
that was not "just put together." VT at 9:46. The Air Force's program
manager stated at the hearing that, "we wanted to make sure that before they
submitted a proposal that they had some proven technology of taking sensors
to a data acquisition unit and providing successful data output of a similar
type to the OS-21 system." VT at 9:46.

18. In view of our decision and recommendation, we need not resolve the
remainder of the protest grounds. However, we do note that the evaluation
scheme for past performance, as implemented by the agency, apparently
inhibited a qualitative evaluation of the various contracts that the
offerors had performed. In this regard, the agency had an internal $60
million threshold before it would determine whether a contract was
considered "very relevant," no matter how otherwise relevant the past
performance. The protesters contend, not without some merit, that if a
contract was below the $60 million threshold, assuming it met other
relevancy criteria, it was rated the same regardless of whether it was a
$50,000 effort or a $50 million effort, so that all offerors received the
same rating regardless of their relative experience. Qualimetrics/SMI's
Comments (Feb. 26, 2001) at 10.