BNUMBER:  B-278839 
DATE:  March 20, 1998
TITLE: I.S. Grupe, Inc., B-278839, March 20, 1998
**********************************************************************

Matter of:I.S. Grupe, Inc.

File:     B-278839

Date:March 20, 1998

Peter B. Schipma for the protester.
Michael Colvin, Department of Health & Human Services, for the agency.
Linda S. Lebowitz, Esq., and Michael R. Golden, Esq., Office of the 
General Counsel, GAO, participated in the preparation of the decision.

DIGEST

Agency's decision not to fund the protester's proposal for a Phase II 
effort under a Small Business Innovation Research program procurement 
was not objectionable where the record supported the agency's 
conclusion that the protester's proposal was technically unacceptable.

DECISION

I.S. Grupe, Inc. protests the decision by the Agency for Health Care 
Policy and Research, Department of Health & Human Services (HHS), not 
to award it a Phase II contract under request for proposals (RFP) No. 
AHCPR-97-0021 for a project the firm proposed under the agency's Small 
Business Innovation Research (SBIR) program.  The protester contends 
that the agency improperly determined that its proposal, entitled 
"Internet Multimedia Cancer Patient Education System," was technically 
unacceptable and would not continue to be funded.

We deny the protest.

The RFP was issued on July 10, 1997, under the SBIR program.  This 
program was established under the Small Business Innovation 
Development Act of 1982, 15 U.S.C.  sec.  638 (1994), which requires 
certain federal agencies, including HHS, to establish SBIR programs.  
Under these programs, the agencies reserve a statutory percentage of 
their research and development budgets for award to small business 
concerns for research or research and development through a 
three-phase process.  The purpose of the work performed under Phase I 
is to determine the scientific or technical merit and feasibility of 
ideas submitted under the SBIR program; under Phase II (awarded on the 
basis of Phase I results), to identify the potential for yielding a 
product or process of continuing interest to the agency; and under 
Phase III (involving private capital), to pursue commercial 
applications of the research or development.  The funding vehicle for 
this HHS SBIR program in both Phase I and Phase II was contracts, 
rather than cooperative agreements or grants.

The RFP provided the following five Phase II evaluation factors and 
respective weights:  (1) the degree to which the Phase I objectives 
were met and feasibility demonstrated--25 percent; (2) the 
scientific/technical merit of the proposed Phase II research, 
including the adequacy of the objectives for addressing the 
problem/opportunity--35 percent; (3) the qualifications of the 
principal investigator, supporting staff, and consultants--20 percent; 
(4) the potential of the proposed research for technological 
innovation--15 percent; and (5) the adequacy and suitability of the 
facilities and research environment--5 percent.  Under the RFP, cost 
reimbursement contracts would be awarded to some or all offerors 
submitting technically acceptable proposals.

The three firms, including the protester, which previously had been 
awarded Phase I contracts, submitted Phase II proposals in response to 
this RFP.  The basis for the protester's proposal was the completion 
of an interactive multimedia educational system for cancer patients 
which would allow an individual patient to determine the materials 
relevant to his/her medical problem and treatment.  Proposals were 
evaluated by a panel of six individuals in accordance with the 
referenced evaluation scheme.  Of relevance to this protest, one of 
the evaluators (whom the agency designated as evaluator No. 5) was a 
medical doctor and the chief executive officer of a media firm 
specializing in medical information for consumers.  Each panel member 
initially assigned points to each offeror's proposal and provided 
narratives of the strengths and weaknesses in these proposals.  The 
evaluators then met as a group, discussed each proposal, and modified 
individual scores and narratives in light of these discussions.  The 
average of the individual scores for an offeror's proposal became the 
consensus score for that proposal.

The evaluators concluded that the protester's proposal, which was 
accompanied by a videotape showing "not the prototype product, but 
only a view of the prototype product," and which would not allow the 
viewer to actually interact with the educational system depicted, was 
technically unacceptable (consensus score--55.3 points out of a 
possible 100).  The agency ultimately awarded Phase II contracts to 
the other two offerors, whose proposals were determined technically 
acceptable (consensus scores--84.7 points and 79.2 points).

The protester challenges the evaluation of its proposal, primarily 
expressing disagreement with the evaluators' conclusions concerning 
the technical merit of the proposal.  The protester first complains 
about the evaluators' viewing of the videotape, which accompanied the 
firm's Phase II written proposal.[1]  The videotape contained a 
partial demonstration of the operational prototype developed under the 
protester's Phase I contract and incorporated a sampling of the 
multimedia objects (text, audio, images, animation, and video) 
generated during that project.  The protester's complaint, however, is 
without merit because the protester, in fact, encouraged the agency to 
distribute copies of the videotape to the evaluators for viewing 
during the evaluation process.  Specifically, the protester stated in 
the cover letter to its proposal that

     [w]e have also enclosed . . . copies of a videotape which you may 
     want to distribute with the copies of the proposal.  We 
     understand and accept that such distribution is at your sole 
     option and that the proposal in text form represents [the firm's] 
     response to your RFP. . . . Since we are working with a 
     multimedia presentation, it is often easier to show things than 
     to describe them in words, which is the reason for including the 
     videotape.  The tape shows some selected multimedia segments from 
     the prototype as examples to support the proposal text.  Only a 
     commonplace VCR is needed to view the videotape, rather than a 
     computer system.  We believe that the videotape is a useful 
     adjunct to the proposal, and we hope that you elect to distribute 
     copies of the tape to the reviewers as well as copies of the 
     text.  [Emphasis added.]

In addition, in the proposal itself, the protester requested that the 
evaluators "please view the videotape sent with the Phase II proposal 
to experience the real Phase I result."  Simply stated, to accommodate 
the protester's requests, the evaluators reasonably viewed the 
videotape in conjunction with the evaluation of the protester's Phase 
II written proposal.

The protester next complains that the evaluators, particularly 
evaluator No. 5, improperly downgraded the overall technical merit of 
its proposal.  The protester maintains that evaluator No. 5 not only 
assigned an unreasonably low number of points to its proposal based in 
large measure on his views of the content and quality of the 
videotape, but also caused the other evaluators to lower their initial 
scores assigned to the firm's proposal, with the cumulative effect 
that the overall consensus score for its proposal was improperly 
skewed.

Because of the experimental and creative nature of an SBIR 
procurement, which is not based on design or performance 
specifications for existing equipment, but rather emphasizes 
scientific and technical innovation and has as its objective the 
development of new technology, the contracting agency is given 
substantial discretion in determining which proposals it will fund.  
Quantum Magnetics, Inc., B-257968, Nov. 30, 1994, 94-2 CPD  para.  215 at 4.  
In light of the agency's discretion, we limit our review of awards 
under SBIR procurements to determining whether the agency violated any 
applicable regulations or solicitation provisions, or acted in bad 
faith.  Virginia Accelerators Corp., B-271066, May 20, 1996, 97-2 CPD  para.  
13 at 2; Bostan Research, Inc., B-274331, Dec. 3, 1996, 96-2 CPD  para.  209 
at 2.  Here, the agency's conclusion that the protester's proposal was 
technically unacceptable is supported by the record.

The record shows that evaluator No. 5 was a medical doctor and the 
chief executive officer of a media firm specializing in medical 
information for consumers.  As such, he used his particular expertise 
in two areas--medicine and media--as the basis for evaluating the 
merits of the protester's technical proposal.  While this evaluator 
believed that the theme of the protester's proposal--providing timely 
medical and treatment information to cancer patients--was the 
proposal's greatest strength, he nevertheless found weaknesses in the 
underlying details of the proposal.  Evaluator No. 5 assigned 18 out 
of a possible 100 points to the protester's proposal, and provided 
narratives of the strengths and weaknesses in the firm's proposal.

For example, evaluator No. 5 noted as a weakness that after viewing 
the videotape, he was concerned that the protester did not demonstrate 
a sensitivity to the needs of health care consumers--the target 
audience--from the perspective of language used.  This evaluator 
referenced a tenth grade literacy level as the goal mentioned by the 
protester for this project, and noted that the language appeared to be 
at a pre-medical literacy level.  The protester objects to these 
characterizations contending that if this evaluator had read the 
firm's Phase II written proposal, he would have seen the statement 
that a "sixth grade reading level is a reasonable comprehension target 
level for typical patients."  However, in the Phase I Final Report, a 
required section of the protester's Phase II proposal, the protester 
included a section entitled "Operational Reviews."  In that section, 
the protester stated that "[a]lmost all the consultants who viewed the 
operational prototype system [i.e., the videotape] were concerned 
about the intended target audience for the material and the 
organization of the presentation. . . . [S]ome reviewers found 
inconsistency with regard to the educational level assumed for the 
target audience."  More specifically, the protester referenced the 
statement of a doctor/consultant for the project who stated that 
"'portions of the text seem comprehensible by a Grade 10 or higher 
audience while other portions seem to be very much like a medical 
school text.'"  In light of the statement of the protester's own 
doctor/consultant, we cannot conclude that evaluator No. 5 was 
unreasonable in his assessment that the videotape appeared to be 
directed to a more educated audience.

Evaluator No. 5 also listed as a weakness that in preparing the 
videotape, the protester neglected to utilize the skills of those who 
clearly could have contributed to making the interactive multimedia 
educational system a higher quality product, for example, medical 
illustrators and instructional designers.  The protester did its media 
work in-house in order to save money.  The protester stated in its 
proposal that "[w]e did animation, both two-dimensional and 
three-dimensional internally. . . without the services of a graphic 
artist. . . . We did audio internally. . . . We imported graphics, and 
we both imported and internally created full motion video.  Production 
values were high throughout, yet no expensive studio facilities were 
used."  However, given the medical and media background of evaluator 
No. 5, we believe he could reasonably downgrade the protester's 
proposal for failing to utilize the skills of various media and other 
professionals who could have created a more professional, interactive 
multimedia project to educate cancer patients.  The evaluator stated 
that "in his eyes," the protester's failure to use media professionals 
"sever[ely] compromis[ed] the integrity and value of the effort . . . 
."

Accordingly, we believe the record supports evaluator No. 5's 
downgrading of the protester's proposal because of weaknesses in the 
proposal details for completing an interactive multimedia cancer 
education system.  Technical evaluators have considerable latitude in 
assigning ratings which reflect their subjective judgments of a 
proposal's relative merits.  MiTech, Inc., B-275078, Jan. 23, 1997, 
97-1 CPD  para.  208 at 5.  A protester's mere disagreement with the 
particular point scores awarded to its proposal does not render the 
evaluation unreasonable.  Id.  

We also point out that the other evaluators expressed concerns similar 
to those raised by evaluator No. 5, noting, for example, that the 
protester used very advanced and medical language; the protester 
assumed a very high literacy level for its audience; and the protester 
failed to utilize the professional services of a graphic artist, an 
audio/visual firm, and an instructional designer or health educator.  
In addition, the other evaluators believed the protester did not 
clearly address in its proposal the target audience, that is, cancer 
patients, as the firm failed to spend sufficient time testing 
materials with this audience.  Since the project was intended to 
educate cancer patients, the evaluators believed the protester had to 
focus on what a patient's needs for cancer education were, for 
example, by addressing treatment alternatives as experienced by actual 
patients.  One of the evaluators even noted that a patient educator 
was needed as a key person for this project.  Finally, with respect to 
the videotape, the evaluators commented that the information presented 
was unfocused and uneven, and the doctor who did most of the videotape 
presentation sounded like a "talking head."

In sum, while evaluator No. 5, as well as the other evaluators, 
believed that the concept proposed by the protester to educate 
consumers with respect to cancer treatments through the use of 
interactive multimedia techniques was an overall strength of the 
firm's proposal, the evaluators nevertheless determined there were 
many weaknesses in the firm's proposed approach, as evidenced by its 
written proposal and accompanying videotape.  While the protester 
expresses disagreement with the weaknesses noted by the evaluators, 
the protester has failed to show that these weaknesses were not 
supported by the record.

Finally, to the extent the protester argues that evaluator No. 5 
unfairly influenced and persuaded the other evaluators to lower the 
scores initially assigned to the protester's proposal, thereby 
resulting in a lower overall consensus score assigned to the firm's 
proposal, we point out that agency evaluators may discuss their 
individual evaluations with each other in order to reach a valid 
consensus score since such discussions generally operate to correct 
mistakes or misperceptions that may have occurred in the initial 
evaluation.  Wastren, Inc., B-276093, May 12, 1997, 97-1 CPD  para.  175 at 
3 n.2.  A consensus score need not be the score the majority of the 
evaluators initially awarded; a score may reasonably be determined 
after discussions among the evaluators.  The overriding concern in 
these matters is whether the final scores assigned accurately reflect 
the relative merits of the proposals.  Id.  In this case, based upon 
the record, including the protester's written Phase II proposal and 
the videotape, we have no basis to question the agency's decision not 
to continue to fund the protester's project because the proposal 
submissions were reasonably determined technically unacceptable.

The protest is denied.

Comptroller General
of the United States

1. Our Office viewed a copy of the videotape, which lasted 
approximately 36 minutes.