[Federal Register Volume 68, Number 16 (Friday, January 24, 2003)]
[Rules and Regulations]
[Pages 3586-3638]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 03-1294]



[[Page 3585]]

-----------------------------------------------------------------------

Part II





Department of Health and Human Services





-----------------------------------------------------------------------



Centers for Medicare & Medicaid Services



-----------------------------------------------------------------------



42 CFR Parts 433 and 438



Medicaid Program; External Quality Review of Medicaid Managed Care 
Organizations; Final Rule

  Federal Register / Vol. 68 , No. 16 / Friday, January 24, 2003 / 
Rules and Regulations  

[[Page 3586]]


-----------------------------------------------------------------------

DEPARTMENT OF HEALTH AND HUMAN SERVICES

Centers for Medicare & Medicaid Services

42 CFR Parts 433 and 438

[CMS-2015-F]
RIN 0938-AJ06


Medicaid Program; External Quality Review of Medicaid Managed 
Care Organizations

AGENCY: Centers for Medicare & Medicaid Services (CMS), HHS.

ACTION: Final rule.

-----------------------------------------------------------------------

SUMMARY: This final rule establishes requirements and procedures for 
external quality review (EQR) of Medicaid managed care organizations 
(MCOs) and prepaid inpatient health plans (PIHPs). It defines who 
qualifies to conduct EQR and what activities can be conducted as part 
of EQR. In addition, under certain circumstances, this rule allows 
State agencies to (1) use findings from particular Medicare or private 
accreditation review activities to avoid duplicating review activities, 
or (2) exempt certain Medicare MCOs and PIHPs from all EQR 
requirements. Also, this rule allows the payment of enhanced Federal 
financial participation (FFP) at the 75 percent rate for the 
administrative costs of EQRs or EQR activities that are conducted by 
approved entities.

EFFECTIVE DATE: These regulations are effective on March 25, 2003. 
Provisions that must be implemented through contracts with MCOs, PIHPs, 
and external quality review organizations (EQROs) are effective with 
contracts entered into or revised on or after 60 days following the 
publication date. States have up until March 25, 2004 to bring 
contracts into compliance with the final rule provisions.

FOR FURTHER INFORMATION CONTACT: Kristin Fan, (410) 786-4581.

SUPPLEMENTARY INFORMATION: To order copies of the Federal Register 
containing this document, send your request to: New Orders, 
Superintendent of Documents, P.O. Box 371954, Pittsburgh, PA 15250-
7954. Specify the date of the issue requested and enclose a check or 
money order payable to the Superintendent of Documents, or enclose your 
Visa or Master Card number and expiration date. Credit card orders can 
also be placed by calling the order desk at (202) 512-1800 or by faxing 
to (202) 512-2250. The cost for each copy is $10. As an alternative, 
you can view and photocopy the Federal Register document at most 
libraries designated as Federal Depository Libraries and at many other 
public and academic libraries throughout the country that receive the 
Federal Register.
    This Federal Register document is also available from the Federal 
Register online database through GPO access, a service of the U.S. 
Government Printing Office. The Website address is http://www.access.gpo.gov/nara/index.html.

I. Background

A. The Balanced Budget Act of 1997

    The Balanced Budget Act of 1997 (BBA) added to the Social Security 
Act (the Act) a new section 1932 that pertains to Medicaid managed 
care. Most of the provisions of section 1932 of the Act will be 
implemented in accordance with the Medicaid managed care final rule 
that was published in the Federal Register on June 14, 2002 (67 FR 
40988).
    Section 1932(c) of the Act, added by section 4705 of the BBA, 
describes how quality measurement and performance improvement methods 
should be applied to Medicaid managed care programs through two 
specific approaches:
    [sbull] All State agencies must develop and implement a quality 
assessment and improvement strategy that includes--(1) Standards for 
access to care; (2) examination of other aspects of care and services 
related to improving quality; and (3) monitoring procedures for regular 
and periodic review of the strategy. (This requirement was addressed in 
the Medicaid managed care final rule published June 14, 2002.)
    [sbull] State agencies that contract with Medicaid managed care 
organizations (MCOs) must provide for an annual external, independent 
review of the quality outcomes, timeliness of, and access to the 
services included in the contract between the State agency and the MCO. 
(This requirement is addressed in this rule.)
    Section 1932(c) of the Act also requires the Secretary--
    In consultation with the States, to establish a method for 
identifying entities qualified to conduct external quality review (EQR) 
(section 1932(c)(2)(A)(ii) of the Act); and
    In coordination with the National Governors Association (NGA), to 
contract with an independent quality review organization to develop the 
protocols to be used in EQRs (section 1932(c)(2)(A)(iii) of the Act).
    Two other provisions of section 1932(c) of the Act are pertinent to 
this rule. They are (1) the requirement that the results of EQRs be 
made available to participating health care providers, enrollees and 
potential enrollees (section 1932(c)(2)(A)(iv) of the Act), and (2) the 
provision that a State agency may, at its option--
    [sbull] Take steps to ensure that an EQR does not duplicate a 
review conducted either by a private independent accrediting 
organization or as part of an external review conducted under the 
Medicare program (section 1932(c)(2)(B) of the Act); and
    [sbull] Exempt an MCO from EQR under certain specified conditions 
(section 1932(c)(2)(C) of the Act).
    Section 4705(b) of the BBA amended section 1903(a)(3)(C) of the Act 
to provide for increased Federal financial participation (FFP) (75 
percent) for the administrative costs the State incurs for EQR or EQR 
activities performed by specified entities under section 1932(c)(2)(A) 
of the Act.

B. Proposed Rule

    On December 1, 1999 we published a proposed rule in the Federal 
Register (64 FR 67223) to implement the EQR statutory provisions. A 
summary of the specific provisions of the proposed regulations precedes 
each section of the comments and responses below. In the proposed rule, 
we discussed the two major purposes we had in developing the rule: (1) 
To provide flexibility for State agencies, and (2) to reflect the well-
accepted advances in the technology of quality measurement and 
improvement. For a more detailed discussion of our basis and purpose 
for the approach taken in the December 1, 1999 proposed rule, see the 
preamble to that document at 64 FR 67223.
    We received 29 comments from States, national and State 
organizations, health plans, advocacy groups, and other individuals on 
the December 1, 1999 proposed rule. The comments generally pertained to 
the types of entities that can be EQROs, EQR activities, nonduplication 
and exemption provisions, and dissemination of EQR rules. We carefully 
reviewed and considered all the comments we received.

C. Agency Information Collection Activities

    On November 23, 2001 we published a notice in the Federal Register 
(66 FR 58741) to comply with the requirement of section 3506 (c)(2)(A) 
of the Paperwork Reduction Act of 1995. We invited public comment 
regarding the burden estimate or any other aspect of the EQR protocols 
we developed in accordance with section 1932(c)(2)(A)(iii) of the Act. 
This

[[Page 3587]]

provision required that we contract with an independent quality review 
organization to develop protocols to be used with respect to EQRs 
required by statute. In response to the requirement under section 
1932(c)(2)(A)(iii) of the Act, we contracted with the Joint Commission 
on Accreditation of Health Care Organizations (JCAHO) which developed 
nine protocols and one appendix to several of the protocols in six 
quality improvement areas. We received 13 comments on the November 23, 
2001 Federal Register notice. We carefully reviewed and considered all 
the comments we received.

II. Provisions of the Proposed Rule and Discussion of Public Comments

A. Basis, Scope and Applicability. (Formerly Sec.  438.1), (Now Sec.  
438.310)

    In this section we proposed to apply provisions to MCOs, prepaid 
health plans (PHPs), and entities with comprehensive risk contracts 
that are exempted by statute from the requirements in section 1903(m) 
of the Act, health insuring organizations (HIOs).
    Comment: Many commenters supported the application of this rule to 
all three of the above types of entities. One commenter, though not 
opposed to the inclusion of PHPs, expressed concern about the cost of 
this requirement when applied to entities that provide services to 
small populations. The commenter suggested that the regulation apply 
only to entities to the extent feasible for the study being performed. 
Another commenter did not agree that the provisions should apply to 
PHPs and stated that there is no specific reference in Federal law to 
these organizations and that we have gone beyond the explicit language 
in section 1932(c) of the Act.
    Response: We continue to believe these provisions should apply to 
most capitated health plans that are not MCOs, but that provide 
inpatient services. The Medicaid managed care final rule eliminated the 
term PHP and replaced it with two types of entities--prepaid inpatient 
health plans (PIHPs) and prepaid ambulatory health plans (PAHPs). That 
rule, under the authority of section 1902(a)(4) of the Act, which 
authorizes the Secretary to establish requirements necessary ``for 
proper and efficient operation of the plan,'' applies the provisions 
related to a State's quality strategy to PIHPs but not to PAHPs. It 
does not apply these quality provisions to PAHPs because these entities 
provide a more limited array of services (for example, transportation 
or dental), and we do not believe it appropriate to require States to 
include these entities in their State quality strategies due to the 
burden it would impose. We, therefore, are revising this rule to be 
consistent with the Medicaid managed care final rule (Sec.  438.204(d)) 
and apply the EQR provisions to PIHPs as specified at Sec.  438.310. We 
have also made changes to clarify the applicability of this rule to 
HIOs to be consistent with the Medicaid managed care final rule.
    We do not agree with the commenter that we should exempt entities 
that have smaller enrolled populations from these requirements. 
Sections 1932(c)(2)(B) and (C) of the Act specifically identify the 
circumstances under which an entity may be fully or partially exempt 
from EQR.
    Comment: One commenter asked if we intend to hold Indian Health 
Services (IHS) and 638 Tribal Facilities to the same standard as MCOs 
to ensure the quality of care provided to Native Americans.
    Response: If an IHS entity or 638 Tribal Facility meets the 
definition of an MCO or PIHP, it would be subject to these provisions.
    Comment: One commenter does not believe that primary care case 
management (PCCM) programs should be subject to these requirements. 
Another commenter believes that the activities in the December 1, 1999 
proposed rule should be applied to PCCM programs.
    Response: The statute does not extend the EQR requirement to PCCMs 
and the Conference Report, pages 859-860, makes clear that PCCMs were 
specifically excluded from the requirements. We have used the authority 
of section 1902(a)(4) of the Act to extend the EQR provision to PIHPs 
because, like MCOs, PIHPs provide inpatient services and are capitated. 
If a PCCM meets the definition of a PIHP, then it would be subject to 
the provisions of this rule. However, traditional PCCMs are reimbursed 
on a fee-for-service (FFS) basis along with a case management fee. 
Under that reimbursement arrangement, the PCCM would not be subject to 
the EQR requirements.
    Comment: Many commenters recommended that external review also 
examine subcontracting managed care entities. One commenter suggested 
that the definition of quality be expanded to include services provided 
through subcontracts with MCOs.
    Response: The MCO or PIHP is fully responsible (Sec.  438.230 of 
the Medicaid managed care final rule) for all activities delegated to 
another entity. Therefore, the EQR should include information on all 
beneficiaries and the structure and operations of all entities that 
provide Medicaid services under either the prime contract or 
subcontract. At Sec.  438.320, we revised our definition of EQR to 
clarify our intent that the EQR provisions apply to all services 
received by Medicaid beneficiaries regardless of whether those services 
are provided by the MCO or PIHP directly or through a subcontract.
    Comment: One commenter is concerned that this rule applies the EQR 
requirement to PHPs despite the BBA's statutory reference only to 
organizations under section 1903(m) of the Act. The commenter asked us 
to clarify whether we intend to apply these requirements to any entity 
that is paid on a prepaid capitation basis for services furnished to 
enrollees, even if the PHP is not at any financial risk for those 
services.
    Response: As noted in an earlier response, the EQR provisions will 
apply to a PIHP defined in the Medicaid managed care final rule as an 
entity that ``provides medical services to enrollees under contract 
with the State agency, and on the basis of prepaid capitation payments, 
or other payment arrangement that do not use State plan payment rates 
and that provides, arranges, or otherwise has the responsibility for 
the provision of any inpatient hospital or institutional services for 
its enrollees * * *'' We do not apply these quality provisions to PAHPs 
because these entities provide a more limited array of services (for 
example, transportation or dental), and we do not require States to 
include these entities in their State quality strategies due to the 
burden it would impose. The application of this rule to PIHPs is not 
based on section 1903(m) of the Act. It is based on section 1902(a)(4) 
of the Act that authorizes the Secretary to establish requirements 
necessary ``for the proper and efficient operation of the plan.'' We 
believe this is consistent with congressional intent.
    PIHP and PAHP designation is not based on whether an entity is at 
financial risk for services provided. Designation is based on prepaid 
capitation payments for a scope of services. Even though there will be 
few PIHPs that are not at financial risk, due to the scope of services 
these entities provide (for example, inpatient services), we believe 
they should be subject to EQR provisions.

B. Definitions (Formerly Sec.  438.2), (Now Sec.  438.320)

    This section of the proposed rule defined ``EQR'' and ``EQRO.'' It 
also defined the terms ``quality'' and ``validation'' as they pertain 
to EQR.

[[Page 3588]]

    Comment: One commenter concurred with our requirement that EQR be a 
multipronged approach which recognizes that none of the activities 
alone can ensure quality in the complex Medicaid population. One 
commenter supported the definitions as proposed.
    Response: We appreciate that the commenters agreed with our 
approach to EQR and the proposed definitions. We have retained the 
multipronged approach to EQR as proposed in the proposed rule.
    Comment: One commenter asked that the definition of quality include 
assessments of structure and process as well as measurements of health 
and functional outcomes. Several commenters recommended that the 
definition of quality include both clinical and nonclinical measures of 
consumer satisfaction and define quality in a way that would be 
meaningful to people with disabilities. One commenter stated that this 
definition should address the multifaceted needs of people who have 
chronic and disabling conditions, for whom there is little likelihood 
of demonstrable improvement. The commenter recommended that we convene 
focus groups of consumers, including people with disabilities and 
families of children with disabilities, to identify how quality should 
be defined from the consumer's perspective and that the definition 
should not focus solely on health outcomes. One commenter concurred 
with the definition of quality as proposed.
    Response: We agree with the commenter that the proposed definition 
of quality did not address situations when beneficiaries have 
conditions where maintenance or improvement of health outcomes is not 
likely. We have, therefore, revised the definition to mean the degree 
to which an MCO or PIHP increases the likelihood of desired health 
outcomes through the provision of health services that are consistent 
with current professional knowledge. The revision is consistent with 
the Institute of Medicine's definition of quality. We do not agree with 
the remaining recommendations by commenters on how to revise the 
definition of quality because we think that the commenters' concerns 
are addressed by other provisions of the regulation. Under Sec.  
438.358, we identify three activities that must be conducted to provide 
information for the EQR. These activities also are required in the 
Medicaid managed care final rule. They include: (1) The review of 
compliance with structural and operation standards; (2) the validation 
of performance measures;\1\ and (3) the validation of performance 
improvement projects. The optional EQR-related activities are 
activities that some States currently conduct as part of EQR and we 
believe are also appropriate to an assessment of quality (such as 
consumer surveys). We are providing States with the flexibility to 
determine which, if any, of these optional activities will be included 
in the EQR and what types of performance measures and performance 
improvement projects to require of their contracting MCOs and PIHPs. We 
suggest in the performance improvement project protocol that projects 
be conducted to address both clinical and nonclinical areas that cover 
the various categories of beneficiaries and services provided. We also 
note, as stated in the Medicaid managed care final rule, that EQR is a 
part of the State's quality strategy, and therefore, States are to 
provide for the input of Medicaid beneficiaries and other stakeholders 
in this component of the strategy.
---------------------------------------------------------------------------

    \1\ In the Medicaid managed care final rule under Sec.  
438.240(c)(2) we permit States to calculate performance measures on 
the MCO's/PIHP's behalf in place of the MCO/PIHP calculating and 
reporting performance measures to the State. Under this 
circumstance, the validation of MCO/PIHP performance measures is not 
required as a mandatory activity but the State must submit the 
State-calculated performance measures to the EQRO for the EQR 
function as specified under Sec.  438.358(b)(2). This issue is 
addressed later in the preamble in response to a comment.
---------------------------------------------------------------------------

    Comment: One commenter suggested amending the definition of EQR to 
read ``* * * quality of health care services furnished or contracted 
for by each MCO * * *''
    Response: We agree with this comment and, as stated previously, 
have revised the final rule to clarify our intent that the EQR 
provisions apply to all services received by Medicaid beneficiaries 
regardless of whether those services are provided by the MCO or PIHP 
directly or through a subcontract (Sec.  438.320).
    Comment: Several commenters stated that the definition of EQR too 
narrowly limits the scope of EQR because the definition implies that 
EQR is primarily concerned with analysis and evaluation of data rather 
than with collection of data. One of the commenters expressed concern 
that this would limit the EQRO's ability to identify and bring to the 
State's attention individual quality of care concerns revealed during 
data abstraction, or to provide provider-specific feedback on 
performance measures. The commenter recommended that the rule avoid any 
reference to ``aggregate'' information in the definition of EQR. One 
commenter recommended that the definition of EQR include the 
development of aggregated data. Another commenter stated that external 
review should not be limited to the review of information. The 
commenter believes the external review of plans should include an on-
site review of provider practices and procedures and that data alone 
are insufficient to evaluate performance.
    Response: We do not agree that the definition of EQR limits the 
scope of EQR. We define EQR as the analysis and evaluation of 
aggregated information. That aggregated information, according to this 
rule, must be obtained from activities that are consistent with 
protocols, as defined in this rule, to ensure that data to be analyzed 
are collected using sound methods widely used in the industry. For each 
activity, as specified in Sec.  438.364, the entity conducting the 
activity must report on the objectives, technical methods of data 
collection and analysis, a description of the data obtained, and 
conclusions drawn from each activity. Therefore, as part of these 
activities, the entity conducting them will need to identify and assess 
quality of care concerns revealed by the activities. The EQR analysis 
will incorporate findings from all activities, including the evaluation 
of MCO or PIHP structure and operations. The findings of the overall 
analysis will need to include an assessment of the strengths and 
weakness with respect to quality, timeliness, and access of care, and 
make recommendations for MCO or PIHP improvement in the EQR results as 
required under Sec.  438.364. Further, we note that under the BBA 
statutory provisions, EQR is a review of a Medicaid MCO under contract 
to the State. EQR of individual providers or provider practices is not 
provided for in the BBA. We believe that the appropriate unit of 
analysis of EQR is the MCO and PIHP, not individual practitioners.

C. State Responsibilities (Sec.  438.350)

    This section of the proposed rule set forth the State's 
responsibilities related to EQR. We proposed that each State agency 
that contracts with MCOs, PHPs, or other entities that have 
comprehensive risk contracts must, except as provided in Sec.  438.362, 
ensure that (1) An annual EQR is performed for these contracting 
entities by a qualified EQRO; (2) the EQRO has sufficient information 
to use in performing the review; (3) the information that the State 
agency provides to the EQRO is obtained through methods consistent with 
protocols specified by CMS; and (4) the results of the EQR are made

[[Page 3589]]

available, upon request, to specified groups and to the general public.
    Section 1932(c)(2)(A) of the Act requires that each contract with 
an MCO ``provide for an annual (as appropriate) external independent 
review, conducted by a qualified independent entity * * *'' In this 
section we interpreted the parenthetical statement (for which there is 
no explanation in the legislative history) to be a reference to those 
MCOs that may be exempted from EQR under section 1932(c)(2)(C) of the 
Act on the basis of ``deemed compliance.'' We invited comment on other 
possible interpretations, which are discussed at the end of this 
section.
    Comment: One commenter noted they concurred with this section of 
the rule.
    Response: We appreciate the commenter's support for the provisions 
in this section of the proposed rule and retain the provision that 
requires the State to ensure that the EQRO has information obtained 
from EQR-related activities and that the information provided is 
obtained through methods consistent with the EQR protocols established 
under Sec.  438.352 in this final rule.
    Comment: Several commenters asked us for a definition, or the 
criteria that we will use to determine if State-established protocols 
are consistent with those developed by us. One of the commenters noted 
that it would be difficult for all States to follow a single set of 
protocols because State Medicaid programs vary as to structure, 
capacity, funding, and governing laws. One commenter asked that we also 
establish criteria for denominators, numerators, and units of 
measurement for performance measures. Other commenters concurred with 
the requirement to use protocols that are ``consistent with'' rather 
than ``identical to'' those developed by us to accommodate the rapidly 
changing field of quality assessment and improvement.
    Response: Section 1932(c)(2)(A)(iii) of the Act required the 
Secretary in coordination with the National Governors Association, to 
contract with an independent quality review organization to develop 
protocols to be used in EQR. In planning for the development of the 
protocols, we had to determine the level of detail to be specified in 
each of the protocols. Because States have flexibility to choose what 
aspects of quality to measure and in order to accommodate different 
methodological approaches to studying quality, we contracted for the 
development of protocols that specified activities and steps of data 
collection and analysis that would produce valid and reliable 
information. These apply regardless of the data collected or the topics 
that States choose. Protocols will be considered ``consistent'' with 
ours to the extent that they affirmatively address each element 
specified in Sec.  438.352, including the activities and steps for 
collecting data. We have revised the regulations under Sec.  438.352(c) 
to clarify that instead of following ``detailed procedures,'' the EQR-
related activities follow ``activities and steps'' specified for 
accurate, valid, and reliable data collection.
    Comment: One commenter recommended that external review be required 
every 3 years rather than on an annual basis. The commenter noted that 
the National Committee for Quality Assurance (NCQA) requires a standard 
external review every 3 years and believes that this rule and the 
protocols should not set a standard more stringent than the industry 
standard.
    Response: Section 1932(c)(2)(A)(i) of the Act clearly states that 
contracts ``shall provide for an annual (as appropriate) external 
independent review.'' We discuss later in this preamble why the 
parenthetical was not intended to modify what is otherwise an explicit 
requirement that EQR be conducted annually. An annual EQR has been a 
statutory requirement since 1986 under section 1902(a)(30)(C) of the 
Act. Pub. L. 106-113 made it clear that the provision was being 
replaced by 1932(c)(2) of the Act. We further note that the EQR 
described in this rule is very different from the accreditation review 
performed by NCQA. However, in the monitoring for compliance with the 
standards protocol that provides accreditation-like data, we only 
provide that information from a review of compliance with standards be 
generated every 3 years. This is consistent with the industry standard.
    Comment: One commenter asked for confirmation that Sec.  438.356(a) 
allows for EQR for a single MCO or PIHP to be performed by more than 
one EQRO.
    Response: We are revising proposed Sec.  438.356(a) to clarify that 
while we allow a State to contract with different EQROs to conduct EQR 
and EQR-related activities for a single MCO or PIHP, we believe and 
continue to require that the final analysis of all the information, as 
distinguished from the EQR-related activities, be performed by a single 
EQRO. This provides State flexibility to use different contractors to 
conduct different activities. Section 438.350 addresses the analysis 
and evaluation of information derived from mandatory and any optional 
activities. We believe that a single EQRO should perform this function 
to ensure that one entity receives all the available information and 
draws the overall conclusions about a particular MCO or PIHP. To 
clarify our intent to require that one EQRO perform the overall 
analysis (that is, conduct EQR) but that multiple EQROs may conduct 
EQR-related activities, we revised the language from the proposed rule 
to (1) remove the reference to ``other related activities'' in the 
definition of EQR, (2) add the reference to EQR-related activities to 
the definition of EQRO at Sec.  438.320, and (3) add the reference to 
EQR-related activities to Sec.  438.370 which provides for the 75 
percent enhanced match. We also revised Sec.  438.356(a) to clarify 
that States may only contract with one entity for EQR but may contract 
with multiple entities to conduct EQR-related activities.
    Comment: One commenter recommended the addition of language 
allowing States the option to employ alternative quality assessment and 
improvement methods approved by CMS to substitute for the EQR 
requirements. The revised language should emphasize the State's 
responsibility under section 1932(c)(1)(A) of the Act to develop and 
implement a quality assessment and performance improvement (QAPI) 
strategy that includes, but is not restricted to, EQR-related 
activities. If CMS seeks to define minimum specifications for a State's 
QAPI strategy, those specifications should be set out in a proposed 
rule and subject to public review and comment.
    Response: Our Medicaid managed care final rule outlined the 
elements of a State quality strategy, of which EQR is one element. 
States have the flexibility to determine how to ensure the quality 
strategy elements are designed and implemented. The public had the 
opportunity to review and comment on the proposed elements in the 
Medicaid managed care proposed rule published August 20, 2001 in the 
Federal Register (66 FR 43614). The EQR proposed rule addresses EQR in 
greater detail than does the managed care final rule, including what 
activities can be funded under the EQR enhanced matching rate. In this 
final rule, we describe optional EQR-related activities for which a 
State can obtain the enhanced Federal match under Sec.  438.370. We 
believe we have provided States with the flexibility to design their 
EQR to best meet State needs while at the same time ensuring, through 
the three mandatory activities, that essential quality activities are 
conducted.
    Comment: Several commenters recommended that we require that States 
coordinate their EQR with the State's quality strategy established

[[Page 3590]]

under Sec.  438.200 through Sec.  438.204 of the Medicaid managed care 
rule and that EQR evaluate compliance with standards for quality, 
timeliness, and access in Sec.  438.206 through Sec.  438.242 of the 
Medicaid managed care proposed rule.
    Response: We agree with the commenter. The Medicaid managed care 
final rule provides that an annual EQR be one element of a State's 
quality strategy. The EQR rule provides that information from a review 
of compliance with structural standards (including quality, timeliness, 
and access) be used in the EQR. Because of this we believe that the two 
rules together will require each State to coordinate its EQR with all 
other components of its State strategy.
    Comment: One commenter agreed with our interpretation of the 
statutory provision requiring an external review annually ``as 
appropriate'' as being a reference to the deemed compliance provision. 
The commenter also suggested that reasons for not conducting a review 
be expanded to include (1) when the MCO is new and there are no 
historical records and (2) when the population of the MCO is too small 
to conduct a particular study.
    Response: We disagree that newly contracting MCOs and PIHPs should 
not be subject to EQR. New MCOs and PIHPs will be required to meet 
structural standards, and we believe that information about MCO and 
PIHP compliance with these standards should be subject to EQR. We 
understand that the calculation of performance measures and the 
implementation of performance improvement projects require time to 
complete and may not be available at the time of the EQR. Therefore, 
while we acknowledge there are mandatory activities for EQR that may 
not be possible the first year of an MCO's or PIHP's operations, we do 
not agree that the MCO or PIHP should be entirely exempt from EQR. We 
also do not agree that small population size should be a reason to 
exempt an MCO or PIHP from EQR. Rather, the State, or MCO or PIHP if 
the State permits, should choose a performance improvement topic for 
which the entity has a sufficient number of enrollees to conduct a 
valid study.
    Comment: Several commenters believe that the ``as appropriate'' 
parenthetical allows CMS the discretion to interpret EQR time frames 
more broadly and to give States discretion to require EQRs less 
frequently than annually. One commenter suggested that ``as 
appropriate'' modifies the word ``annual,'' not ``review.''
    Response: We do not believe that the Congress intended for us or 
the States to have discretion to provide for reviews less frequently 
than annually. As discussed above, section 1932(c)(2) of the Act 
replaces a statutory requirement for annual review that has applied 
since 1986. There is no indication in the legislative history that the 
Congress intended to change this. To the contrary, there is a 
persuasive alternative explanation for the Congress having inserted the 
parenthetical language. Section 1932(c) of the Act, unlike section 
1902(a)(30)(C) of the Act has exemptions from the EQR requirement. 
Annual reviews for exempt entities are not appropriate.
    Comment: One commenter interpreted the parenthetical to allow 
States to conduct reviews more frequently, not less frequently. If the 
EQR identified problems, the EQRO could be authorized to conduct 
follow-up evaluations, as appropriate, to ensure progress toward 
compliance.
    Response: We do not agree with the commenter's interpretation 
because we believe that if problems are identified in the reports that 
the EQRO provides the States, the States can follow-up on any 
corrective action. Because we were not persuaded by any of the comments 
received for a different or additional interpretation of the 
parenthetical ``as appropriate,'' we are retaining in the final rule 
the interpretation that it refers to ``deemed compliance'' under 
section 1932(c)(2)(C) of the Act.

D. External Quality Review Protocols (Sec.  438.352)

    In this section, we proposed that EQR protocols must specify: (1) 
The data to be gathered, that is, the substantive areas to be covered 
by the protocol; (2) the sources of the data; (3) detailed procedures 
to be followed in collecting the data to promote its accuracy, 
validity, and reliability; (4) the proposed methods for valid analysis 
and interpretation of the data; and (5) all instructions, guidelines, 
worksheets and any other documents or tools necessary for implementing 
the protocol. At the time the proposed rule was published, the 
protocols were under development. The strategy and timeline for 
protocol development were undertaken in response to BBA language that 
directed the Secretary to ``contract with an independent quality review 
organization'' to develop the protocols. The contract procurement 
process and scope of work necessitated that the protocols be completed 
after publication of the proposed rule. On November 23, 2001, we 
published a notice in the Federal Register (66 FR 58741) announcing the 
completion of the protocols and asking for comment on their burden or 
any other aspect of the protocols. Comments received on the November 
23, 2001 Federal Register notice are addressed later in this preamble.
    In developing the protocols, we instructed our contractor to draw 
from existing protocols that have been tested for reliability and 
validity and that have been used in the public and private sectors to 
conduct reviews of the quality of MCO and PHP services, consistent with 
current industry practice. We also expressed a preference for protocols 
that are in the public domain. The principle reason for not including 
the protocols in our regulation is because quality measurement is a 
rapidly changing field. The protocols must be revised regularly to 
reflect the changing state-of-the-art in quality improvement. Protocols 
developed in the private sector for validation of performance measures 
and administration of consumer surveys are usually revised annually. 
The delays inherent in revising regulations would make it difficult to 
make frequent changes. In addition, the protocols are detailed and 
lengthy, as they provide optional worksheets and recording documents in 
addition to the required activities and steps.
    We proposed that all activities that provide information for EQR 
must be undertaken consistent with the protocols. Use of the CMS 
protocols or others consistent with ours will ensure that the conduct 
of the activities is methodologically sound, thereby maintaining a 
standard of quality for the review. However, by requiring protocols 
that are ``consistent,'' rather than ``identical,'' with those that we 
specify, we leave the States free to improve their protocols 
continuously, as the art and science of quality measurement improves.
    Comment: One commenter asked that the protocols not pose an undue 
burden on physicians, clinical, or nonclinical personnel, noting that 
many physicians contract with more than one MCO and that duplicative 
information gathering should be avoided.
    Response: EQR focuses on the MCO's and PIHP's structure and 
processes, and their ability to manage access to and provide quality 
services to Medicaid beneficiaries. The review process is not directed 
to individual physicians or other clinical or nonclinical personnel. 
However, it will be necessary for MCOs and PIHPs to request information 
from providers in order to conduct some of the activities required in 
this regulation. In recognition of the potential for burden, our 
request for proposal (RFP) to procure the development of the

[[Page 3591]]

protocols specified that, ``the protocols must be sensitive to the 
effect the burden to produce or provide additional data and information 
will have on organizations' ability to carry on their day-to-day 
operations.'' We also specified that the protocols incorporate, as much 
as feasible, the tools, techniques, and methods to assess and improve 
health care quality already in place in the private sector. As a 
result, we believe the protocols impose the minimal additional burden 
necessary to carry out the statutory requirement.
    Comment: In order to allow for parents to choose an MCO for their 
child on the basis of pediatric care, one commenter stated that the 
protocols should require that data on pediatric populations be analyzed 
apart from data on the MCO's adult population. The commenter also 
suggested that pediatricians and pediatric subspecialists have input 
into the development of the protocols.
    Response: As required by statute, the protocols were developed by 
an independent quality review organization. In the scope of work for 
that contract, we required that the organization convene a panel 
composed of (1) current EQRO contractors; (2) CMS representatives; (3) 
State Medicaid agency directors, (4) managed care directors and quality 
system managers; (5) State licensure agencies; (6) advocacy groups; (7) 
health plans; (8) accrediting agencies; and (9) other experts in the 
area of quality improvement. A number of these panel members had 
experience with child health issues. We published a notice in the 
Federal Register on November 23, 2001 announcing the completion of the 
protocols and asking for comment on their burden. At the same time, the 
protocols were also made available on our website. The protocols are a 
methodologically sound set of generic instructions that will guide the 
reviewer in assessing quality. These instructions can be used for the 
entire Medicaid population in the MCO or PIHP or, in some instances, 
can be used for subpopulations such as children who receive Medicaid 
services. Some protocols address how MCOs, PIHPs, and States can 
stratify by specific populations, such as older adults or children with 
special health care needs. In addition, we note that States currently 
use many performance measures related to care for children. We, 
therefore, do not believe it necessary for the protocols to address 
pediatric populations apart from adult populations.
    Comment: One commenter asked that we provide a definition for and 
examples of performance measures and performance improvement projects. 
One commenter agreed that we should not include the protocols in the 
proposed rule, given the dynamic state of quality evaluation and 
measurement. The commenter asked that we clarify what protocols for 
``calculating performance measures'' means, that is to clarify whether 
it refers to protocols for the development of measures, the calculation 
of performance thresholds from reported measures, or some other EQR 
function.
    Response: The definition and explanations of performance 
measurement and performance improvement projects are discussed in both 
the Medicaid managed care final rule and, in detail, in the protocols 
for calculating performance measures, validating performance measures, 
conducting performance improvement projects and validating performance 
improvement projects. In general, we refer to performance measurement 
as the calculation of the rate at which a desired event occurs. Readers 
are referred to the protocols available at http://www.cms.hhs.gov/medicaid/managedcare/mceqrhmp.asp for further discussion.
    Comment: Many commenters believed that the protocols should require 
MCOs to report on Americans with Disabilities Act (ADA) compliance 
issues for themselves and their providers to ensure that persons with 
disabilities have an opportunity to benefit from covered services that 
is equal to persons without disabilities.
    Response: Compliance with the ADA provisions is addressed in the 
Medicaid managed care final rule and in the EQR protocol entitled 
Monitoring Medicaid Managed Care Organizations (MCOs) and Prepaid 
Inpatient Health Plans (PIHPs)--a protocol for determining compliance 
with the Medicaid managed care final rule provisions. It is the State's 
responsibility to ensure that its MCOs and PIHPs comply with Federal 
laws, including ADA.
    Comment: Many commenters recommended that the sample for 
calculating performance measures, including baseline and follow-up 
measures for performance improvement projects, should be sufficient to 
look at specific measures of clinical care; and that the protocols 
should describe how reviewers will analyze the quality of care when 
data are missing. The commenters also believed that the protocols 
should require that MCOs use a common core of widely used, objective 
performance measures that are issued annually and revised as needed to 
reflect advances in performance measurement, that these measures and 
their methods of calculation be publicly available, and that they 
include measures for persons with special health care needs. The 
commenters also recommended that MCOs be required to (1) collect 
specified HEDIS measures; (2) conduct the Consumer Assessment of Health 
Plan Study (CAHPS) survey; and (3) conduct a focus study annually of 
specialized services to persons with special health care needs. The EQR 
should evaluate these measures in making findings on the quality of 
care. Finally, the commenters asked that instructions be provided on 
how to adapt the measures to FFS and PCCM settings and for those 
enrolled less than 12 months.
    Response: As stated before, the protocols are a set of 
methodologically sound generic instructions that will guide a reviewer 
in assessing quality. The protocols include instructions on proper 
sampling methodology, assessing missing data, and processes for 
analyzing data. The protocols do not specify which performance measures 
are to be used. Performance measures are chosen by the State or MCO or 
PIHP and will vary over time. The Medicaid managed care final rule 
gives us the authority to require specific performance measures and 
levels if we decide to do so in the future. The results of the EQR, 
however, will be made available to the public upon request and will 
identify the specific measures collected, the technical methods of data 
collection and analysis, and the conclusions drawn from the data.
    The BBA placed the requirement for EQR on capitated managed care 
programs, but not on FFS or PCCM settings. Therefore, we do not in this 
rule provide an explanation of how to adapt these activities to the 
FFS/PCCM environment. We do, however, encourage States to address the 
quality of care provided in these service delivery systems. Through a 
new partnership initiative with State Medicaid and State Children's 
Health Insurance Programs (SCHIP), we will be discussing how best to 
apply performance measures to these two delivery systems.
    Comment: One commenter asked that we retain the ability of State 
agencies to continue to improve the protocols as advancement occurs in 
the art and science of quality measurement. Several commenters stated 
that because the protocols may quickly become out of date because the 
field of quality improvement is constantly changing, they should not be 
promulgated as regulation. These commenters were concerned about CMS 
developing

[[Page 3592]]

detailed and lengthy protocols instead of either guidelines for States 
or streamlined protocols that specify only the basics for ensuring 
statistically sound, reliable, and valid results. One of these 
commenters stated that our intent appears to limit State flexibility 
and suggested that CMS significantly simplify the protocols to ensure 
feasibility for State agencies. This commenter also asked that CMS 
obtain State input on the draft protocols.
    Several commenters believed that CMS should require that States use 
the protocols. One commenter felt that the proposed rule allows States 
to develop their own external review protocols. This commenter asked 
CMS to mandate the use of the protocols in order to comply with section 
1932(c)(2)(A)(iii) of the Act which directs the Secretary to ``* * * 
contract with an independent quality review organization to develop 
protocols to be used in external reviews conducted * * *'' The 
commenter asserted that mandating the protocols would promote 
efficiency, lessen burden on the States, and promote the development of 
standardized data and information about services provided in Medicaid 
managed care.
    Response: This regulation provides States with the option to use 
the protocols developed by us or protocols that are consistent with our 
protocols. We believe that by allowing States to use ``consistent'' 
protocols, States will be able to improve the protocols over time as 
the state-of-the-art advances and at the same time ensure that reliable 
and valid methods are used when conducting EQR-related activities.
    The protocol documents include a discussion of the activities and 
steps necessary to soundly conduct the quality assessment function 
addressed by each protocol. In addition, each protocol includes 
guidance on how to implement the essential elements of the protocol as 
well as optional worksheets and appendices that States may use at their 
discretion. The activities and steps contained in the protocols are 
generic, relatively brief, but contain the essential components for a 
methodologically sound review that the statute envisions. Therefore, we 
believe that the protocols allow for State flexibility while ensuring 
the methodologically sound and valid EQR.
    Comment: Several commenters noted that it is difficult to determine 
the full extent of the impact of the protocols on EQR activities until 
they are published. These commenters stated that they hope the 
protocols will respect States' individuality and provide flexibility 
whenever possible to allow for tailoring of EQR activities to local 
conditions and circumstances. One commenter further stated that there 
are many clinical guidelines and protocols that are already published, 
easily available, and in current use (for example, those developed by 
the Agency for Health Care Policy and Research (AHCPR) now the Agency 
for Health Care Research and Quality (AHRQ), American Heart 
Association, etc * * *) that are not mentioned in the proposed rule.
    Another commenter stated that the protocols should be subject to 
full public scrutiny because they carry the full weight of the 
regulation. The commenter believes the protocols significantly exceed 
both the intent of the Congress in the BBA and the proper role of this 
regulation. Specifically, the commenter noted that the statute does not 
specify the activities that the protocols should address or other 
details included. The commenter was also concerned that States will 
find the 75 percent match for EQR activities a strong incentive to 
outsource this function, which the commenter believes appropriately 
rests with the government. As a result, this commenter believes that 
activities now done by the State according to locally developed 
protocols will be shifted to contract staff to be performed using 
externally derived standard protocols.
    Another commenter asked that current State practices not be totally 
dismissed and that consideration be given to the quality improvement 
system for managed care (QISMC) standards and how they can be 
incorporated into the EQR process.
    Response: We published a notice in the Federal Register on November 
23, 2001 (64 FR 58741) announcing the completion of the protocols and 
asking for comment on their burden. At that time, the protocols were 
also made available on our website. Comments on the protocols and our 
responses are incorporated in this preamble. We believe the protocols 
are generic and can be used by all States. They are not clinical 
protocols like those published by AHCPR (now AHRQ), the American Heart 
Association, and other organizations. We believe that the protocols are 
consistent with the intent of the Congress in the BBA. We also note 
that we have provided States with great flexibility to conduct all EQR-
related activities, allowing States to perform EQR-related activities 
either themselves or through the use of contractors, as long as they 
are performed consistent with our protocols. While the enhanced Federal 
financial match for EQR-related activities is not available under the 
statute if conducted by State personnel, other provisions of Medicaid 
law provide for enhanced Federal financial match for qualified medical 
activities when conducted by State staff who qualify as skilled and 
professional medical personnel.
    The protocols are based on existing protocols already in use in the 
public and private sector. The contractor used QISMC guidelines as well 
as other public and private sector protocols in developing all the 
protocols. With respect to the QISMC standards (as opposed to their 
interpretive guidelines) we note, for Medicaid, that the QISMC 
standards were superceded by the Medicaid managed care final rule. 
QISMC standards are no longer current for the Medicaid program. For 
each protocol developed, specific information can be found in the 
protocol regarding which public and private sector protocols were 
reviewed and the extent to which they were incorporated.
    Comment: One commenter was concerned that the JCAHO does not have a 
traditional background in this area and may take a different approach 
than NCQA.
    Response: The BBA specified that the protocols be developed by an 
``independent quality review organization.'' The JCAHO was selected 
through an open competitive procurement process, which required them to 
provide evidence of their experience in protocol development. In 
addition, they developed the EQR protocols using existing protocols 
widely used in the public and private sector, including protocols used 
by national accrediting organizations, and national consulting firms 
which have developed quality measurement tools for us in the past.
    Comment: One commenter asked if health plans will have to create an 
entirely different audit response to the protocols in addition to 
responding to the existing standards of NCQA and of other State 
entities.
    Response: Because the protocols were based on quality assessment 
approaches already in use by public and private quality oversight 
organizations, we believe that the methods MCOs and PIHPs use to 
respond to existing private and public sector audits will be able to be 
used to respond to EQR. In addition, the nonduplication provisions 
under Sec.  438.360 are revised in the final rule to allow States in 
certain circumstances to exempt both Medicare+Choice (M+C) 
organizations and MCOs and PIHPs meeting standards of national 
accrediting organizations approved and recognized by CMS for M+C 
deeming

[[Page 3593]]

from compliance with some structural standards.
    Comment: One commenter stated that the protocols being developed 
are, in fact, EQR-related activity protocols and that there does not 
appear to be any protocol that will guide the analysis and evaluation 
of the data and information provided by these EQR-related activities. 
This may cause the analysis and evaluation to vary due to lack of 
equivalent specifications for these processes. The commenter 
recommended that the rule more clearly define requirements for EQR and 
distinguish between EQR and EQR-related activities.
    Response: The commenter is correct that we do not provide a 
protocol for the analysis and evaluation of information provided as a 
result of the EQR activities in the aggregate. We do not believe that 
we should develop a protocol for the analysis and evaluation of all EQR 
information. The information derived from EQR activities will vary 
enormously. For instance, the variation in the types of services 
provided and the populations covered under the MCO and PIHP contract 
will impact the performance measures chosen and performance improvement 
projects to be conducted. Other activities are optional for States. The 
approach to analysis depends upon the findings of the individual EQR-
related activities and we expect these findings to be as individual as 
the MCOs and PIHPs being reviewed. Therefore, we do not believe that we 
can adequately predict all the possible variations of information that 
will be provided to an EQRO and, therefore, we do not provide for a 
protocol on how to conduct an analysis and evaluation of this 
information. We believe it is more appropriate for us to require that 
the activities that provide information for the analysis and evaluation 
be done in a methodologically sound manner. We do specify 
qualifications for EQROs and thereby believe that EQROs will have the 
skills necessary to perform qualitative and quantitative analysis of 
EQR-related information and draw proper conclusions. In addition, each 
EQRO must provide results as specified in Sec.  438.364 that include a 
technical report specifying the objectives of, methods used, 
description of data obtained, and conclusions drawn from the EQR.
    Comment: Many commenters were concerned that there has been no 
public review process for the protocols and that the meetings of the 
expert panel have been closed to the public. The commenters recommended 
that the public have the opportunity to review and comment on the draft 
protocols, that the protocols be issued annually, and the public have 
the opportunity to comment on any changes to the protocols. The 
commenters also stated that the protocols should be made publicly 
available on the CMS website. Several commenters asked that we provide 
an opportunity for interested parties and the public to comment on the 
protocols. They noted that providing the opportunity for all affected 
entities to review and provide comment on the protocols before they are 
finalized will allow for a better quality product and lend credibility 
to the protocols. One of the commenters further noted that even though 
CMS convened an expert panel to review the protocols as they were being 
developed, consumer participation was very limited.
    Response: As stated earlier, on November 23, 2001, we published a 
notice in the Federal Register announcing the completion of the 
protocols and requesting comment on their burden or on any other aspect 
of the protocols. Comments on that notice and our responses to those 
comments are incorporated into this preamble. We will be publishing a 
notice in the Federal Register every 3 years on the protocols as 
required by the Paperwork Reduction Act. This notice will provide the 
opportunity for the public to comment on the burden or any other aspect 
of the protocols. The protocols are available to the public on the CMS 
Web site at http://www.cms.hhs.gov/medicaid/managedcare/mceqrhmp.asp.
    Comment: One commenter requested that in developing the protocols, 
JCAHO take into consideration that some factors that affect MCO 
performance are not within the control of the MCO, such as instability 
in eligibility status and changes in the characteristics of the 
enrolled Medicaid population.
    Response: We agree that measuring performance on the Medicaid 
population needs to take into account issues such as changes in 
eligibility status. The protocol on performance measures recognizes 
those issues.
    Comment: Because of the length of the protocols and the need to 
change them on an ongoing basis, one commenter requested that we 
clarify that the protocols be issued as guidelines rather than 
requirements and that we clarify the flexibility States will have in 
implementing them.
    Response: Section 1932(c)(2)(A)(ii) of the BBA requires that 
protocols be used in the conduct of EQR activities. We provide States 
the option to use our protocols or protocols consistent with those we 
develop.

E. Qualifications of External Quality Review Organizations (Sec.  
438.354)

    Section 438.354 of the proposed rule set forth the requirements 
that an entity would be required to meet in order to qualify as an EQRO 
under the new BBA external review provisions in section 1932(c)(2) of 
the Act. The proposed rule did not specify categories of entities that 
would be qualified to perform EQR under section 1932(c)(2) of the Act. 
This is a departure from the existing external review requirement in 
section 1902(a)(30)(C) of the Act (which will no longer be in effect 
when these final regulations are implemented), under which only certain 
entities could perform external review. (These entities were: (1) A 
``quality improvement organization'' (QIO) that contracts with Medicare 
to perform review (QIOs were formerly known as quality control peer 
review organizations, or ``PROs''); (2) an entity that meets the 
requirements to contract with Medicare as a QIO; and (3) a private 
accreditation body. Only contracts with the first two categories were 
eligible for a 75 percent matching rate under the pre-BBA rules.)
    Under proposed Sec.  438.354, in order to qualify, entities would 
be required to meet specified competence and independence standards. We 
proposed two tests of independence. Under the first proposed test, the 
EQRO and any subcontractors would have to be independent from the State 
Medicaid agency and from any MCO or PHP they review. Second, the 
relationship between the MCO/PHP and the EQRO could not involve any 
potential conflicts of interest. We specifically requested comments on 
(1) how better to identify situations that create conflict of interest; 
(2) the proposal to allow State entities to qualify as EQROs; and (3) 
our decision in the proposed rule to apply the ``independence'' 
requirement to subcontractors as well as contractors.
    We also proposed that EQROs be selected by State agencies through 
an open, competitive procurement process. As noted in the preamble to 
the proposed rule, CMS would not, under our proposal, approve EQR 
contracts. However, contracts entered into by the States would be 
subject to review to ensure that, as a condition for FFP at the 75 
percent rate, the State agency followed all applicable procedures and 
criteria. This proposed procedure is consistent with current practice, 
which is for State agencies to use competitive procurements to select 
EQROs that perform review under section 1902(a)(30)(C) of the Act. It 
is also standard practice for our regional office staff to monitor 
implementation of

[[Page 3594]]

Medicaid managed care initiatives. For EQR, regional office staff may 
review the State's most recent RFP for external review services, the 
EQR contract, or the EQR reports.
    Comment: One commenter asked that a review of the current EQR 
process under section 1902(a)(30)(C) of the Act be performed by an 
independent review body to assist the Secretary in deciding whether 
current contractors are performing adequately.
    Response: Section 1932(c)(2)(A)(ii) of the Act clearly instructed 
us, in consultation with States, to establish a method to identify 
entities qualified to conduct EQR. We chose to pursue a method that 
would allow States to have access to the greatest number of entities 
with the qualifications necessary to perform EQR and EQR-related 
activities. Therefore, we did not limit ourselves to a review of 
current contractors permitted to perform review under section 
1902(a)(30)(C) of the Act, but attempted to discern all types of 
contractors that States have found capable of performing EQR-related 
activities. We believe this will provide States with much needed 
flexibility to promote greater competition and improvement among 
potential EQR contractors.
    Comment: One commenter supported the provisions in the proposed 
rule that allowed for a variety of organizations to serve as an EQRO, 
but cautioned that EQRO criteria should include an unbiased approach to 
managed care. The commenter expressed concern that an anti-managed care 
organization could be awarded the contract, and that this would 
adversely affect the organization's ability to objectively make an 
assessment of MCO strengths and weaknesses and making recommendations 
for improvement.
    Response: A State may contract with any entity to conduct EQR as 
long as the entity meets the competency and independence criteria. EQR 
is an important component of a State's quality strategy, and we trust 
that States will select entities to conduct EQR that will perform 
objective reviews.
    Comment: Many commenters supported this provision because it 
provides States with more flexibility to contract with a range of 
organizations while still obtaining the 75 percent matching rate 
currently limited to contracts with QIOs, and entities that meet the 
requirements to contract as QIOs. Several of these commenters 
specifically supported the competence and independence standards 
proposed. One commenter agreed that the regulation should require 
organizational qualifications.
    One commenter, however, found the requirements vaguely defined, and 
recommended that we stipulate additional requirements, such as proper 
licensure or certification from accrediting organizations for 
performance of validation of performance measures and surveys. Another 
commenter expressed concern that the proposed competency criteria would 
encourage the use of entities that are less qualified than the QIOs 
with which most States currently contract. The commenter believed that 
QIOs as nonprofit organizations, were independent, objective, and had 
access to needed physicians and experience in quality improvement. The 
commenter recommended that Sec.  438.354(b)(1) be revised to read, 
``require an organization to have staff with appropriate credentials 
and demonstrated experience.''
    Response: The BBA required us to work in consultation with States 
to establish a method for the identification of entities qualified to 
conduct EQR. We believe that had the Congress desired to retain the 
three categories of entities allowed to perform EQR under section 
1902(a)(30)(C) of the Act, it would have done so. Similarly, the 
Congress could have easily stated that only QIOs should perform EQR. 
The Congress chose neither of these approaches, but instead asked us to 
establish a method to identify qualified entities. We believe that the 
Congress chose to respond to States' frequently stated desires to have 
a greater range of organizations with which to contract. Therefore, 
under the auspices of the National Academy for State Health Policy 
(NASHP), we worked with States, consumer advocates, and other 
stakeholders to provide us with their recommendations on a methodology 
to identify qualified entities. Many commenters strongly supported the 
competency provisions we proposed under Sec.  438.354(b). Therefore, 
the final rule retains these requirements from the proposed rule. We 
leave it up to States to determine if they would like to impose 
additional requirements such as certified vendors. We agree that 
demonstrated experience should be required of an EQRO, and in response 
to this comment, we have changed Sec.  438.354 (b)(1) to require staff 
with demonstrated experience.
    We also made some revisions to proposed Sec.  438.354(a) to clarify 
that these provisions apply to those entities a State contracts with as 
an ``EQRO,'' regardless of whether the EQRO performs EQR or specific 
EQR-related activities.
    Comment: One commenter felt that the proposed conflict of interest 
requirements failed to recognize that since the State contracts with 
the EQRO, the EQRO would be reluctant to tell the State what it may not 
want to hear. The commenter recommended having the EQRO funded by an 
external Federal agency, such as AHRQ (formerly AHCPR), or to require 
or create financial incentives to have the State report on comparable 
performance measures for all MCOs licensed in the State.
    Response: Section 1932(c)(2) of the Act explicitly requires States 
that contract with Medicaid MCOs to provide for an EQR of each MCO, and 
provides for an enhanced Federal match rate for this review. We believe 
that it is clear that the Congress intended that States share the costs 
of EQR, and be the contracting party. We do not agree with the 
commenter's assumption that the State will not want to be informed if 
an MCO or PIHP is not performing adequately. We believe the provisions 
in this rule will encourage States to use EQROs to conduct numerous 
quality activities, both because of the flexibility that the rule 
provides to States, and because of the availability of the 75 percent 
enhanced match for these activities without regard to whether the 
entity performing review is a QIO or meets the requirements to contract 
as a QIO.
    Comment: One commenter requested that EQROs be required to include 
clinical staff with pediatric training in order to be qualified to 
review a Medicaid MCO. One commenter recommended that the entity be 
required to have staff with knowledge of section 504 of the 
Rehabilitation Act of 1973, and of titles II and III of the ADA, based 
on the commenter's research suggesting that individuals who have 
mobility impairments routinely encounter physical barriers to care. The 
commenter's research also indicated that access to preventive care was 
significantly lower for individuals who use wheelchairs, and few PHPs 
know which of their clinicians are accessible to patients with mobility 
or sensory impairments.
    Response: We do not agree that it is necessary to include specific 
requirements for EQROs to have clinical staff with pediatric training 
in order to qualify to review an MCO or PIHP. Section 438.354(b)(3) 
requires that the organization have the clinical skills necessary to 
carry out the EQR activity, which we believe requires that the EQRO or 
its subcontractor have the necessary training. We also do not agree 
with the commenter's suggestion that we specifically require an entity 
to have staff with knowledge of the Rehabilitation Act or the ADA. 
While

[[Page 3595]]

MCOs and PIPHs are required to comply with these laws, there are 
separate enforcement mechanisms for ensuring compliance with their 
provisions. We note that it is the responsibility of an EQRO to assess 
the MCO's or PIHP's ability to provide access to services in a timely 
manner. If this is accomplished for all enrollees, this would, in 
effect, constitute compliance with these laws. Through its review of 
compliance with State-established structural standards, as required in 
Sec.  438.358(b)(3) of the final rule, the EQRO must ensure that 
Medicaid beneficiaries, including those who are disabled, do not 
encounter barriers to care.
    Comment: One commenter suggested modifying proposed Sec.  
438.354(b)(1)(iii) to read ``* * * include quality assessment and 
improvement technologies and methods.''
    Response: We agree with the commenter's suggestion that the word 
``methods'' be used and believe that this term already encompasses 
technologies that may be employed by the State as a method for 
assessing and improving quality. Accordingly, in response to this 
comment, we are revising Sec.  438.354(b)(1)(iii) to use the word 
``methods.''
    Comment: One commenter supported our proposal to allow State 
agencies to qualify as EQROs in certain situations. Another commenter 
believed it would also be appropriate for the State HMO licensing 
organization to be eligible to be an EQRO. Conversely, one commenter 
felt that EQROs should be independent of most State agencies, 
particularly Medicaid purchasing or managed care licensing authorities. 
Another commenter believed that it was extremely important that the 
definition of independence be explicit for State Medicaid agencies, and 
that CMS's regional offices should review determinations as to the 
independence to make sure that true independence is obtained. This was 
based on concern over what the commenter saw as an inherent conflict of 
interest permitted under our proposed rule. In the commenter's view, 
this conflict arises from the fact that State agencies, departments, 
and universities are ultimately accountable to State legislatures and 
the Governor who act on purchasing decisions made by the State Medicaid 
agency, and who appoint members to boards of these entities. One 
commenter expressed the view that no State agency is truly independent 
and recommended prohibiting State entities from serving as EQROs.
    Response: Section 1932(c)(2) of the Act requires that a State 
contract with an independent organization in order to get the enhanced 
75 percent FFP for EQR. The expert panel composed of State 
representatives, advocacy organizations, and other stakeholders that 
was convened under the auspices of the NASHP recommended that we allow 
State agencies to qualify under certain circumstances as EQROs. Because 
we agree with this recommendation and believe it to be reasonable with 
the safeguards on independence we have in place, the final rule retains 
the independence requirements that permit State Agencies under certain 
circumstances to qualify as EQROs. We note that we have received only a 
few comments opposing our proposal to let State entities qualify as 
EQROs. CMS regional office staff will assess the EQRO contracts to 
ensure compliance with the provisions of this rule as part of regular 
monitoring reviews.
    Comment: One commenter did not agree with the requirement that a 
State entity be governed by a board or similar body, the majority of 
whose members are not government employees, in order to qualify as an 
EQRO. The commenter believed that State universities should be 
permitted to be EQROs because they can produce high quality work for 
significantly less cost than QIOs.
    Response: We understand that the requirement will limit the number 
of State entities that can qualify as EQROs, including some State 
universities. We took this recommendation from the expert panel 
convened under the auspices of the NASHP. This panel included State 
licensure and Medicaid representatives. We are aware that several 
States have State entities that meet the criteria set forth in the 
proposed rule. We have received minimal comments opposing this 
provision. We conclude that this is a feasible arrangement, and think 
that the provisions related to the governing board are appropriate and 
necessary in order to fulfill a requirement for meaningful 
independence. We also believe it represents a reasonable compromise 
between banning State entities altogether, and allowing any entity to 
serve as an EQRO. Therefore, the final rule retains the governing board 
provision.
    Comment: One commenter representing a Medicaid program not 
operating in the continental United States felt that the proposed 
independence criteria would have the effect of precluding all of its 
governmental procurement possibilities related to EQR. The commenter 
recommended that the independence criteria be waived, or that 
implementation be postponed, due to the financial burden the commenter 
believed that the rule would impose on it because it would have to 
contract with EQROs in the continental USA.
    Response: The statute requires that the EQRO be an independent 
entity. Consistent with the interpretation of ``independence'' under 
the existing external review requirement in section 1902(a)(30)(C) of 
the Act, we interpret this to mean independent from both the MCO/PIHP 
and from the State. Thus, it is not clear how this final rule would 
create a financial burden by referring a contract with an outside 
entity, since this is already required. We do not agree that exceptions 
should be made based on a Medicaid program's ability to contract with 
an EQRO locally. We recognize that many State agencies, departments, 
and universities do not meet these criteria. However, as noted above, 
several States do have State entities that meet the independence 
criteria. We also note that this regulation provides more flexibility 
than in the past for a variety of organizations to qualify as EQROs.
    Comment: One commenter disagreed with our proposal to apply the 
independence requirement to subcontracts, suggesting that this would 
result in States being unable to take advantage of the experience of 
nationally renowned experts affiliated with academic health centers 
that have ownership interests in MCOs that serve Medicaid 
beneficiaries. In contrast, one commenter endorsed applying 
independence criteria to EQRO subcontractors as balanced and 
reasonable.
    Response: The independence provisions are broad enough to allow for 
a variety of organizations to qualify as EQROs and a variety of experts 
to subcontract with EQROs. In formulating the provisions, we sought 
balance between providing flexibility to States to choose from numerous 
qualified entities, and ensuring that entities were sufficiently 
independent from the State and the MCOs and PIHPs. We realize these 
requirements will limit some contracting opportunities when experts or 
the organizations for which they work do not meet the independence 
criteria.
    Comment: Many commenters agreed with the expert panel 
recommendation that the EQRO should not share management or corporate 
board membership with the MCO it reviews. The commenters also suggested 
that the individuals employed by the EQRO or subcontracting with the 
EQRO should be free of any potential conflicts of interest with the MCO 
that they review.

[[Page 3596]]

    Response: In the preamble of the proposed rule, we explained that 
we did not solely rely upon the recommendation that an EQRO should not 
share management or corporate board membership with the MCO it reviews, 
because we do not think this criterion is stringent enough to ensure 
against conflict of interest. Therefore, we incorporated in Sec.  
438.354(c)(3)(i), the concepts of ``control'' in 48 CFR 19.101, which 
effectively preclude affiliation between the EQRO and the MCO/PIHP 
under review. Specifically, this means that there can be no control 
through common management (which includes interlocking management, 
common facilities, and newly organized concerns) as well as through 
stock ownership, stock options and convertible debentures, voting 
trusts, and contractual relationships (which includes joint ventures, 
that is, procurement and property sale assistance and franchise and 
license agreements). We retain this provision in our final rule. In 
order to provide further clarification in Sec.  438.354(c)(3)(i) of the 
final rule (Sec.  438.354(c)(3) of the proposed rule), we now specify 
the different types of control addressed in Sec.  19.101. In 
determining whether this type of control exists, the details in Sec.  
19.101 under each category would apply.
    Comment: Several commenters recommended strengthening the 
requirements for EQRO independence from MCOs by revising Sec.  
438.354(c)(3) to read as follows: ``A private entity may not (1) have 
managed care licensing authority, including the authority to certify 
managed care plans in compliance with standards that serve as the basis 
for deemed certification with Federal or State regulatory standards; 
(2) deliver any health care or related services to Medicaid recipients 
for which it is paid by the Medicaid State agency or by a managed care 
plan. Related services include enrollment services, grievance 
resolution, external review of health care coverage decisions, or other 
similar activities; (3) conduct, on the State's behalf, any other 
ongoing Medicaid program operations related to oversight of the quality 
of MCO services; and (4) have financial interest that would prevent it 
from exercising independent judgement when engaging in EQRO 
activities.'' The commenters also suggested adding a new Sec.  
438.354(c)(4) providing that ``a private entity must be governed by a 
board or similar body, the majority of whose members are not MCO 
employees.'' Another commenter did not agree with the provision that 
prohibits an organization from performing EQR if it also conducts 
ongoing Medicaid program operations related to quality, arguing it 
could be less expensive to use a single contractor to perform multiple 
functions. One of the commenters found the definition of control in 48 
CFR 19.101 a useful concept, but felt that it has little relevance to 
the potential organizational relationships between EQROs and MCOs in 
the Medicaid program.
    Response: The independence criteria set forth in the proposed rule 
did not address those private organizations that provide health care 
services to Medicaid beneficiaries or that conduct ongoing Medicaid 
program operations related to quality. We agree with the commenters 
that organizations performing these functions have a conflict of 
interest. Therefore, in response to this comment, we are revising Sec.  
438.354(c)(3)(ii) in this final rule to preclude private organizations, 
as well as State entities, that provide health care services to 
Medicaid beneficiaries from qualifying as EQROs. We also are revising 
Sec.  438.354(c)(3)(iii) to preclude private organizations as well as 
State entities, that conduct ongoing Medicaid managed care operations 
related to quality from qualifying as EQROs. We narrow the scope of 
this provision from entities that conduct program operations to 
entities that conduct managed care related operations in order to allow 
States to contract with entities that conduct quality activities for 
the States such as FFS medical and utilization review activities. We 
agree with the last commenter who agrees that it will be more efficient 
for States to use a single contractor to perform multiple functions; 
therefore, we intend to allow entities that conduct limited quality 
activities such as providing technical assistance to States in the 
collection of encounter data or who assist the State in other quality 
improvement areas to qualify as an EQRO. These activities would not be 
considered ongoing operations conducted on behalf of the State.
    We do not permit an entity to qualify as an EQRO if that entity 
conducts activities that State staff would otherwise conduct in 
Medicaid managed care program operations related to quality oversight. 
As an example, a State university or consulting firm that designs and 
implements or has significant responsibility for the State's Medicaid 
managed care program operations would not qualify as independent.
    We do not agree with the commenter who recommended that the 
independence provisions should preclude any organization from being an 
EQRO that has the authority to certify managed care plans in compliance 
with standards that serve as the basis for deemed certification with 
Federal or State regulatory standards. These organizations, while they 
may provide services under contract to a State, follow their own 
independently set standards and procedures. We believe that States 
should be permitted to contract with these organizations to consolidate 
review processes. This is consistent with congressional intent as 
indicated by the nonduplication and deemed compliance provisions in 
sections 1932(c)(2)(B) and (C) of the Act.
    As stated above, we agree with the commenters' suggestions to 
revise the independence criteria as it applies to private organizations 
that deliver health care services to Medicaid beneficiaries or who, on 
behalf of the State, conduct Medicaid managed care program operations 
related to quality. However, we do not agree with the commenters' 
suggestions to add to this provision health care-related services such 
as enrollment services, grievance resolution, and review of health care 
coverage decisions. We leave it to the States to determine if health 
care-related services are Medicaid managed care program operations 
related to quality, in which case the organizations would be precluded 
from qualifying as an EQRO. In addition, States have the flexibility to 
adopt a more strict standard for ``independence'' if they wish and to 
deny entities that provide any health care-related services from 
contracting as an EQRO.
    We agree with the commenters' suggestions that the final regulation 
include a provision to prohibit an EQRO from having a financial 
interest that would prevent it from exercising independent judgement 
when engaging in EQRO activities. The types of ``control'' addressed in 
48 CFR 19.101 address financial relationships involving such things as 
stock options and convertible debentures. To be consistent with other 
CMS regulations, however, and in order to respond to this comment, we 
believe the financial relationship between organizations must be 
addressed in the conflict of interest requirements. Therefore, we 
revised Sec.  438.354(c)(3)(iv) to address direct and indirect 
financial relationships. We also have added a definition for financial 
relationships under Sec.  438.320.
    We believe the language in proposed Sec.  438.354(c)(2) addresses 
the suggestion by one commenter that we add a provision requiring a 
private entity to be governed by a board or similar body, the majority 
of whose members are not

[[Page 3597]]

MCO employees. By referencing 48 CFR 19.101, specifically Sec.  
19.101(f)(1), a concern is considered controlling through interlocking 
management if officers, directors, employees, or principal stockholders 
serve as a working majority of the board of directors or officers of 
another concern. As noted above, to provide clarification, the final 
rule under Sec.  438.354(c)(3)(i) (Sec.  438.354(c)(3) of the proposed 
rule) specifies the elements that constitute control of one entity over 
another as those in 48 CFR 19.101.
    Comment: Several commenters expressed support for our independence 
requirements. One commenter supported our proposal to allow States to 
contract with entities that possess the necessary skill and expertise 
to conduct the mandatory and optional EQR activities, but suggested 
that we query State agencies for specific citations or contract 
language that they have used to define independence, or for concrete 
examples of situations that may create conflicts of interest. The 
commenter also suggested that we consider delineating specific 
competence standards for each of the mandatory activities. One 
commenter agreed that it is critical for CMS to establish a set of 
criteria to which States must adhere when selecting EQROs.
    Response: At the expert panel meeting convened under the auspices 
of the NASHP, we asked the panel for recommendations on how to define 
conflict of interest. This panel included State representatives as well 
as representatives from advocacy organizations and other stakeholders. 
The expert panel recommended that independence be established by 
requiring the disclosure of any ownership interest of greater than 5 
percent of the entity seeking to become an EQRO. As was discussed in 
the proposed rule, we believe this ``disclosure of ownership'' 
requirement is inadequate to ensure independence, first, because is 
does not preclude an entity from being an EQRO but only requires 
disclosure of the financial interest, and second, because there may be 
other types of conflicts such as interlocking management, common 
facilities, and so forth. Moreover, in the proposed rule, we requested 
comments on how better to identify situations that create conflict of 
interest. As noted above, we made some changes based on comments we 
received.
    We do not believe that it is necessary for us to revise the 
competency requirements to address each EQR activity. The criteria 
outlined in the proposed rule were intentionally broad to provide 
States with the flexibility to contract with one or multiple entities 
that have the skills necessary to conduct the particular activity/
activities under contract. For example, if a State wants to have one of 
its EQROs conduct only encounter data validation, to meet the 
requirement under Sec.  438.354(b)(3), the EQRO would not need to 
possess the clinical skills but would need the ``nonclinical skills'' 
in its organization (or through a subcontract) to conduct encounter 
validation.
    Comment: A commenter believed that the proposed rule did not make 
clear who, specifically, would be responsible for designating an entity 
as an EQRO. The commenter recommended that this responsibility rest in 
our Office of Clinical Standards and Quality, as it already has 
oversight responsibility for Medicare's Health Care Quality Program.
    Response: Under this rule, States are required to select and 
thereby designate EQROs through an open, competitive procurement 
process. CMS will not be designating EQROs, as it currently does in the 
case of QIOs and entities claiming that they meet the standards to 
contract as a QIO. When monitoring State Medicaid managed care 
programs, CMS regional office staff have the opportunity to review 
RFPs, contracts, and EQR results to ensure compliance with the EQR 
provisions.

F. State Contract Options (Sec.  438.356)

    This section set forth proposed requirements State agencies would 
be required to follow, and options that they would have selecting 
EQROs. We proposed that State agencies may contract with more than one 
EQRO. The final rule in Sec.  438.356 (a)(1) and (a)(2) reflects 
clarifications made to the provisions based on comments discussed in an 
earlier section of the preamble.
    We also proposed that each EQRO be permitted to use subcontractors. 
EQROs that use subcontractors are accountable for, and required to 
oversee, all EQR activities performed by the subcontractors. In 
addition, we proposed that each EQRO be required to meet the competency 
requirements, and each EQRO and EQRO subcontractor be required to meet 
the independence requirement. We also proposed that State agencies 
follow an open competitive procurement process that is in accordance 
with State law and regulation and consistent with 45 CFR part 74, as it 
applies to State procurement of Medicaid services.
    Comment: Several commenters supported the language in Sec.  438.356 
as proposed. One commenter specifically agreed that all subcontractors 
should be required to meet the test of independence, and that the 
contract must be procured through a competitive bid process.
    Response: We appreciate the commenter's support for the provisions, 
and have retained them in the final rule.
    Comment: One commenter believed that a competitive bidding process 
was the most appropriate way for States to secure efficient cost-
effective reviews.
    Response: We agree that competitive bidding provides the best means 
to select a qualified contractor at the best price, and we retain the 
requirement for competitive procurement of EQROs in the final rule.
    Comment: One commenter asked us to clarify whether the State 
Medicaid agency could contract directly with a State organization 
without using a competitive procurement process if the State 
organization otherwise meets the standard of being ``independent,'' and 
meets the requirements of a qualified EQR.
    Response: The Department of Health and Human Services has 
regulations governing the extent to which States are required to 
competitively procure contracts. Those regulations apply to EQRO 
contract as cited under Sec.  438.356(e).

G. Activities Related to External Quality Review (Sec.  438.358)

    Section 438.358 proposed a requirement that EQR utilize information 
obtained from specified mandatory activities that must be performed by 
the State agency, a State agent, or the EQRO. Proposed Sec.  438.358 
also identified optional activities that the State agency or its agent 
may perform, or have the EQRO perform, to produce additional 
information for use in EQR. The mandatory activities are consistent 
with the requirements set forth in the Medicaid managed care final 
rule. The optional activities were not included in that rule. They are, 
however, activities that States have had their EQR contractors perform 
in the past.
    We proposed that each year, the EQRO must use information obtained 
from the validation of performance improvement projects performed that 
year, and the validation of performance measures reported that year, by 
the MCO. To be consistent with the private sector, however, we proposed 
that information used by the EQRO from a review of MCO and PHP 
compliance with State structural and operational standards be from the 
most recent review performed within the previous 3 years.

[[Page 3598]]

    Proposed Sec.  438.358 also would allow States to have their EQROs 
provide technical guidance to groups of MCOs and PHPs to assist them in 
conducting the mandatory and optional EQR-related activities.
    Comment: One commenter requested that States be required to provide 
technical support to MCOs to ensure that pediatric measures are 
implemented. The commenter also expressed a concern that the proposed 
EQR regulations did not separately address children with special health 
care needs, noting that it was critical that CMS require State Medicaid 
managed care programs to provide adequate protections and 
considerations for these children.
    Response: States have the flexibility to provide technical support 
to MCOs and PIHPs on pediatric measures as well as generic measures, 
preventive care measures, measures for disabled adults, or any other 
measures. This rule does not require this technical support, however, 
because we do not believe that it would be necessary in all cases.
    With respect to special needs children, this regulation implements 
the BBA EQR provisions by specifying who is qualified to conduct EQR 
and what information should be included in such a review. The Medicaid 
managed care final rule requires States to have quality strategies that 
must include procedures that assess the quality and appropriateness of 
services provided to all Medicaid enrollees under MCO and PIHP 
contracts. This includes children with special health care needs. The 
EQR will evaluate activities undertaken by MCOs and PIHPs in accordance 
with the State strategies. States can elect to have their MCOs and 
PIHPs determine what measures to collect or States can require MCOs and 
PIHPs to collect specified measures appropriate to the populations 
served.
    Comment: One commenter strongly recommended that these regulations 
mandate that States require MCOs to develop and administer a provider 
satisfaction survey. The commenter thought this would allow the MCOs to 
use the results of the surveys to identify additional approaches to 
enhance quality of care. It also would allow States to identify MCOs 
that may be poised to experience a rapid withdrawal of providers, which 
could place beneficiaries at risk of having difficulty accessing care, 
or otherwise disrupt their medical home. Another commenter felt that 
the validation of consumer or provider surveys would be difficult. This 
commenter asked whether we were proposing that EQROs contact 
respondents to ask them if the answers that were recorded were the 
answers given.
    Response: This rule does not require that States have their MCOs 
and PIHPs develop or administer consumer or provider surveys. It does, 
however, allow States to have their EQRO administer or validate a 
consumer or provider survey, and receive the 75 percent enhanced match 
for this activity as long as the EQR survey protocol or a consistent 
protocol to the one we developed is used. The EQR survey protocol does 
not require that respondents be contacted to validate survey responses. 
We agree that this would be costly and burdensome. The survey protocols 
outlines generic steps that must be followed to ensure reliable and 
valid methodological approaches to administer and validate surveys.
    Comment: One commenter recommended that we require that EQROs 
measure and report the participation of pediatricians, pediatric 
medical subspecialists, and pediatric surgical specialists when 
conducting activities related to the establishment of provider 
networks.
    Response: EQRO reviews for compliance with structural and 
operational standards will include a review of the delivery network. 
The review will ensure, consistent with the Medicaid managed care final 
rule, that MCOs and PIHPs maintain and monitor a network of appropriate 
providers to furnish services covered under the contract and that they 
consider the anticipated Medicaid enrollment with particular attention 
to the needs of enrolled children; the expected utilization of 
services; and the geographic location of providers and enrollees. When 
developing and maintaining their provider network, MCOs and PIHPs will 
also need to consider the characteristics and health care needs of 
enrollees.
    Comment: One commenter believed that while it arguably was 
reasonable to require external auditing of broad, publicly disclosed 
quality performance measures, the same mandate should not be imposed on 
other quality improvement data such as the findings of focused clinical 
studies. In this commenter's view, these types of data are intended to 
promote MCO self-assessment and stimulate quality improvement 
activities, and should not be subject to an external audit.
    Response: We do not agree with the commenter that the findings of 
focused studies or other quality improvement projects should not be 
subject to an EQR. Our Medicaid managed care final rule requires MCOs 
and PIHPs that contract with States to provide Medicaid services to 
conduct performance improvement projects, calculate performance 
measures, and comply with structural and operational standards. In 
order to ensure compliance with these requirements, we believe a review 
of all these activities is essential to determine the quality, 
timeliness, and access to services provided to Medicaid beneficiaries. 
However, Sec.  438.364 requires that only the aggregated findings of 
the EQRO analysis of all information derived from the EQR activities be 
produced, and it is only this summary information that is to be made 
available to the public upon request.
    Comment: One commenter believed that it was vital to include in EQR 
a range of activities beyond ``focused studies'' and medical record 
review. This commenter felt that the mandatory activities proposed 
would require the collection and use of data from multiple sources, and 
that we may want to consider mandating the validation of primary data 
sources such as encounter data and survey data. Another commenter asked 
that focused studies be a mandatory activity, and that MCOs be required 
to show measurable improvement in them. One commenter supported our 
establishing mandatory activities as well as the optional activities 
that are eligible for the 75 percent matching rate.
    Response: We are aware of the importance of the integrity of the 
MCO's and PIHP's underlying information systems for the conduct of some 
EQR activities, and we address this issue in the protocols for review 
for compliance with structural and operational standards, performance 
measures, and encounter data. We do not include focused studies as one 
of the mandatory activities in this regulation because the Medicaid 
managed care final rule requires that MCOs and PIHPs conduct 
performance improvement projects. A performance improvement project 
begins with a focused study to select a clinical or nonclinical topic 
and measure performance in that area, but takes steps beyond a focused 
study to implement activities to improve performance. This regulation 
requires that the State include information regarding the validation of 
these studies as part of EQR.
    Comment: Several commenters were concerned that this rule 
potentially would permit EQROs to analyze and evaluate data collected 
by a party not subject to the same conflict of interest requirements as 
the EQRO. These commenters were concerned that the EQRO would be held 
accountable for the validity, accuracy, and reliability of

[[Page 3599]]

the MCOs' projects without necessarily having access to the raw data. 
One of the commenters suggested that there be continued discussions 
with the QIO community about the need for raw data files from MCOs in 
order to evaluate the performance improvement projects and performance 
measures. The commenter also felt that EQR performance measures should 
be standardized and consistent to allow comparisons among the States, 
and among the MCOs operating in more than one State. Another commenter 
recommended that the final rule require that EQR activities be carried 
out by the EQRO. If the information provided for the EQR is collected 
by the State or another agency, the commenter suggested that the EQRO 
be required to validate the data or information before analyzing it or 
forming conclusions about quality, timeliness, and access.
    Response: In order to receive the enhanced 75 percent Federal match 
provided for in section 1903(a)(3)(C)(ii) of the Act, we believe most 
States will use an EQRO to conduct the mandatory EQR-related 
activities. However, in order to provide flexibility to States to 
coordinate their quality oversight activities, we permit States or 
their agents to perform the mandatory EQR activities, and only require 
that States use an EQRO for the conduct of EQR (as defined under Sec.  
438.320) and for the production of the EQR results as specified under 
Sec.  438.364. If a State chooses not to have an EQRO conduct the 
mandatory activities, the State still needs to use, or have its 
contractor use, our protocols or protocols that are consistent with 
ours when conducting these activities. The State will also need to 
provide the EQRO with the information specified under Sec.  
438.364(a)(1)(i) through paragraph (iv) for each of the EQR-related 
activities as required in Sec.  438.350(b). We believe this last 
requirement may not have been clear in our proposed rule, and we have 
therefore provided a cross-reference to Sec.  438.364(a)(1)(i) through 
paragraph (iv) in Sec.  438.350(d) in this final rule. This 
clarification addresses the comments above by identifying the types of 
information we expect to be provided to an EQRO if the State or a 
contractor other than the EQRO is conducting the EQR-related activity. 
We also provide clarifying language in a new Sec.  438.358(a) of this 
final rule, which sets forth a general rule making clear that a State 
can conduct, or have another State contractor or the EQRO conduct, the 
mandatory and optional EQR-related activities that provide information 
for the EQR function.
    We do not agree that the EQRO must revalidate activities already 
validated by the State or another State contractor that uses our 
protocols. We believe the use of the protocols will ensure that each of 
the activities, including an assessment of the underlying data systems, 
is conducted using reliable and valid methods.
    We are not requiring standardized performance measures. In our 
Medicaid managed care final rule, we require States to require MCOs and 
PIHPs to use standard measures. The Medicaid managed care final rule 
also gives CMS the authority to prescribe standard measures in 
consultation with States and other stakeholders. Currently, States have 
the flexibility to determine which measures they will require of their 
MCOs and PIHPs. The CMS protocol for performance measures sets out a 
standard method to validate performance measures. We have also 
developed a protocol for calculating performance measures, as this is 
an optional EQR-related activity.
    Comment: One commenter believed that allowing the use of 
information obtained by the State or its agent for EQR means the 
information is not truly independent. The commenter further contended 
that the methods used by the State or its agent do not have to be 
consistent with the EQR protocols, since the State or its agent is not 
an EQRO.
    Response: Consistent with provisions at Sec.  438.350(b) and (c), 
whoever conducts the mandatory or optional EQR-related activities must 
use the protocols or methods consistent with the protocols. We have 
made this clear in the final rule.
    Comment: Several commenters noted that the activities under Sec.  
438.358 are currently in some cases conducted by the State, the county, 
or both. They added that having the EQRO perform this same activity, or 
even review these activities would be redundant and costly. One of 
these commenters suggested that we allow these activities to be done 
directly through the State or county survey process.
    Response: EQR-related activities may be conducted by the State or 
by any State contractor other than the MCO or PIHP as long as the 
activities are conducted consistent with our protocols. However, if a 
State chooses to have its EQRO conduct these activities it can obtain 
the enhanced 75 percent Federal match under section 1903(a)(3)(C)(ii) 
of the Act.
    Comment: One commenter asked that we clarify whether information 
derived from optional activities performed by other fiscal government 
agencies could be used by the EQRO.
    Response: As long as the other agency uses our protocols or methods 
consistent with the protocols, the information derived from EQR-related 
activities performed by other State agencies can be used as part of 
EQR. The State, however, would not be able to receive the enhanced 75 
percent Federal match unless the other government agency qualified as 
an EQRO, and the contract to conduct the activities was procured 
consistent with Sec.  438.356(e). We clarify in this final rule that 
the information obtained from optional EQR-related activities must be 
from information derived from optional activities conducted within the 
preceding 12 months.
    Comment: Several commenters believed that MCOs should be required 
to report on standardized performance measures for specific conditions. 
One of these commenters also recommended that MCOs be required to 
report on aggregate measures of changes in health status for all people 
who meet a definition of disability. The commenter further urged that 
the development of these measures be a priority for both quality 
assurance and reimbursement purposes.
    Response: As stated previously, the Medicaid managed care final 
rule provides States with the authority to specify what performance 
measures to require their MCOs and PIHPs to calculate and report. We 
are allowing this flexibility because State Medicaid managed care 
programs differ in the services they contract for and the populations 
served by MCOs and PIHPs. We think it is important that States be able 
to make comparisons across their contracting MCOs and PIHPs and, where 
this information is available, we require that it be provided as part 
of the EQR results as specified in Sec.  438.364(a)(4). However, while 
the Medicaid managed care final rule provides CMS with the ability to 
prescribe performance measures in consultation with States and other 
stakeholders, at this time we are not requiring the collection of 
comparative data nationwide.
    We are also not requiring that States collect health status 
information from their MCOs and PIHPs. States are free to do this if 
they choose, and an increasing number of States are assessing the 
health status of MCO and PIHP enrollees for purposes of risk adjusting 
payments, or for quality activities. This rule also allows States to 
have their EQRO administer consumer surveys and obtain an enhanced 
Federal match of 75 percent. Approximately 30 States currently 
administer consumer surveys, primarily the CAHPS survey, which collects 
health status information from the perspective of consumers.

[[Page 3600]]

    Comment: One commenter felt that the EQR-related activities were 
not clearly defined, and were limited in scope. The proposed language 
did not appear to the commenter to require the State to provide actual 
data to the EQRO, only information on the validation of the data. The 
commenter was concerned that the State could report to the EQRO that 
the data are valid, without actually providing the data itself.
    Response: We do not agree with the commenter that the EQR-related 
activities are limited in scope. The activities reflect those that 
States have used existing EQR contractors to conduct in the past. These 
activities are more fully explained in the protocols that we reference 
in this final rule. On November 23, 2001, we published a notice in the 
Federal Register announcing the completion of these protocols noting 
their availability on our website and asking for comment on the extent 
to which they impose a burden, as well as any other issues the 
commenters wished to raise. Our protocols clearly define EQR 
activities, and the steps needed to conduct these activities in a valid 
and reliable manner. As noted in the preamble of our proposed rule, the 
full content of the protocols themselves was not included in the 
proposed rule, and is not included in this final rule because the 
protocols are more detailed than appropriate for Federal regulations, 
will need to be revised as the state-of-the-art of quality improvement 
changes, and States may use other protocols as long as they are 
consistent with those we developed. The need for the EQRO to have raw 
data will depend on the activities a State chooses to have its EQRO 
perform. For the actual conduct of EQR as defined in Sec.  438.320, as 
well as the mandatory activities, access to raw data will not be 
needed. If the EQRO conducts all of the mandatory activities, it will 
be responsible for validating the methodological approach used by the 
MCO and PIHP for the conduct of performance improvement projects, and 
the calculation of performance measures. Regardless of who conducts the 
EQR-related activities, the CMS protocols, or a method consistent with 
the CMS protocols, must be used, and the information derived from the 
activity, as specified in Sec.  438.364(a)(1)(i) through paragraph 
(iv), must then be provided to the EQRO.
    Comment: One commenter did not support our decision to make 
performance improvement projects a mandatory activity, while focused 
studies are an optional activity. The commenter expressed concern that 
performance measures tend to focus on things that are easy to fix, and 
do not always provide a reliable picture of quality across a broad 
range of concerns.
    Response: As the state-of-the-art of quality assessment and 
improvement has changed, we have found it more suitable to implement 
performance improvement projects than focused studies. Focused studies 
aim to assess the quality of care provided at a point in time, whereas 
performance improvement projects, in addition to assessing a focused 
area of care at a point in time, aim to initiate an intervention to 
improve care over time. In our proposed rule, we discussed the 
limitations of solely using focused studies, without information from 
other quality activities, to assess the care provided to all enrollees 
of a State Medicaid managed care program. It is for these reasons that 
improvement projects are mandatory while focused studies are optional. 
We note, however, that States may employ focused studies and use an 
EQRO to conduct this activity, thus accessing the enhanced 75 percent 
Federal match under section 1903(a)(3)(C)(ii) of the Act.
    In this rule, we provide for a multipronged approach to quality 
improvement that uses information from three sources: (1) Determination 
of compliance with standards, (2) validation of performance improvement 
projects, and (3) validation of performance measures. We believe that 
this approach will provide for a reliable assessment of the quality, 
timeliness, and access to care provided to Medicaid beneficiaries by an 
MCO/PIHP.
    Comment: One commenter interpreted the proposed rule to prohibit 
States and EQROs from conducting focused studies, and to instead 
require States to perform comprehensive reviews of all areas of the MCO 
contracts every year. This commenter recommended that we reconsider the 
scope of annual review, suggesting that a 1 year cycle does not allow 
sufficient time to procure an EQR contract, conduct and complete EQR 
activities, and report results on the EQR as specified in this rule. 
The commenter also recommended that we allow for a multiyear rotational 
approach to quality measurement and improvement (for example, rotate 
specified performance measures, focused clinical topic reviews). One 
commenter similarly believed that 1 year was too short a period of time 
in which to conduct the activities under Sec.  438.358 (a)(1)(i) and 
(ii) of the proposed rule. This commenter suggested that this time 
period instead be left up to the State agency. Another commenter 
recommended that we require only that the information used by the EQRO 
for validation of performance improvement projects be from the most 
recent review performed within the previous 3 years, rather than 
requiring a yearly review.
    Response: Section 1932(c)(2) of the Act requires an annual external 
review. In the final rule, we require that there be three sources of 
information used in this review. First, for performance improvement 
projects, this final rule requires that there be performance 
improvement projects underway during the previous 12 months. We 
understand that an MCO or PIHP may have multiple projects underway at a 
given time, and these projects may be at various stages of 
implementation. In response to this comment, we have revised the 
language under proposed Sec.  438.358(a)(1)(i) (now Sec.  
438.358(b)(1)) to clarify that performance improvement projects need to 
be underway during the preceding 12 months, instead of having been 
completely performed during the preceding 12 months. Consistent with 
private sector practices, we therefore would allow States to use a 
multiyear rotational approach when conducting performance improvement 
projects and calculating performance measures. Second, for performance 
measures, the rule requires that one or more measures be reported 
annually. Finally, as was indicated in our proposed rule, EQR also 
needs to employ information from a review of structural and operational 
standards, conducted within the previous 3-year period.
    Comment: Many commenters suggested that the list of mandatory 
activities include an examination of reasons for disenrollment and 
termination.
    Response: Under Sec.  438.358(b)(3) of this final rule, we require 
a review of MCO and PIHP compliance with State standards, in accordance 
with the Medicaid managed care final rule. This includes standards for 
enrollment and disenrollment. The Medicaid managed care final rule 
includes standards for disenrollments requested by the beneficiary, as 
well as those requested by the MCO or PIHP. In addition, the Medicaid 
managed care final rule requires MCO and PIHP compliance with State 
standards for health information systems. As part of the health system 
provisions, we require that the State ensure that the MCO or PIHP 
information system provides information including, but not limited to, 
utilization rates, grievances, and numbers of appeals and 
disenrollments. We believe these provisions adequately address the 
commenter's concern, and

[[Page 3601]]

that no additional requirements are necessary.
    Comment: One commenter noted that there was no cross-reference in 
the proposed EQR rule to the requirements in the then proposed Medicaid 
managed care rule that required MCOs to measure performance and conduct 
performance projects, and to comply with State-mandated standards. The 
commenter suggested that we make this cross-reference to the applicable 
sections in the Medicaid managed care rule.
    Response: We have in this final rule added cross-references to the 
appropriate citations in the Medicaid managed care final rule.
    Comment: One commenter recommended that we establish a core set of 
State standards for MCOs and evaluate these during the EQR process. The 
commenter was concerned that allowing States to determine the measures 
to be collected would provide little or no comparable plan or State 
level data.
    Response: We do not agree that this rule should specify 
standardized performance measures for States or their contracting MCOs 
and PIHPs. The Medicaid managed care final rule specifies that States, 
through their contracts, must require their MCOs and PIHPs to calculate 
performance measures or submit data to the State that enables the State 
to measure MCO's or PIHP's performance. Many States currently require 
that standard performance measures be collected across MCOs. In 
addition, we believe that States will require that specified measures 
be calculated over time to enable the State to evaluate MCO and PIHP 
performance. In Sec.  438.364(a)(4), we require that the EQR results 
include comparative information, as determined appropriate by the 
State. Furthermore, Sec.  438.10(i)(2)(ii) of the Medicaid managed care 
final rule requires, for those States that provide for mandatory 
managed care under section 1932(a)(1)(A) of the Act, that the State 
provide comparative information annually. This must include, to the 
extent available, quality and performance indicators as required under 
Sec.  438.10(i)(3)(iv). In addition, the Medicaid managed care final 
rule provides that CMS may, in collaboration with States and other 
stakeholders, prescribe standard performance measures.
    Comment: One commenter asked us to clarify how proposed Sec.  
438.358(a)(1) fulfills the statutory requirement of EQR, and 
specifically how this information relates to a review of ``the quality 
outcomes and timeliness of, and access to, the items and services for 
which the managed care organization is responsible under the 
contract.''
    Response: In order to make an assessment about the quality, 
timeliness, and access to services provided by MCOs and PIHPs, there 
must be information from which an assessment can be made. Section 
1932(c)(A)(iii) of the Act required us, in coordination with the NGA, 
to contract with an independent quality review organization to develop 
protocols to be used in EQR. In order to develop protocols, we first 
needed to define EQR, as it was not defined under section 
1902(a)(30)(C) of the Act. We also needed to determine what activities 
we consider necessary or appropriate to provide information for a 
quality review. The EQR activities in Sec.  438.358(b) and (c) are 
activities that (1) the expert panel convened under the auspices of the 
NASHP recommended be included as part of EQR; (2) a survey of States by 
the Department of Health and Human Services' Office of Inspector 
General identified as quality review activities used by States; and (3) 
a survey of States by NASHP confirmed as activities most frequently 
used by States for EQR. The EQRO must develop a report, based on the 
information provided, as specified in Sec.  438.364, that includes a 
detailed assessment of each MCO's and PIHP's strength and weaknesses 
with respect to the quality, timeliness, and access to health care 
services furnished to Medicaid beneficiaries.
    Comment: A commenter noted that the rule does not clearly identify 
which entities are qualified and competent to undertake the validation 
of performance measures and performance improvement projects. In the 
commenter's view, as drafted, the rule could be interpreted as allowing 
entities other than EQROs, including the State or the MCO itself, to 
undertake these tasks. The commenter recommended that we clarify what 
types of entities can engage in validation activities and at a minimum 
require those entities to be competent and independent.
    Response: The State, an EQRO, or other State contractor can 
undertake any of the EQR-related activities. However, it is only when 
an EQRO, that meets the competency and independence criteria, conducts 
any of these activities that a State can obtain the enhanced 75 percent 
Federal match under section 1903(a)(3)(C)(ii) of the Act. Regardless of 
who conducts the activity, the CMS protocols (or other protocols 
consistent with ours) must be used to gather information for the 
mandatory and optional activities used in EQR. We did not intend to 
allow the MCO or PIHP itself to be able to conduct any EQR-related 
activities and in response to this comment we have revised Sec.  
438.358 so that it is clear that ``the agent'' must be an entity other 
than an MCO or PIHP.
    Comment: One commenter recommended that we modify the regulation to 
grant State agencies the discretion to adapt these requirements to more 
appropriately address the circumstances of small or new MCOs and PHPs. 
The commenter suggested that enrollment in some MCOs and PHPs may be 
too small for an EQRO to validate the data for performance improvement 
projects or performance measures. Similarly, for an MCO that is not yet 
operational or which has only been operating for a short amount of 
time, there may be insufficient experience to use to evaluate for 
compliance with standards.
    Response: We do not agree that we should modify the regulation to 
allow States to adapt the requirements to address small or new MCOs and 
PIHPs. If enrollment in an MCO or PIHP is small, the entire applicable 
population, as opposed to a sample, can be used when conducting 
performance improvement projects, calculating performance measures, or 
validating these activities. Regarding compliance with State standards, 
all MCOs and PIHPs that contract with a State to provide Medicaid 
services must be in compliance with the contracting requirements in the 
Medicaid managed care final rule. Regardless of when the EQR is 
conducted, MCOs and PIHPs should have procedures in place to be 
compliant with these provisions. Therefore, an assessment of compliance 
with these standards must be conducted and the findings provided to the 
EQRO to make its assessment regarding quality, timeliness, and access 
to services provided by the MCO or PIHP to Medicaid beneficiaries.
    Comment: One commenter felt that State Medicaid agency staff should 
conduct the review of MCO compliance with structural and operational 
standards, as the review requires extensive knowledge of the State 
Medicaid program, its regulations, and the MCO contract. This commenter 
believed that this requirement was duplicative of current practice and 
unnecessarily burdensome, and did not provide States needed flexibility 
to choose which activities it wants to have its EQRO conduct. The 
commenter suggested deleting this provision. Another commenter urged 
that the review of compliance with standards be an optional instead of 
mandatory activity. The commenter noted that States conduct this 
activity through

[[Page 3602]]

various means, and that mandating this be done through EQR would mean 
an increase in Federal and State funding for the EQR contract. One 
commenter believed that the proposed requirement for review of 
structural and operational standards went beyond the statute's 
reference to ``quality outcomes, and timeliness of, and access to items 
and services for which the organization is responsible under 
contract.'' This commenter recommended that we reevaluate the extent of 
this review to ensure that it is consistent with the intent of the 
statute. The commenter further noted that this review was so broad that 
it would encompass most of the areas currently reviewed by States under 
their general contract responsibilities.
    Response: States are not required to contract with an EQRO to 
conduct a review of the MCO's or PIHP's compliance with State 
structural and operational standards. A State can conduct this activity 
using the CMS protocols or protocols consistent with ours and provide 
the results of the review to the EQRO. The regular 50 percent 
administrative FFP match would be available to the State for this 
activity if it is not conducted by the EQRO. The EQRO will use this 
information in conjunction with information derived from the other two 
mandatory activities and any optional EQR-related activities conducted 
to determine quality of, timeliness of, and access to the quality of 
care provided by the MCO or PIHP. This final rule provides States with 
the flexibility to determine which activities it wants to have its EQRO 
conduct. Although we prescribe mandatory activities, which are 
consistent with the requirements set forth in the Medicaid managed care 
final rule, the State does not have to have its EQRO conduct these 
activities. A State is only required to have an EQRO conduct the 
analysis and evaluation of the information derived from the activities 
to determine if an MCO or PIHP is providing access to quality services. 
We do not believe that the scope of the mandatory activities goes 
beyond the statutory provisions under section 1932(c) of the Act which 
require States to have a quality assessment and improvement strategy 
which includes access standards, and measures to assess care, including 
grievance procedures and marketing and information standards. 
Furthermore, the statute requires that States implement monitoring 
strategies that address the quality and appropriateness of care. We, 
therefore, retain the review of MCO and PIHP compliance with State 
standards as a mandatory activity in our final rule.
    Comment: One commenter believed that the intent and usefulness of 
the proposed language in Sec.  438.358 requiring the EQR to ``use 
information'' obtained from the mandatory and optional EQR-related 
activities was unclear. The commenter recommended changing the language 
to read ``The State or the EQRO shall/must conduct'' the EQR-related 
activities.
    Response: Sections 1932(c)(2)(A)(ii) and (iii) of the Act required 
us to (1) in consultation with States, develop a method to identify 
qualified entities for the conduct of EQR, and (2) in coordination with 
the NGA, develop protocols to be used in EQR. In order for us to 
determine who was qualified to conduct EQR and for us to develop 
protocols to be used in an EQR we first needed to define EQR. Based on 
the advice of an expert panel convened under the auspices of the NASHP, 
the proposed rule, and this final rule, define EQR as the analysis and 
evaluation by an EQRO of aggregated information. Based on this 
definition, the expert panel confirmed the types of activities that 
would produce information as it relates to the quality, timeliness of, 
and access to care provided to our beneficiaries. These are the 
mandatory and optional activities found in this section of our rule. To 
provide consistency with the definition of EQR, and because we do not 
require that States contract with an EQRO to conduct these activities, 
we retain the language that an EQR must use information derived from 
the EQR-related activities in the final rule.
    Comment: Many commenters did not agree with our proposal to require 
that information be used from a review of structural standards every 3 
years, and cited the statutory language requiring ``an annual * * *'' 
review. Many commenters recommended that all activities be done 
annually, citing reasons such as the changing status of provider 
networks, and pressures to control utilization. One commenter claimed 
that we did not adequately explain our rationale for permitting the use 
of data and information that may be up to 3 years old. The commenter 
argued that given the volatility of both the managed care market place 
and State Medicaid programs, the problems identified in Medicaid 
managed care systems throughout the country, and the fact that the 
majority of beneficiaries are children, allowing the use of 3-year-old 
data was inadequate. The commenter suggested that an evaluation of 
quality, timeliness, and access to services must be timely to allow for 
effective interventions to correct the problems.
    Response: Reviews of MCO and PIHP compliance with structural and 
operational standards are very time consuming and costly. To be 
consistent with private industry standards, we proposed that 
information from the review of MCO and PIHP compliance with standards 
be from the most recent review conducted within the previous 3 years. 
Both NCQA and JCAHO perform their accreditation reviews once every 3 
years. As stated earlier, our rule takes a multipronged approach to 
quality assessment and improvement. This is one reason why we require 
the EQR to use information from a minimum of the three mandatory 
activities to render a decision regarding the quality and timeliness of 
and beneficiary access to health care services. We believe that this 
comprehensive approach addresses the commenters' concerns, and that 
annual reviews for compliance with structural standards is not 
justified.

H. Nonduplication of Mandatory Activities (Sec.  438.360)

    Proposed Sec.  438.360 provided State agencies, under certain 
circumstances, the option not to require a review of MCO or PHP 
compliance with certain structural and operational standards specified 
in proposed Sec.  438.358(a)(2) if the MCO or PHP is a certified M+C 
organization with a current Medicare contract, and has been evaluated 
and approved by us, our contractor, or certain approved accrediting 
organizations as a part of accreditation for compliance with these 
standards. The December 1, 1999 proposed rule also provided that a 
State agency under certain circumstances may similarly avoid duplicate 
reviews of all mandatory activities (listed in paragraphs (a)(1) and 
(a)(2) of proposed Sec.  438.358) for any MCO or PHP that serves only 
individuals who are eligible for both Medicare and Medicaid. Under the 
December 1, 1999 proposed rule, if the State agency exercises this 
option, each MCO and PHP must make available to the State agency all 
reports, findings, and other results of the Medicare quality review or 
the accreditation survey that is to substitute for the Medicaid review.
    Comment: Several commenters supported provisions designed to avoid 
duplication in the EQR process.
    Response: We retain the nonduplication provisions in the final rule 
while providing clarifying language on their applicability, as 
discussed in responses to comments below, in order to better explain 
our intent.

[[Page 3603]]

    Comment: Several commenters asked that the provisions in this 
section not be restricted to Medicaid MCOs that have M+C contracts. The 
commenters believe that the BBA does not restrict the nonduplication 
provision to these organizations.
    Response: We agree with the commenters that the BBA does not 
require that an M+C contract be in place in order for the 
nonduplication provisions to apply. In response to these comments, we 
have changed the final rule to allow States, under certain 
circumstances, to elect not to review structural and operational 
standards of an MCO or PIHP that has been accredited by a national 
accrediting organization approved by CMS under the procedures in 42 CFR 
422.158 as applying standards at least as stringent as Medicare, where 
the standards are comparable to those imposed by the State under Sec.  
438.204(g). The EQRO must review the reports, findings, and other 
results of the accreditation review to use in the EQR.
    Comment: Several commenters recommended that we amend our 
regulations to permit accreditation programs that address only a 
portion of the Sec.  438.358(a)(2) requirements. One commenter wanted 
us to retain the provision that allows an EQRO to use a review 
conducted by a private accrediting organization, or as part of an 
external review conducted under the Medicare program. Another commenter 
suggested that we revise Sec.  438.360(b) to allow a State to exempt an 
MCO from a review of the mandatory activities, as opposed to exempting 
the MCO from the mandatory activities.
    Response: We agree with the commenters that a State should be 
permitted to use only certain portions of a Medicare or accreditation 
review in place of a portion of a Medicaid review. As stated above, the 
final rule provides States with the option of using a Medicare or (if 
approved by CMS under Sec.  422.158) private accreditation review to 
serve as the Medicaid compliance review of any or all of the standards 
required to meet provisions under Sec.  438.204(g) as long as the MCO 
or PIHP meets the requirements of Sec.  438.360(b) or (c). Because we 
received numerous comments on the applicability of this provision, we 
have revised the language in this section to more clearly explain our 
intent to apply it to MCOs and PIHPs that have been reviewed by an 
accrediting organization approved under Sec.  422.158. We also 
clarified the regulations text to better identify the activities and 
standards to which this section applies, and what information needs to 
be provided to States and us to comply with this provision.
    Comment: One commenter did not agree with provisions in Sec.  
438.360(b)(3) or (c)(3) requiring that a State receive a copy of all 
findings pertaining to the most recent accreditation review. The 
commenter contended that standard-specific information is adequate and 
that all review materials such as noted deficiencies, corrective action 
plans, and summaries of unmet accreditation requirements are excessive 
and unnecessary. The commenter suggested that we require MCOs to 
provide the State with applicable reports, findings, and results. Many 
commenters agreed that we should require that States receive and review 
information from the Medicare review or accreditation review.
    Response: We agree that requiring all reports, findings, and other 
results of the Medicare review or accreditation review could be 
excessive. We have revised the language Sec.  438.360(b)(3) and (c)(3) 
to reflect that the reports, findings, and results provided can be 
limited to those applicable to the standards for which the Medicare or 
accreditation review or quality activities will substitute for the 
Medicaid review activities.
    Comment: One commenter asked us to clarify whether the 
nonduplication provision exempts the MCO from a review for compliance 
with standards, such as enrollee rights, maintaining a grievance 
system, or using practice guidelines. One commenter recommended that we 
allow deeming of credentialing and recredentialing requirements if the 
MCO is NCQA certified.
    Response: We provide that the State may permit the findings from 
other allowable reviews to substitute for a duplicate review of the 
structure and operations of the MCO or PIHP. Under this provision, an 
MCO or PIHP is not exempted from a review of standards under Sec.  
438.204(g). Rather, States are permitted the option of using Medicare 
reviews or accreditation findings, including a review of credentialing 
and recredentialing procedures, instead of conducting a separate (and 
potentially duplicative) review, as long as the provisions under Sec.  
438.360 are met. This would apply to information on compliance with 
standards such as the requirements set forth in proposed Sec.  
438.358(a)(2)(i) through (a)(2)(xiii) cited by the commenter.
    Comment: Many commenters agreed that external reviews need to 
validate performance measures specific to the Medicaid population in 
the case of Medicaid contracts. In contrast, one commenter recommended 
that an MCO fully accredited by a private accrediting organization 
should also be exempt from calculating performance measures (for 
example, HEDIS). The commenter believed that this would eliminate the 
need for new-capacity building or criteria to ensure consistency.
    Response: We do not agree that an accredited MCO or PIHP should be 
exempt from a validation of performance measures calculated under Sec.  
438.358(a)(1) unless it provides services to dual eligibles only. As 
stated in our December 1, 1999 proposed rule, we believe the types of 
data collected, measures calculated, and studies conducted, on the 
Medicare population would differ from those for the Medicaid population 
unless the MCO or PIHP served only dually eligible Medicare and 
Medicaid beneficiaries. We believe this argument is also valid when 
applied to the commercial population. We, therefore, retain the 
language as written in the December 1, 1999 proposed rule. We note that 
if the accrediting organization, acting as the EQRO of the State, 
validates the performance measures required of the MCO or PIHP by the 
State, the State can obtain the 75 percent match under section 
1903(a)(3)(C)(ii) of the Act for having the accrediting organization 
conduct that activity.
    Comment: One commenter recommended that we revise the regulation to 
give State agencies discretion to determine what EQR activities are 
duplicative.
    Response: We do not agree that States should have discretion to 
determine what EQR activities are duplicative. Except in the case of an 
MCO or PIHP that provides services to dual eligibles only, we limit the 
nonduplication provisions to the structure and operational standards 
reviewed under Sec.  438.358(b)(3).
    Comment: Several commenters noted that accrediting organizations 
differ in how they characterize the status conferred when MCOs meet 
their accreditation standards. For example, these commenters pointed 
out that not all accrediting organizations use the term ``full 
accreditation.'' One commenter recommended that we clarify proposed 
Sec.  438.360(b)(2)(ii) to avoid confusion regarding what accreditation 
level must be attained to meet the requirements of the paragraph. 
Another commenter asked us to clarify ``fully accredited'' and 
recommended that we negotiate with accreditors seeking to be recognized 
under this section to determine what type of accreditation would meet 
the intent of this section.

[[Page 3604]]

    Response: We understand that accrediting organizations use 
different terms to describe the extent to which MCOs or PIHPs meet 
their standards. However, in this provision of the regulation, we are 
not requiring that the MCO or PIHP achieve a certain level of 
accreditation. Rather, we are allowing States to use information 
gathered in the private accreditation process that is shared with the 
State to assess compliance. To make this more clear, in response to 
this comment, we have removed the term ``fully accredited'' from the 
regulations text. We also have revised the language of this section in 
order to make our intent more clear. We now specify that accrediting 
organizations that have been approved by us for M+C deeming under Sec.  
422.158 meet the requirements of this provision.
    Comment: Several commenters did not agree with permitting States to 
avoid mandatory activities by relying upon information gathered from a 
Medicare or private accreditation review in order to assess MCO 
compliance with structural and operational standards. Some of these 
commenters specifically strongly opposed the exemption from mandatory 
activities when an MCO has a Medicare contract. They believed that 
activities such as review for the availability of services, 
establishment of provider networks, enrollee information, 
confidentially, and use of practice guidelines all have Medicaid and 
pediatric components that would not be examined under a Medicare 
review. If an exemption is allowed, the commenters suggested that 
additional activities be required to ensure compliance in problem-prone 
or sensitive areas that reviews by Medicare or private accrediting 
organizations may not adequately address. One of the commenters 
recommended that if an MCO is being considered for the exemption, that 
there must be substantial overlap between the Medicare and Medicaid 
products in (1) geographic service area, (2) network composition and 
management, (3) quality management structures and processes, and (4) 
levels of accreditation. Many commenters suggested that unless our 
quality review or accreditation has established the quality of the 
Medicaid provider network and administrative structures, these 
activities should not be exempted under nonduplication.
    Response: The Congress clearly intended that we provide States the 
option to avoid duplicating review activities conducted for Medicare or 
by accrediting organizations. We limit the applicability of this 
provision to the mandatory activity designed to help States assess 
structural and operational standards for all MCOs and PIHPs other than 
those serving only dual eligibles. For the latter, under Sec.  438.360, 
we also permit States to use this option with respect to the validation 
of performance measures or the validation of performance improvement 
projects. We believe proposed Sec.  438.360 generally places sufficient 
parameters on States that choose to exercise this option.
    We retain the provision that permits States to use this option to 
assess compliance with standards. We note that Sec.  438.207 of the 
Medicaid managed care final rule requires that MCOs and PIHPs submit 
documentation to the State of compliance with requirements in the 
Medicaid managed care final rule that requires MCOs and PIHPs to 
maintain a network of providers that is sufficient in number, mix, and 
geographic distribution to meet the needs of the enrollees in the MCO 
or PIHP. In addition, Sec.  438.207 requires that any time there has 
been a significant change in MCO or PIHP operations that would affect 
adequate capacity, additional documentation must be submitted. We 
believe this information adequately complements any review of 
availability of services that would be conducted by Medicare or an 
accrediting organization that provides information for the EQR.
    We are concerned, however, that the wording of proposed Sec.  
438.360 has caused some confusion about the intent of this provision. 
Specifically, our words ``A State may exempt an MCO from mandatory 
activities * * *'' may be interpreted by some as exempting an MCO or 
PIHP from oversight, rather than an exemption from State Medicaid 
reviews that duplicate Medicare and private accreditation reviews. To 
clarify this, we have removed the word ``exempt'' from this provision 
in the final rule (noting also that the Congress did not use this word 
in the corresponding statutory provision) and replaced it with language 
reflecting the fact that these provisions do not exempt MCOs from 
review for compliance with structural and operational standards, but 
instead permit States to use information generated through Medicare or 
private accreditation review to assess compliance with these standards, 
in lieu of engaging in their own otherwise ``mandatory'' review 
activity.
    In addition, in response to the commenters' concerns about 
permitting States to substitute Medicare or private accreditation 
review for direct State review, we are adding a new paragraph (4) to 
Sec.  438.360(b) and (c) requiring that States identify in their 
qualities strategies those standards and activities they plan to 
monitor through the use of Medicare or private accreditation review 
data, and explain why direct State review would ``be duplicative.'' 
This will help ensure that this approach is only taken when State 
review would truly be needlessly duplicative of review already 
performed.
    Comment: One commenter was concerned that proposed Sec.  438.360 
appeared to allow the nonduplication exemptions to last indefinitely, 
and believed that it was not unusual for plan performance to vary 
significantly from year to year due to organizational changes. The 
commenter recommended that States be required to develop mechanisms to 
periodically re-evaluate MCO compliance with standards during the 
course of a 3-year period, and to re-institute a direct Medicaid agency 
review if accreditation, Medicare, or State oversight indicate 
potential quality problems.
    One of the commenters cited recent OIG studies that identified 
significant issues with accrediting bodies, and did not think that 
States should relinquish their direct MCO oversight responsibilities to 
the accreditation industry.
    Response: Neither the statutory nor conference committee language 
discussed any time limit on a State using Medicare or accreditation 
review data in its assessment of an MCO or PIHP in lieu of a direct 
Medicaid review. We believe it appropriate to allow States to make the 
determination as to whether this remains appropriate. We note that the 
new paragraph (4) that we have added to Sec.  438.360(b) and (c) 
requires that States explain and justify their use of this approach, 
and believe that it is appropriate to permit the approach to be used 
for so long as this justification remains valid. Therefore, we do not 
specify a time limit in the final rule.
    With respect to the commenter's recommendation for periodic re-
evaluation every 3 years, Sec.  438.360 requires that information 
obtained from a Medicare review or a review by an accrediting 
organization be provided to the State, which must then provide the 
information to the EQRO for use in the EQR. Because this information 
must be obtained from a review of compliance with standards conducted 
within the past 3 years, this requirement should address the changes in 
plan performance that the commenter is concerned about. Moreover, the 
Medicaid managed care final rule requires that States have a quality 
strategy that has procedures for assessing the quality and 
appropriateness of care provided to

[[Page 3605]]

Medicaid beneficiaries, and that States must regularly monitor and 
evaluate MCO and PIHP compliance with operational standards.
    As noted in earlier responses, we believe the Congress clearly 
intended States to have the option of avoiding duplicate reviews of 
MCOs that have been accredited by a national accrediting organization, 
and we accordingly allow for this in the final rule.
    Comment: One commenter recommended that we clarify that States may 
only eliminate elements of the EQR process, whether mandatory or 
optional, if components of the M+C evaluation process or private 
accreditation review are the same as or similar to those of the 
Medicaid review process. Several commenters felt that this provision 
should address two concepts: whether the standard or requirement is 
duplicative, and whether the methodology of the review is duplicative. 
One commenter asked that we clarify what we mean when we say, under 
Sec.  438.360(b)(2), that the ``* * * methodologies must be * * * 
established by the State, not CMS.'' The commenter noted that it is the 
State, not CMS, that establishes the standards for the mandatory 
activity under Sec.  438.358(a)(2) and therefore it is not clear what 
benchmark we intend to use to determine comparability.
    Response: This section of the regulation applies only to mandatory 
activities as specified in Sec.  438.358(b). Because the optional 
activities are not required, we do not address optional activities in 
the nonduplication provisions. As stated earlier, we have clarified the 
regulations text to better explain that Medicare or accreditation 
standards must be comparable to those established by the State. We have 
removed the reference to standards and review procedures needing to be 
as stringent as those established by CMS because we agree with the 
commenter, that it is the State, not CMS, that will establish standards 
to comply with Sec.  438.204 of the Medicaid managed care final rule. 
As for review methodology, the statute required that we establish 
protocols to be used in EQR. The protocols we developed include generic 
activities and steps to be followed to ensure that the EQR activities 
are conducted in a reliable and valid manner.
    Comment: One commenter asked that, because implementation of 
proposed Sec.  438.360(b)(2)(ii) would depend upon our approval and 
recognition of private accrediting organizations under Sec.  422.158 as 
having standards and review procedures as stringent as those 
established by Medicare, we move forward to make these later 
determinations so this provision can be implemented in a timely fashion 
when these regulations become final.
    Response: We have already received and approved applications for 
M+C deeming from several accrediting organizations: (1) NCQA, (2) 
JCAHO, and (3) the Accreditation Association for Ambulatory Health Care 
(AAAHC).
    Comment: One commenter was confused about the distinction between 
proposed Sec.  438.360 and proposed Sec.  438.362, and felt they were 
redundant. The commenter also objected to our provisions applying to 
dual eligibles, specifically the State's option of permitting 
information obtained from performance improvement projects and 
performance measures specific to dual eligibles to substitute for 
Medicaid specific information.
    Response: We do not agree that Sec.  438.360 and Sec.  438.362, 
which permit States to exempt an MCO or PIHP from EQR in its entirety, 
are redundant. However, we agree that proposed Sec.  438.360 was 
potentially confusing in its use of the word ``exempt.'' We have 
revised the language in Sec.  438.360 to clarify that Sec.  438.360 
allows States to use the findings of Medicare or accreditation reviews 
in place of a Medicaid review in order to avoid duplication, but does 
not exempt MCOs or PIHPs from EQR, as does Sec.  438.362 where it 
applies. We think that there is a clear distinction between Sec.  
438.360 under which analysis and evaluation of information must still 
be conducted, and Sec.  438.362 under which the MCO or PIHP is exempted 
from the EQR function. We disagree with the commenter concerning the 
appropriateness of the dual eligible provision. In the case of dual 
eligibles, Medicare review necessarily is targeted to the population 
involved. We therefore believe that Medicaid review could be 
particularly duplicative in this case.
    Comment: One commenter requested that if accreditation is to be 
used as the basis for exemption, regulations require that the MCO be 
specifically accredited with respect its Medicaid line of business, and 
that information from this Medicaid enrollee review be provided to the 
State.
    Response: We do not agree that we should limit the applicability of 
the nonduplication provisions in Sec.  438.360 to MCOs or PIHPs 
accredited specifically for their Medicaid product. Most accrediting 
organizations do not conduct separate reviews for an MCO's or PIHP's 
Medicaid product. With respect to the commenter's second point, we do 
require that the findings of the accreditation be provided to the State 
and then, in turn, to the EQRO to be used as part of the EQR.
    Comment: One commenter urged that we allow for the use of review 
findings of related ``focus studies'' of groups that Medicaid serves 
(for example, the elderly or disabled) which are conducted by other 
types of certified Medicare organizations.
    Response: As long as a focused study is conducted using a 
methodology consistent with our protocols, and the study population is 
composed of Medicaid beneficiaries, a State can have its EQRO use the 
review findings. In addition, if the organization that conducts the 
focused study is the State's EQRO, the State can obtain the 75 percent 
enhanced match for its review of these findings.
    Comment: One commenter believed that the activities under proposed 
Sec.  438.358(a)(2) are not the same regardless of the populations 
served, and specifically that there is a difference when serving 
individuals with disabilities. To address this concern, the commenter 
felt that the EQRO must be knowledgeable and sensitive to people with 
disabilities in order to effectively assess an MCO's compliance with 
standards.
    Response: As specified in Sec.  438.354, an EQRO must meet certain 
competency requirements, including having staff with knowledge of 
Medicaid beneficiaries. In addition, our Medicaid managed care final 
rule requires, under the State's quality strategy, that the State have 
procedures in place for assessing the quality and appropriateness of 
care and services furnished to enrollees with special health care 
needs. This includes individuals with disabilities.
    Comment: One commenter recommended that audits conducted by the 
State licensing organization be coordinated with the EQRO, and that the 
audit of components conducted by the State licensing organization be 
``deemed'' to have been performed by the contracted EQRO.
    Response: States can use their State licensing organization to 
assess MCO or PIHP compliance with State standards, or perform any of 
the mandatory or optional EQR-related activities identified in Sec.  
438.358. If a State wants to use this information for the EQR, the 
review must, at a minimum, use our protocols or protocols that are 
consistent with ours. Thus, there would be no reason to ``deem'' these 
reviews to have been performed by the EQRO, other than to claim the 75 
percent match that would apply if the EQRO performed these functions. 
As noted

[[Page 3606]]

above, however, if a State uses entities other than EQROs to perform 
activities, the 75 percent match rate under section 1903(a)(3)(C)(ii) 
of the Act would not be available. We hope and anticipate that States 
will coordinate the EQR and EQR-related activities with other State 
quality activities currently in place.
    Comment: Many commenters believed that direct Medicaid agency 
external reviews should always be performed with respect to grievance 
systems because these commenters believe that the Medicaid fair 
hearings process is unique.
    Response: The EQRO is not responsible for reviewing the State's 
fair hearing process. It must review information about the MCO or PIHP 
internal grievance system. In order for a State to use a Medicare or 
accreditation compliance determination to substitute for a Medicaid 
review of the MCO's or PIHP's grievance system, the State will need to 
address in its quality strategy the basis for considering the Medicare 
or accrediting organization's standard comparable to the State's 
grievance processes standard that needs to comply with the provisions 
of subpart F of the Medicaid managed care final rule.
    Comment: One commenter expressed concern that we excluded Medicare 
beneficiaries who are eligible for Medicaid as a result of spenddown 
requirements from the definition of dually eligible persons.
    Response: We have not excluded from the definition of dually 
eligible those Medicare beneficiaries who are eligible for Medicaid as 
a result of spenddown requirements. We consider any person who is 
receiving both Medicare and Medicaid benefits as a ``dually eligible'' 
person.
    Comment: One commenter believed that the meaning of MCO in proposed 
Sec.  438.360, and Sec.  438.362 was not clear. The commenter noted 
that corporate entities may be wholly owned subsidiaries of other 
corporate entities, and may hold multiple licenses. The commenter also 
noted that in some cases a plan may have a large Medicaid product and a 
very small Medicare product, calling into question the assumption that 
adequate management of the Medicare enrollees is an appropriate proxy 
for their Medicaid enrollees. The commenter recommended a more complete 
definition of MCO, as it relates to the MCO's Medicare and Medicaid 
product lines being incorporated into the rule.
    Response: The definition of MCO as used in this regulation is 
defined in Sec.  438.2 of the Medicaid managed care final rule. 
According to this definition, an MCO is the entity that holds the 
Medicaid comprehensive risk contract. We believe that this definition 
addresses the commenter's concern, as the Medicare review provisions 
will only apply if the same entity that holds the Medicaid contract 
holds the Medicare contract.
    Comment: One commenter recommended that we make clear that a State 
may undertake optional EQR activities, even if it has exempted an MCO 
from a portion of or all of the mandatory activities.
    Response: A State may conduct the optional EQR activities when it 
uses Medicare or accreditation review findings for the mandatory 
activities. As long as the State uses the protocols developed by us or 
protocols consistent with ours, the information derived from the 
optional activities can be used in the EQR.
    Comment: One commenter believed that when an MCO is accredited by a 
private accrediting body, the States should be strongly encouraged not 
to duplicate the review performed by the private accrediting body.
    Response: The final rule provides States the option to use the 
findings of an accrediting body instead of conducting its own review of 
MCO or PIHP compliance with certain standards, if the MCO or PIHP has 
been accredited by a national accrediting organization recognized by 
us. We believe that States should have the discretion to make this 
decision, and individuals who believe that this option should be 
adopted should encourage States to do so.

I. Exemption From External Quality Review (Sec.  438.362)

    Proposed Sec.  438.362 provided an option for a State agency to 
exempt an MCO or PHP from the EQR requirements in section 1932(c)(2)(A) 
of the Act if: (1) The MCO or PHP has a current Medicare contract under 
part C of title XVIII or under section 1876 of the Act; and (2) for at 
least 2 years, the MCO or PHP has satisfied EQR requirements under 
section 1932(c)(2)(A) of the Act with respect to its Medicaid contract. 
In addition, we proposed that the Medicaid and Medicare contracts be 
required to cover all or part of the same geographic area. We also 
proposed that the State agency require each exempted MCO and PHP to 
annually provide the State with copies of all Medicare reviews 
performed by us, by our agent or any private accrediting organization, 
with respect to the quality, timeliness, and access to its services.
    Comment: Many commenters opposed this exemption of certain MCOs 
from EQR. One of the commenters felt that this provision completely 
abrogates the responsibility of the States and CMS to monitor the 
quality of Medicaid managed care systems for children. One commenter 
agreed with this provision, as long as it was an option for States.
    Response: In the BBA, the Congress expressly provided States with 
the option of exempting from EQR those MCOs that provide Medicare 
services and also have had experience serving the Medicaid population. 
This provision, however, does not exempt States from monitoring MCOs 
and PIHPs for compliance with the mandatory activities listed in Sec.  
438.358. These activities, required of MCOs and PIHPs under our 
Medicaid managed care final rule, are essential to ensure the quality 
of services provided to Medicaid beneficiaries by MCOs and PIHPs. For 
example, the BBA requires that States have a quality strategy in place 
when contracting with MCOs and PIHPs. States will still need to ensure 
MCO and PIHP compliance with the BBA provisions and our regional 
offices will continue to monitor States for compliance regardless of 
whether or not an EQR is conducted.
    Comment: One commenter asked how this provision would impact a 
Medicaid plan that gave up its M+C product. Specifically, the commenter 
asked if there would be an immediate requirement for an EQRO review.
    Response: Under Sec.  438.362(a)(1), the MCO and PIHP must have a 
current Medicare contract. Therefore, as EQR is an annual requirement, 
the year following the termination of the M+C plan, the State is 
required to contract with an EQRO to, at a minimum, review and analyze 
information from the validation of performance improvement projects 
conducted by the MCO or PIHP and performance measures calculated by the 
MCO or PIHP that year. The State will also need to ensure MCO or PIHP 
compliance with structural and operational standards. If the MCO or 
PIHP had been reviewed by Medicare or an accrediting organization 
within the previous 3 years, that information could be used in the EQR. 
If this were the year that the MCO or PIHP was to be reviewed for 
structural and operation standards, the State or its contractor, or the 
EQRO would have to conduct a review.
    Comment: Several commenters asked us to clarify who we considered 
appropriate to determine whether an MCO or PIHP performed acceptably in 
previously conducted EQRs, as this was not a requirement under the 
section 1902(a)(30)(C) of the Act EQR requirements. Some of the 
commenters

[[Page 3607]]

stated that it would not be appropriate for the State to make the 
determination, as the independent nature of the EQR might be 
compromised. Many commenters asked us to clarify what we consider to be 
acceptable performance and recommended that an MCO or PHP be required 
to perform acceptably on quality, timeliness, and access in order for a 
State to allow for the exemption.
    Response: Whether an MCO or PIHP has performed acceptably is 
determined by the State based on the results of the EQR, which must 
include a detailed assessment of each MCO's and PIHP's strengths and 
weaknesses with respect to quality, timeliness, and access to health 
care services provided to Medicaid beneficiaries. If a State elects to 
exempt an MCO or PIHP from an EQR it must, as specified in Sec.  
438.362(a)(3), ensure that an MCO or PIHP not only have had a Medicaid 
contract for 2 years but that the MCO or PIHP has also been subject to 
an EQR as specified in this rule. This effectively means that no MCO or 
PIHP could be exempted under Sec.  438.362 until EQR under this final 
rule is in effect for at least 2 years. As long as the provisions under 
this section are met, the State will determine the length of time for 
which it will exempt an MCO or PIHP from EQR. The State will be able to 
use information obtained from the Medicare or accreditation reviews, as 
the submission of Medicare review findings is required under Sec.  
438.362(b).
    Comment: One commenter was concerned that similar geographic 
coverage areas do not necessarily ensure similar administration, 
networks, benefits, and quality improvement projects for the different 
beneficiaries who are served by the Medicare and Medicaid programs. 
Another commenter agreed with the requirement that the two contracts 
cover the same geographic area, but was concerned that practice 
patterns tend to vary geographically for given clinical topics and 
specific types of treatment. The commenter suggested we change the 
geographic requirement to require similar or identical service areas 
instead of overlapping areas. Two commenters supported the requirement 
that the two contracts cover all or part of the same geographic area, 
but suggested that we include additional requirements that the two 
contracts must (1) include the same provider networks and (2) offer the 
same or similar benefit and services to consumers. The commenters 
believe this is important because M+C plans serve markedly different 
populations, provide different benefit packages, and often offer 
different provider networks than Medicaid plans. One commenter asked us 
to clarify whether the Medicaid and Medicare services areas have to be 
identical for MCOs and PHPs to qualify for exemption.
    Response: Under Sec.  438.362(a)(2), we require that the Medicare 
and Medicaid contracts cover all or part of the same geographic area in 
order for a State to exempt the MCO or PIHP from EQR. We required an 
overlap of service areas in this provision because we believe this will 
increase the likelihood that the findings from the Medicare review will 
serve as a proxy indicator of the care delivered to the MCO's or PIHP's 
Medicaid beneficiaries. We have made some clarifying language changes 
to the regulations text in the final rule to more clearly state our 
intent that the contracts must cover all or part of the same geographic 
area within the State that is allowing the MCO or PIHP exemption from 
EQR. However, we think that requiring identical service areas or the 
same or similar benefit packages is too restrictive, and may 
effectively exclude the use of an exemption intended by the Congress.
    Comment: Several commenters asked that we not restrict the 
exemption provision to M+C organizations, but also allow it to apply to 
MCOs and PHPs that have undergone or achieved ``excellent'' status by a 
private accreditation review.
    Response: In the BBA, the Congress applied the total exemption in 
section 1932(c)(2)(C) of the Act only to M+C organizations. 
Consequently, we have not applied this provision to commercial MCOs and 
PIHPs. However, we address nonduplication provisions related to EQR 
activities as they apply to private accreditation under Sec.  438.360.
    Comment: Several commenters concurred with the requirement that an 
MCO or PHP must demonstrate acceptable performance determined by the 
EQR for the 2-year period before exemption. One of these commenters, 
however, was concerned that the regulation appears to allow exempt 
status to last indefinitely, and noted that it is not unusual for plan 
performance to vary significantly from year to year due to 
organizational changes. Several commenters recommended that States be 
required to develop mechanisms to periodically re-evaluate an MCO's 
exempt status, and to re-institute EQR if accreditation, Medicare, or 
State oversight indicate potential quality problems. One commenter 
opposed our proposal to require that the MCO have complied with EQR 
requirements for 2 prior years. This commenter believed that this 
interpretation was unduly restrictive, and inappropriately limited the 
discretion given to State agencies to exempt MCOs based on the State 
agencies' experience with the MCOs or PHPs.
    Response: We believe that the language in this rule properly 
reflects congressional intent to allow States the option to exempt a 
Medicare MCO from EQR. Once an entity is exempted, and continues to 
meet the criteria for exemption, we believe that the Congress intended 
that the Medicare quality review requirements serve as a proxy for the 
Medicaid EQR requirements. Because the State will have access under 
Sec.  438.362(b) to data from these reviews, any problems that develop 
should be recognized through this process. We thus do not believe it 
would be appropriate to require States affirmatively to re-evaluate an 
MCO's or PIHP's EQR-exempt status.
    With respect to our requirement that 2 years of success in Medicaid 
EQR be required, as noted in the preamble to the proposed rule, we 
considered several interpretations of the statutory provision that 
requires at least 2 years of Medicaid contracting in order for this 
exemption to apply. We concluded that the Congress' intent in requiring 
2 years of Medicaid contracting experience was to ensure that the MCO 
had sufficient quality measures in place to meet Medicaid EQR standards 
before it could be exempted from Medicaid review. Since these EQR 
standards are new, this necessarily would require that an MCO have a 
Medicaid contract for 2 years under these EQR requirements before the 
exemption in Sec.  438.362 would apply. This ensures that all MCOs and 
PIHPs have been subject to Medicaid EQR at some point, and have been 
found to be compliant with Medicaid standards in this review.
    We emphasize again, however, that the EQR requirements, from which 
MCOs and PIHPs can be exempted under Sec.  438.362 are only one part of 
the Quality Strategy provided for in the BBA. Other BBA provisions 
require States contracting with MCOs to ensure the quality and 
appropriateness of care and services furnished to Medicaid 
beneficiaries. We believe that if States find MCOs or PIHPs not to be 
providing appropriate quality care, they would exercise their option to 
require an EQR.
    Comment: Many commenters agreed that MCOs should be required to 
submit copies of reviews performed by Medicare or an accrediting 
organization. One commenter did see the benefit in receiving Medicare 
review reports. One of the commenters cautioned that accreditation 
reviews are generally performed less frequently than annually.
    Response: We only require that information from the Medicare or

[[Page 3608]]

accreditation review be provided annually. We are not requiring that 
Medicare or accreditation reviews be conducted annually. If no new 
information is obtained in a specific year, it is not necessary for the 
MCO or PIHP to provide the State information provided the previous 
year. If a State chooses to exempt the MCO or PIHP, this does not 
relieve the State from ensuring that access to timely and quality 
services is being provided. Findings from a Medicare or accreditation 
review will provide the State a useful source of information to 
determine access to quality services for Medicaid beneficiaries. To 
better explain the types of information we are requiring be provided if 
a State chooses this option, and to address situations in which an 
entity is accredited by a private accrediting body approved by CMS 
under Sec.  422.158, we have added clarifying language that makes a 
distinction between when a Medicare review is conducted by us or our 
contractor and when an accreditation review based on deemed compliance 
by such an approved entity. The findings of an accreditation review of 
an MCO or PIHP must be from a review of the Medicare line of business 
as this provision only applies to an M+C organization.
    Comment: Many commenters recommended that MCOs that have 
established distinct provider networks for Medicaid and Medicare 
beneficiaries not be exempt from EQRs.
    Response: As explained in an earlier response, we attempted to 
address differences inherent in Medicare and Medicaid contracts by 
requiring the contracts to have some geographic overlap. We do not 
believe, however, it is necessary or appropriate to require that 
Medicare and Medicaid beneficiaries of the MCO or PIHP use the same 
providers. We believe that an MCO or PIHP that demonstrates 
satisfactory compliance in M+C external review has demonstrated that it 
has appropriate quality safeguards in place, and that these would 
extend to all providers, whether seen by Medicare, Medicaid, or 
commercial enrollees.
    We note that in providing for this exemption in section 
1932(c)(2)(C) of the Act, the Congress did not require that Medicare 
and Medicaid enrollees use the same providers. It did require, however, 
that the entity have 2 years of Medicaid contracting experience. Under 
our interpretation of this requirement, discussed in a previous comment 
response, an MCO or PIHP would be required to demonstrate satisfactory 
results from 2 years of Medicaid EQR under part 438 before it would be 
eligible for the exemption under Sec.  438.362. Thus, even if different 
providers are used by Medicaid enrollees than Medicare enrollees, the 
MCO or PIHP would have demonstrated for 2 years that the Medicaid 
providers performed satisfactorily in EQR before being exempted from 
this review. Having already demonstrated that its Medicaid providers 
met quality standards, the fact that it continues to satisfy quality 
standards in future years under Medicare external review is an 
indication that the entity is continuing its level of commitment to 
quality.
    Comment: Many commenters recommended that the regulations specify 
that in the case of mergers and acquisitions, MCOs be treated as new 
contractors in the Medicaid program, and be subject to an EQR.
    Response: We do not agree with the commenter that the regulations 
should specify that all MCOs and PIHPs that have been acquired or 
merged with another MCO or PIHP be treated as new contractors. There 
are a variety of scenarios that occur when a merger or acquisition 
occurs as indicated by the complex rules that govern how private 
accrediting organizations address these situations. In addition, States 
have their own laws and regulations governing mergers and acquisitions. 
We, therefore, believe the States are in the best position to determine 
quality improvement requirements for newly formed entities and this 
regulation provides States the option to allow for the exemption as 
long as all the provisions in this section are met.
    Comment: One commenter asked that we revise Sec.  438.362(b)(1) to 
specify that the State agency must require each exempted MCO to provide 
it annually with copies of Medicaid reviews performed by State agents 
or any private accrediting organization with respect to the quality, 
timeliness, and access to services instead of Medicare review findings.
    Response: We are not revising Sec.  438.362(b)(1) to require 
Medicaid review findings be submitted to the State because if a State 
or its agent conducted a review, there would be no need to require the 
MCO or PIHP to submit the review findings, as the State would already 
have this information. There is a need, however, for the MCO or PHP to 
submit Medicare review findings if a State chooses to exempt an MCO or 
PIHP from EQR, which is why this requirement is included in Sec.  
438.362(b). The exemption provision does not relieve a State from the 
responsibility for ensuring the adequacy of care provided by an MCO or 
PIHP, and the data from Medicare quality reviews are a source of 
information that will be necessary for States to use to determine the 
appropriateness of exempting an MCO or PIHP from an EQR the following 
year.
    Comment: One commenter recommended allowing States the flexibility 
to decide if their Medicaid services can properly be evaluated by a 
Medicare review.
    Response: States have the flexibility to determine if Medicaid 
services can be appropriately evaluated by a Medicare review. This 
provision provides States with the option to exempt an MCO or PIHP from 
EQR. It does not require the exemption.

J. External Quality Review Results (Sec.  438.364)

    In Sec.  438.364, we proposed a requirement that the product of EQR 
be a detailed technical report, containing (1) a detailed assessment of 
each MCO's and PHP's strengths and weaknesses with respect to quality 
of the health care services furnished to Medicaid enrollees, (2) 
recommendations for improving the quality of the services furnished by 
each MCO and PHP, (3) comparative information about all MCOs and PHPs 
as determined appropriate by the State agency, and (4) an assessment of 
the degree to which each MCO and PHP addressed effectively the 
recommendations for quality improvement, as made by the EQRO during the 
previous year's EQR. Proposed Sec.  438.364 also specified that the 
State must provide the results of the EQR to members of the general 
public upon request, and that the information released may not disclose 
the identity of any patient.
    Comment: One commenter suggested that, because of the differing 
nature of adult and child health care needs, all data produced during 
the course of an EQR should be available by age groups so that parents 
may choose an MCO on the basis of the provision of quality pediatric 
care.
    Response: This rule requires information from a variety of 
activities to be provided to an EQRO and included in the analysis and 
evaluation of the care provided by MCOs and PIHPs. Not all of the EQR 
activities provide detailed information that can be broken out by age 
groups or other categories. For example, a review for compliance with 
structural and operational standards would not yield beneficiary 
specific information. However, encounter data could potentially provide 
that information. In addition, the populations served by MCOs and PIHPs 
are likely to vary along multiple dimensions, including age,

[[Page 3609]]

income, diagnosis, and ethnic group. Because of the variability in the 
populations served by particular MCOs and PIHPs, we have provided 
States flexibility to determine the content of the results made 
available and the manner in which it is presented. To the extent that 
this information identifies quality issues pertaining to a specific 
population, the State may include that information in the results it 
makes available. However, we are not in the final rule requiring that 
EQR results be available by age groups, as this may not always be 
possible or appropriate for a given MCO or PIHP or for given data.
    Comment: One commenter contended that not all quality improvement 
studies monitor quality, timeliness, and access. The commenter 
accordingly suggested that neither the State nor the EQRO should be 
required to summarize the strengths and weaknesses of the MCO or PIHP 
for each of these elements. The commenter also believed that if 
multiple studies are conducted, project time lines are not likely to 
coincide. In addition, the commenter recommended that proposed Sec.  
438.364(a)(5) be revised to require ``An assessment of the degree to 
which each MCO has addressed effectively the recommendations for 
quality improvement as made by the EQRO during the previous measurement 
of the measure or of a similar measure, as appropriate to the study 
performed.''
    Response: The commenter suggesting that the State or EQR should not 
be required to summarize strengths and weaknesses of an MCO or PIHP for 
``each of the elements'' of quality, timeliness, and access implies 
that the results of the EQR process need not address all three of these 
areas. Because section 1932(c)(2)(A) of the Act requires that an annual 
EQR include all three of these elements, it is essential that strengths 
and weakness identified by the EQR process with regard to each are 
described in the results. Because there appears to be confusion on this 
point, we have revised Sec.  438.364(a)(1) to specifically reference 
``timeliness and access.''
    The commenter's suggestion that Sec.  438.364(a)(5) be revised to 
permit the use of a ``previous measurement of a measure,'' as opposed 
to the previous year's EQR recommendations (as the baseline against 
which improvements in MCO or PIHP performance are assessed) is 
inconsistent with the clear direction of section 1932(c)(2) of the Act 
that EQR be an annual review. Further, the Medicaid managed care final 
rule requires performance measurement and improvement projects be 
underway on an annual basis. Consequently, we retain but modify the 
language of the proposed rule requiring the EQR to contain as 
assessment, as opposed to a ``detailed'' assessment of the degree to 
which each MCO and PIHP has addressed effectively the recommendations 
for quality improvement, as made by the EQRO during the previous year's 
EQR.
    Comment: One commenter believed that the reference to ``strengths 
and weaknesses'' in proposed Sec.  438.364(a)(2) implies a subjectivity 
that the commenter found inappropriate in carrying out the EQRO's 
responsibilities. The commenter recommended that the EQRO be required 
to report objectively on the performance of each MCO based on the 
measures selected. This commenter also questioned whether having an 
EQRO make recommendations for improving care and assessing the degree 
to which an MCO has met the previous year's recommendations are 
appropriate elements of the reports, because this is currently--and 
appropriately in the commenter's view--the province of the State (that 
is, identifying deficiencies in contract performance and holding MCOs 
accountable for correcting these deficiencies). The commenter requested 
that we exclude from the EQR reports, an EQRO's recommendations for 
improving care and assessing the degree to which the previous year's 
recommendations were met. If we retain these provisions, the commenter 
asked that Sec.  438.364(a)(3) be revised to (1) allow the MCO the 
opportunity to submit a corrective action plan, which, if accepted 
would be adopted by the EQRO as its recommendation or (2) at a minimum, 
have the opportunity to comment on the EQRO's proposed recommendations. 
The commenter also suggested that Sec.  438.364(a)(5) be revised so 
that the recommendations made by the EQRO are reviewed and approved by 
the State before finalizing the recommendations.
    Response: We do not agree with the commenter that the report of EQR 
results should not address MCO and PIHP strengths and weaknesses. While 
we agree that the EQRO should consider the information produced by 
various EQR-related activities in an objective manner, the results of 
the analysis and evaluation of information will likely identify 
differences in the performance of MCOs and PIHPs with respect to issues 
under study. We believe that it is reasonable to expect the EQRO to be 
able to identify MCOs and PIHPs that had higher or lower scores on the 
State's standardized performance measures, and MCOs and PIHPs that had 
stronger evidence of compliance with certain standards. It is also 
reasonable for interested parties to expect this information to be 
publicly available. We note that this is common practice in the private 
sector where private accrediting organizations release comparative 
information on health plans.
    We agree with the commenter that the State is the entity 
responsible for holding MCOs and PIHPs accountable for contract 
performance. The EQR is a source of information States can use to 
determine the adequacy of MCO and PIHP contractual performance 
regarding quality, timeliness, and access to services. The State may 
choose to require MCOs and PIHPs to submit corrective action plans 
based on the EQR results. In addition, as the State is the entity that 
holds the contract with the EQRO, the State may specify that it have 
the opportunity to review, comment, or approve the recommendations. The 
EQR results will be provided to us upon request, and will most often be 
requested and used by our regional office staff when conducting managed 
care program monitoring reviews. As a result, we retain the language 
included in the proposed rule.
    Comment: One commenter concurred with proposed Sec.  438.364, and 
specifically supported the requirement that EQR results (including 
assessments of MCO strengths and weaknesses and recommendations for 
improvement) be documented in sufficient detail and made publicly 
available. The commenter felt this was vital in order to allow 
interested parties to evaluate the conclusions of the EQR. Another 
commenter concurred with proposed Sec.  438.364, and noted that the 
report required therein could be made available on the internet, to all 
interested parties, thus reducing the burden of report distribution.
    Response: We agree with the commenters. Because the proposed 
language at Sec.  438.364(b) could be interpreted to require the 
release of information in hard copy format only, in response to this 
comment we have modified the regulations text to indicate that the 
State must provide the information specified in paragraph (a) of this 
section, upon request, through print and electronic media, to 
interested parties.
    Comment: One commenter noted that State staff currently perform the 
activities in paragraph (a)(2) of proposed Sec.  438.364, and that 
requiring an EQRO to do this would increase the cost of the EQRO 
contract. The commenter also believed that the EQRO should not be 
making recommendations on improving the health care services furnished 
by each MCO, as specified under paragraphs (a)(3) and (a)(5) of 
proposed

[[Page 3610]]

Sec.  438.364. The commenter felt that the MCO should be responsible 
for designing interventions for improving its members' quality of care, 
and the EQR process should evaluate the effectiveness of these MCO 
interventions. Another commenter recommended these sections be deleted, 
contending that the Act does not require an external entity to perform 
any of the activities listed under paragraphs (a)(2) through (a)(5).
    Response: As stated earlier, we agree that the State is ultimately 
responsible for rendering decisions about MCO and PIHP performance, and 
that EQR results represent one source of information States can use to 
determine MCO and PIHP performance. However, the Congress, in the BBA, 
stated that the EQRO is to perform a review of ``the quality outcomes 
and timeliness of, and access to the items and services for which the 
organization is responsible.'' The Congress further required that the 
results of the reviews be made available to multiple parties. We 
believe that a review requires the EQRO to make judgements regarding 
the MCOs' and PIHPs' performance in these areas and that the judgements 
can reasonably be expected to point to the MCOs' and PIHPs' strengths 
and weaknesses, recommendations about the quality, timeliness, and 
access to services provided by MCOs and PIHPs, and for how to make 
improvements. In order to enable the EQR process to be as effective and 
useful as possible, we retain these provisions in the final rule.
    Comment: One commenter recommended that the regulation specify that 
the EQR results be made available in alternative formats for persons 
with sensory impairments, when requested.
    Response: This comment appropriately suggests accommodations for 
persons with disabilities. At the end of Sec.  438.364(b), in response 
to this comment we have added a sentence requiring States to make the 
EQR results available in alternative formats for persons with sensory 
impairments when requested.
    Comment: Several commenters believed that while it may make sense 
to mandate disclosure of valid, reliable, and objective performance, 
and satisfaction measures, States should not be required to disclose 
the results of other health plan operations, such as contractual 
compliance, and quality improvement studies. In the view of these 
commenters, EQR activities should promote a frank assessment of 
performance in order to provide MCOs and PIHPs the knowledge necessary 
to perform better in the future. The commenters suggested that if the 
results of quality improvement studies were made public, MCOs would not 
treat the process as an unfettered opportunity to assess their own 
performance. Instead, the commenters believed they would tend to 
conduct studies in a way that is likely to generate favorable outcomes 
and, thereby, meaningful quality improvement efforts. One of these 
commenters also noted that if the primary audience for this information 
was Medicaid enrollees, we needed to consider whether such a detailed 
technical report would be relevant to our beneficiaries' needs.
    Response: As we indicated in the preamble to the proposed rule, we 
proposed to require only that summary information made generally 
available is sufficient to enable interested parties to evaluate the 
conclusions of the EQR. The State is not expected to provide more 
detailed underlying data to beneficiaries or the general public. 
However, to clarify the level of detail to be provided in the EQR 
results, in response to this comment, we are revising Sec.  
438.364(a)(1)(iii) to require only that a description of data be 
provided in the technical report, as opposed to requiring that the 
actual data obtained be provided. Our intention was never to require 
that raw data be provided. In addition, as noted above, we are 
providing clarifying language in Sec.  438.364(a)(1) to make clear that 
the technical report conclusions address timeliness and access to care 
as well as quality of care.
    We note that section 1932(c)(2)(A)(iv) of the Act specifies that 
EQR results be made available to providers, enrollees, and potential 
enrollees. In the proposed rule, we broadened this requirement to 
specify that the results be made available to the general public. To 
ensure that adequate information is available for beneficiaries, as 
well as providers, beneficiary advocates, and other stakeholder, we 
believe that some detail in the report is warranted. In addition to 
making the EQR results available, States have the flexibility to 
repackage these results in order to address specific audiences more 
appropriately.
    Comment: Many commenters agreed with our effort to ensure public 
access to EQR results. The commenters also recommended that the 
findings of private accreditation reviews be made available to the 
public when they substitute for all or part of the EQR. They stated 
that this is consistent with the President's Advisory Commission of 
Consumer Protection and Quality in the Health Care Industry 
recommendation that when a private accreditation is used, there must be 
full disclosure of the standards, survey protocols, and the detailed 
information from the surveys.
    Response: Section 438.364 identifies the results of the EQR process 
that must be made available and to whom it must be made available. When 
an EQRO is using private accreditation or Medicare review results under 
the nonduplication option under Sec.  438.360, the EQR results, in 
accordance with Sec.  438.364(a)(1), must still include the information 
required under paragraphs (a)(1)(i) through (a)(1)(iv) of this section. 
We believe that when a State chooses to use the results of a Medicare 
or private accreditation review to replace a Medicaid review, that 
there must be information on the data obtained from the Medicare or 
accreditation review and conclusions drawn from the data consistent 
with Sec.  438.364(a)(1)(iii) and (a)(1)(iv).
    Comment: One commenter asked us to clarify whether the regulation 
envisions that the full technical report be available to the public, or 
whether only certain information about the technical report will be 
made available. The commenter recommended that we establish guidelines 
for preparation of a summary report that must be developed from the 
technical report. The commenter believes that a summary report will be 
more useful to the public and will avoid the potential for the release 
of proprietary information that might appear in the reports.
    Response: As we stated in the preamble of the proposed rule, we are 
only requiring that States make available summary-level information 
that is ``sufficient to enable interested parties to evaluate the 
conclusions of the EQR.'' The State is not expected to provide more 
detailed underlying data or proprietary information to beneficiaries or 
the general public. As we noted earlier, to provide clarification on 
the level of detail to be provided in the EQR results, we are revising 
Sec.  438.364 (a)(1)(iii) to require that a description of data be 
provided in the technical report as opposed to requiring that the data 
obtained be provided.

K. Federal Financial Participation (FFP) (Sec.  438.370)

    Proposed Sec.  438.370 provided that FFP would be available (1) at 
the 75 percent rate for EQR, the conduct of EQR activities, and the 
production of EQR results, by EQROs and their subcontractors, and (2) 
at the 50 percent rate for EQR-related activities performed by entities 
not qualifying as EQROS. The 50 percent rate applies even if the 
activities are of the same type as those that would be matched at the 
75 percent rate if performed by an EQRO.

[[Page 3611]]

    Comment: Several commenters asked us to clarify whether a State 
must contract with an EQRO in order to fulfill its EQR obligations 
under these regulations, and specifically whether it would fail to 
fulfill its obligation under the law if it contracts with an entity not 
qualified to be an EQRO.
    Response: To fulfill its obligations under this regulation, a State 
must contract with an EQRO to conduct an analysis and evaluation of the 
aggregated information produced from, at a minimum, the mandatory EQR-
related activities and produce the EQR results as required under Sec.  
438.364. In response to this comment, we have made clarifying changes 
to Sec.  438.370 to better explain for what activities and functions 
States can obtain a 75 percent, or 50 percent match. That is, States 
can obtain the 75 percent enhanced match for EQR (the analysis and 
evaluation of information produced from EQR-related activities), EQR-
related activities, and the production of EQR results as long as these 
functions and activities are conducted by an EQRO. States can obtain 
the 50 percent match for EQR-related activities conducted by entities 
not qualified as EQROs. However, States must contract with an EQRO that 
meets the requirements of Sec.  438.354 to perform the EQR function of 
analyzing and evaluating the aggregate information from EQR-related 
activities. If a State did not so contract, it would be out of 
compliance with the requirement in section 1932(c)(2) of the Act for 
EQR.
    Comment: One commenter asked whether the enhanced FFP is available 
for the optional activities a State may include in an EQR. Another 
commenter supported the enhanced FFP rates provided for in the Act.
    Response: The enhanced FFP is available for the optional EQR 
activities as long as they are conducted by an EQRO that meets the 
requirements of Sec.  438.354 using the appropriate CMS protocol or a 
consistent protocol.
    Comment: One commenter requested clarification as to whether the 
upper payment limit (UPL) can be adjusted to take into account 
administrative expenses and if not, whether States will be able to 
request waivers of the UPL to reflect these additional expenses.
    Response: The Medicaid managed care final regulation replaced the 
UPL requirements at Sec.  447.361 with new rate setting rules (Sec.  
438.6) by incorporating and expanding requirements for actuarial 
soundness. These new requirements recognize administrative costs and 
allow for States to adjust capitation rates to reflect MCO and PIHP 
administrative costs.
    Comment: One commenter recommended that we revise Sec.  438.370 to 
require States to appropriate a portion of the enhanced FFP to cover 
each MCO's administrative cost associated with meeting this EQR 
requirement.
    Response: We believe that the statute does not permit States to use 
the enhanced funds to pay for MCO and PIHP administrative costs 
associated with EQR. The 75 percent enhanced match is only available 
for costs incurred by States for contracting with an EQRO. However, as 
noted above, with the elimination of the UPL, States now reflect 
administrative costs in capitation payments to MCOs and PIHPs.
    Comment: One commenter asked us to clarify whether validation 
activities are reimbursable at the 75 percent enhanced FFP rate for EQR 
activities.
    Response: The following validation activities are reimbursable at 
the 75 percent enhanced match as long as they are conducted by an EQRO 
that meets the requirements of Sec.  438.354 and the EQRO uses 
protocols developed by us, or protocols consistent with our protocols: 
validation of performance measures, validation of performance 
improvement projects, validation of consumer or provider surveys, and 
validation of encounter data.

L. Miscellaneous Comments on the Preamble of the December 1, 1999 
Proposed Rule

    We noted in the preamble to the proposed rule that we followed two 
principles in its development: first, to provide flexibility to State 
agencies; and second, to reflect well-accepted advances in the methods 
of quality measurement and improvement.
    The proposed rule also acknowledged that in a separate rule 
published in 1998, we had proposed to eliminate the requirements in 
Sec.  434.53 that States have a system of periodic medical audits.
    The proposed rule included a proposed effective date of 60 days 
following publication with provisions that must be implemented through 
contracts with EQROs to be effective with contracts entered into or 
revised on or after 60 days, but no longer than 12 months from the 
effective date. We received the following comments relating to the 
above issues.
    Comment: Several commenters expressed support for the approach 
taken in the proposed rule in providing flexibility for States, and 
asked us to retain mechanisms States already have in place for EQR. 
Several commenters, however, found that the proposed rule did not 
afford States the flexibility and discretion afforded by the BBA. One 
commenter argued that States that demonstrate that their quality 
improvement processes meet or exceed the goals of these regulations 
should be permitted to continue with current arrangements. The 
commenter further contended that section 1932(c)(1)(B) of the Act, 
which requires that the Secretary's standards not preempt any State 
standards that are more stringent than those in the proposed rule, 
supports their position.
    Response: Section 1932(c)(1)(B) of the Act refers to the quality 
assessment and improvement strategy that States are required to develop 
and implement. The components of this strategy were set forth in the 
Medicaid managed care final rule published on June 14, 2002. The EQR 
requirement is one component of this overall State strategy. We agree 
that the statute allows States to exceed the requirements of the 
quality assessment and improvement strategy as outlined in the Medicaid 
managed care final rule. However, the BBA also required the Secretary 
to undertake the activities set forth in this rule; that is, establish 
a method for identifying qualified entities to conduct EQR, develop 
protocols to be used for EQR, and otherwise implement the EQR 
provisions of the BBA. States will continue to have the flexibility to 
exceed the requirements included in this rule and conduct optional 
EQRO-related activities.
    Comment: Several commenters asked us to explain how QISMC, the 
final Medicaid rules, and the EQR compose a cohesive vision and how 
States should integrate the proposed rule into other quality assessment 
and performance improvement activities. One of the commenters believed 
that the proposed rule appeared to set a standard for an overall 
evaluation rather than a specific external review study. Since QISMC 
sets overall standards, the commenter believed that a nonduplicative 
connection to QISMC was important. The second commenter asked us to 
clarify how the EQR regulations will fit in with current and pending 
State requirements.
    Response: This final rule, as did the proposed rule, provides for 
an overall evaluation by an EQRO of the MCO's or PIHP's ability to 
provide timely and quality services to Medicaid beneficiaries as 
required by section 1932(c)(2) of the Act. The mandatory EQR activities 
are based on standards and activities that States must have in place 
under subpart D of the Medicaid managed care final rule.
    Key elements of the QISMC document were incorporated into the 
Medicaid

[[Page 3612]]

managed care final rule, as appropriate. However, in other instances 
the QISMC standards, which we previously offered to States as 
guidelines and not requirements, were not appropriate as requirements 
in the regulations text. Further, the QISMC standards in a number of 
ways have become outdated. For example, the QISMC document does not 
sufficiently address individuals with special health care needs. 
Individuals looking for a cohesive vision of a quality improvement 
system for Medicaid managed care should look to three documents: (1) 
The Medicaid managed care final rule, (2) this EQR final rule, and (3) 
the EQR protocols developed in response to the BBA statutory 
requirement. The QISMC document has been superseded by these three 
documents for the purposes of Medicaid. Each of these documents is 
accompanied by text describing how they should be integrated into State 
quality improvement systems.
    Comment: One commenter contended that the proposed rule 
significantly reduced State flexibility in defining the content and 
cycle of EQR, exacerbated what the commenter considered a double 
standard for quality oversight between Medicaid FFS and Medicaid 
managed care, and placed new requirements on States not previously 
required of managed care programs. The commenter was concerned that 
this rule would create another reason to discourage MCOs and 
potentially PIHPs (especially those that provide behavioral health 
services) from participating in Medicaid resulting in fewer managed 
care options for Medicaid agencies and beneficiaries.
    Response: We do not agree that this regulation significantly 
reduces State flexibility. EQR is not a new requirement on States. EQR 
has been a requirement for States contracting with MCOs since section 
1902(a)(30)(C) of the Act was enacted in OBRA 1986. The BBA introduced 
new requirements for EQR and provided parameters we are obligated to 
follow in developing this regulation. The new requirement in section 
1932(c)(2)(A)(iii) of the Act that protocols be developed which must be 
followed by States necessarily limits State flexibility to some extent. 
However, we believe that we have provided appropriate flexibility in 
implementing this statutory requirement. To do this, in collaboration 
with an expert panel that included State participants, we defined what 
activities we considered to be essential for an EQR. The statute also 
requires that EQR be conducted annually. While flexibility as the 
nature of review under EQR may have been limited somewhat by the 
requirement in section 1932(c)(2)(a)(iii) of the Act that protocols be 
followed, the new rule provides States with substantial new flexibility 
by allowing an expansion of the types of entities with which States can 
contract to conduct EQR activities, and extends the 75 percent match 
rate to these types of entities. In addition, this final rule allows a 
State to conduct EQR-related activities itself or through other State 
contractors. Thus, we do not believe that this rule will discourage 
managed care contracting.
    Comment: One commenter expressed concern that the rule will limit a 
State's ability to maintain and improve distinct State quality 
initiatives due to more extensive Federal quality improvement 
initiatives. Specifically, the commenter believes the rule would 
require States to either externalize or duplicate ongoing State quality 
improvement activities.
    Response: We do not believe that these EQR requirements will result 
in a duplication of any ongoing State quality improvement activities. A 
State may conduct any of the EQR-related activities internally or 
through other State contractors. The State will need to conduct the 
activities using our protocols or protocols consistent with ours if the 
information is to be used as part of the EQR. Therefore, at a minimum, 
our protocols or protocols consistent with ours must be used for the 
mandatory activities. As stated earlier, the protocols are generic 
instructions to ensure that the activities are conducted in a 
methodologically sound manner. If a State chooses to conduct EQR 
activities internally or have a State contractor other than the EQRO 
conduct the activities, the State expenses will be matched at 50 
percent. States must contract with an EQRO for only one function, that 
is for the analysis and evaluation of the aggregated information 
provided from the EQR activities and the development of the EQR 
results. States can also continue to conduct other quality initiatives 
outside of the scope of EQR and claim the 50 percent administrative 
match.
    Comment: One commenter contended that the proposed rule exceeded 
our statutory authority. Specifically, the commenter argued that with 
this rule, we effectively assumed control of a State's quality 
assessment and performance improvement strategy by specifying (1) the 
details of QI activities through detailed protocols developed without 
input from individual States, and (2) which activities can be performed 
by a State government entity, and which must be delegated to the EQRO. 
The commenter recommended that the proposed rule be withdrawn and 
redrafted to: (1) Allow for public review and comment of the protocols, 
and (2) permit States to carry out their statutory responsibilities as 
reflected in section 1932 (c)(1)(A) of the Act. The commenter also 
doubted that uniformity of EQR results could be accomplished in light 
of State programs that demand custom-tailored management and oversight 
models.
    Response: We do not agree that we have exceeded our statutory 
authority in developing this regulation. The statute clearly required 
us to develop protocols to be used in the external review. We developed 
the protocols, as mandated, through an independent quality review 
organization with the guidance of an expert panel that included State 
representation, as required by the statute. A Federal Register notice 
announcing the completion of the protocols was published on November 
23, 2001 (66 FR 58741). In that notice, we asked for comment on the 
extent to which burdens were imposed by the protocols, or on any other 
aspect of the protocols. Comments received from that solicitation, and 
our responses, are included in the preamble to this final rule.
    We also believe we have provided significant flexibility to States 
as to which activities must be performed by an EQRO, as the only 
activity that must be conducted by the EQRO is the analysis and 
evaluation of the aggregated information produced from the EQR 
activities, and production of the results of that review as defined in 
Sec.  438.364. The State can conduct the mandatory EQR-related 
activities, or have another State contractor conduct these activities, 
as long as the State uses our protocols or protocols consistent with 
ours.
    Comment: One commenter believed that the EQR activities in the 
proposed rule were duplicative of the scope of work required in 
Independent External Evaluations of waivers under section 1915(b) of 
the Act, and recommended that the proposed rule be withdrawn until we 
develop a unified, coordinated approach to waiver oversight.
    Response: The EQR activities in this rule are not duplicative of 
activities conducted as a part of independent assessments under section 
1915(b) of the Act. The independent assessment requirement is a review 
of a State's mandatory managed care program under the authority of 
section 1915(b) of the Act. It reviews how adequately a State ensures 
access to quality services in the mandatory managed care waiver 
program, and the costs of the waiver program. The unit of analysis of 
the

[[Page 3613]]

independent assessment under section 1915(b) of the Act is the State's 
managed care program as a whole, not individual MCOs or PIHPs. In 
contrast, the EQR review is a review of individual MCOs and PIHPs. The 
EQR requirement applies to all MCOs and PIHPs regardless of whether the 
program is voluntary or mandatory or whether it is authorized under a 
waiver. Further, EQR is conducted annually, whereas the review under 
section 1915(b) of the Act is conducted for the first 2-year period of 
the waiver, and the first renewal period (assuming the review results 
are acceptable). In addition, the independent assessment that we 
require in the case of a waiver under section 1915(b) of the Act 
applies to PCCM programs as well as programs with capitated 
arrangements. The EQR requirement does not apply to PCCM programs.
    Comments: One commenter supported the proposed elimination of the 
requirement in Sec.  434.53 for a system of periodic medical audits.
    Response: While we note that this comment does not directly pertain 
to this proposed rule, we agree with the commenter. We believe that the 
system of periodic medical audits under Sec.  434.53 is an out-dated 
approach to quality assessment and improvement which would be 
duplicative of EQR activities. (In this sense, the matter is relevant 
to this final rule.) Consequently, the Medicaid managed care final rule 
published on June 14, 2002 eliminated this requirement, as well as 
other regulations in subpart E of part 434.
    Comment: Several commenters thought the proposed time period for 
bringing contracts into compliance with the new EQR requirements did 
not provide sufficient time for States. One commenter suggested that 
the new EQR rules apply to contracts entered into or revised on or 
after 90 days, but no longer than 18 months from the effective date. 
One commenter believed that States needed more than a year to implement 
this rule. One commenter recommended implementation of the redrafted 
rule on January 1 to be consistent with NCQA and other planning cycles 
and allow up to 180 days before implementation.
    Response: To be consistent with the Medicaid managed care final 
rule, we have retained the effective date of this rule to be 60 days 
following its publication. However, we have revised the time frame for 
provisions to be implemented through contracts with MCOs, PIHPs, and 
EQROs so that they must be effective with contracts entered into or 
revised on or after 60 days following the publication date. States have 
up until no longer than 12 months from the effective date to bring 
contacts into compliance with the final rule provisions.

M. Collection of Information Requirements: December 1, 1999 Proposed 
Rule

    In the December 1, 1999 proposed rule, we asked for comment on the 
following provisions that contain information collection requirements: 
nonduplication of mandatory activities (Sec.  438.360), exemption from 
external quality review (Sec.  438.362), and external quality review 
results (Sec.  438.364).
A. General Comments
    Comment: One commenter contended that the burden to the MCO of 
working with the EQRO is not included.
    Response: As part of the MCO and PIHP contracts with States, MCOs 
and PIHPs are required to work with States on a routine basis. This 
includes working with State contractors. We do not believe that working 
with EQROs adds burden for MCOs and PIHPs but continue to believe that 
it is part of the normal course of business for MCOs and PIHPs with 
Medicaid contracts. Further, a requirement for EQR is not new. It has 
been in place since the late 1980's under section 1902(a)(30)(C) of the 
Act.
    Comment: One commenter felt that while the financial impact of this 
rule may be difficult to quantify, the proposed regulations would 
significantly increase the time and administrative burden on States, 
EQROs, MCOs, and PHPs well beyond the hourly estimates in the preamble.
    Response: We do not agree that the regulation will significantly 
increase the time and administrative burden of States, EQROs, MCOs, and 
PIHPs beyond what we estimated in the proposed rule. Through our data 
and information collection, we know that the EQR-related activities 
referenced in this rule are those that are already typically required 
by States. Similarly, MCOs have previously been complying with EQR 
requirements subsequent to the enactment of section 1902(a)(30)(C) of 
the Act in 1986.

Section 438.360 Nonduplication of Mandatory Activities

    Comment: Several commenters argued that the estimate of the total 
burden for the State for the proposed nonduplication provisions was too 
low, and asked how the estimate of 4 hours was determined. One 
commenter asked what data the MCO would need to provide to the State 
under proposed Sec.  438.360(b)(2) and (c)(2).
    Response: We estimated that it would take State staff approximately 
4 hours to collect, copy, and disseminate the reports, findings, and 
other results of Medicare reviews or information obtained from the 
accreditation reviews and sent to the State. Because we received 
several comments indicating that this estimate was low, but commenters 
did not provide us with what they believe the estimate to be, we have 
increased the burden hours by 100 percent, to 8 hours. In accordance 
with Sec.  438.360(b)(3) of the final rule, the MCO or PIHP needs to 
provide to the State any reports, findings, or results from an 
accreditation review or our review for Medicare for the standards in 
Sec.  438.204(g) that are being substituted in place of a Medicaid 
review. In addition, if the MCO or PIHP provides services to dually 
eligible individuals and the State allows the MCO or PIHP to provide 
information from a Medicare review of performance measures and 
performance improvement projects for the EQR in place of separate 
Medicaid measures and projects, under Sec.  438.360(c)(3), the MCO or 
PIHP will need to provide the results of Medicare review activities to 
the State.

Section 438.362 Exemption From External Quality Review

    We did not receive any comments on the information collection 
burdens associated with complying with this provision.

Section 438.364 External Quality Review Results

    Comment: One commenter noted that the preamble of the proposed rule 
addresses the burden of disseminating information, but not of creating 
the content listed. The commenter believed that the burden for creating 
the information required to comply with Sec.  438.364(a)(2) would be 
significant, and would serve no purpose other than to comply with the 
rule. The commenter recommended deleting Sec.  438.364(a)(2). Several 
commenters argued that the effort to compile and aggregate the data, 
analyze, and formulate the review reports will take a significant 
number of hours above the estimated number.
    Response: The proposed rule did not address the burden of 
conducting EQR activities, because we had not completed the protocols 
at the time the proposed rule was published. A request for comment on 
the information collection requirement burden of the protocols was 
solicited in our November 23, 2001 Federal Register notice. We did, 
however, address in the proposed rule the burden associated with 
creating the EQR results report. We estimated

[[Page 3614]]

that it would take 160 hours for an EQRO to prepare and submit the EQR 
results. Since we received several comments stating that it would take 
more time than the 160 hours we proposed, but commenters did not 
provide us with time estimates, we are increasing the burden hours by 
25 percent.
    We do not agree that the burden of Sec.  438.364(a)(2) is 
significant, or that it serves no useful purpose. We believe that an 
assessment of the strengths and weaknesses of MCOs and PIHP performance 
as it relates to the quality, timeliness, and access to health care 
services was the intent of the statutory provision that requires the 
results of EQR be made available to beneficiaries and providers. We 
retain these EQR results provisions in the final rule.

N. Impact Statement

    To comply with Executive order 12866 and the Regulatory Flexibility 
Act we examined the impact of the December 1, 1999 proposed rule. We 
determined that the net impact of the proposed rule would be below the 
$100 million annual threshold, and that a regulatory impact analysis 
was, therefore, not required.
    Comment: Several commenters believed that the proposed rule would 
result in greater costs and burden to States and MCOs than we estimated 
in the impact statement. The commenters stated that we did not estimate 
the increased costs to States and MCOs for external review for 
compliance with standards. The commenter also felt that we did not 
consider the negative impact of external auditing on other MCO 
activities, or new and ongoing infrastructure and labor, needed to 
comply with these provisions. One commenter contended that these 
activities would require MCOs and their providers to devote significant 
staff time to collect, organize, and prepare for review of large 
quantities of quality assurance data. Another commenter felt that due 
to the independence requirements, the net results would be that fewer 
entities would qualify to conduct EQR.
    Response: We do not agree with these comments. The only activity 
that must be conducted by an EQRO is the analysis and evaluation of the 
information obtained from the EQR activities. If a State chooses to, it 
can conduct any of the EQR-related activities and receive the 50 
percent administrative match as long as the activities are conducted 
using our protocols or protocols consistent with those we developed. In 
addition, many States are already conducting or having State 
contractors conduct many of the EQR activities. As we stated in our 
proposed rule, most States are already obtaining a 75 percent matching 
rate for many of these activities and we, therefore, believe there will 
not be a significant increase in Medicaid expenditures, and that no new 
significant infrastructure will be needed. We do not believe that this 
requirement will cause MCOs to devote significantly more time to 
collect, organize, and prepare for EQR than is already required by 
States to ensure compliance with their contracts with MCOs and PIHPs.
    Because this will be a new requirement on PIHPs, we acknowledged in 
the proposed rule that there may be additional cost to the Federal 
government, since States currently conducting these activities receive 
a 50 percent administrative match, but under this rule they can now 
obtain the enhanced 75 percent FFP. We do not believe these costs are 
significant. Based on an analysis of 2001 Quality Improvement 
Organization funding on the CMS-64, we estimate a cost of $5,800,000.
    Comment: One commenter, while supportive of holding MCOs 
accountable by measuring quality of care, noted that there is no such 
requirement for the Medicaid FFS program, and that these costs are, 
therefore, not reflected in the rate-setting methodology for managed 
care plans. This commenter also noted that undertaking these reviews 
has a significant cost implication for both the MCOs and the State.
    Response: The statutory quality assessment provisions implemented 
in this final rule do not apply to the Medicaid FFS program. Moreover, 
there is no statutory or legislative history to indicate that the 
Congress intended that these provisions should apply to Medicaid FFS. 
The Collection of Information Requirements and Impact Statement address 
what we believe to be the cost implications of this requirement as it 
pertains to Medicaid capitated programs. We note that in the Medicaid 
managed care final rule, a new methodology was adopted for setting 
capitation rates. This methodology permits States to reflect MCO and 
PIHP administrative costs (including costs of complying with quality 
assessment requirements that do not apply under FFS Medicaid) in 
capitation rates.
    Comment: One commenter believed that requiring an independent 
organization to conduct a review of an MCO's structural and operational 
standards would add an additional administrative expense to the 
program.
    Response: States currently review MCOs and PIHPs for compliance 
with State standards. If conducted by the State, this expense is 
reimbursed at a 50 percent administrative match. However, some States 
currently define this activity as part of EQR, and thus receive the 75 
percent enhanced Federal match. Under the provisions of this rule, if a 
State chooses to contract with an EQRO to conduct a review of MCO and 
PIHP compliance with State standards, a State can obtain a 75 percent 
enhanced match rate. While this may increase Federal expenditures, we 
do not believe that the increase will be significant, as some States 
already have their EQROs conduct this activity. Thus, we do not believe 
this affects our conclusions regarding the need for a regulatory impact 
analysis.
    Comment: One commenter believed that the proposed reporting 
requirement would increase costs.
    Response: States currently have their EQROs develop reports. We 
believe that this will not add significantly to the current costs 
incurred by the Medicaid program.
    Comment: One commenter believed that our proposed decision to 
extend EQR requirements to PHPs would increase costs to States, and 
that we have not fully analyzed this financial impact.
    Response: We stated in our proposed rule that applying this 
provision to PHPs might result in additional costs. Although States are 
currently conducting a variety of quality activities with their PIHPs 
and receiving a 50 percent administrative match for their costs, they 
now may obtain the enhanced 75 percent FFP match for these activities. 
Again, while this will result in some additional Federal costs, State 
costs will decline. We do not believe these costs are significant. As 
stated in a previous response, based on an analysis of 2001 Quality 
Improvement Organization funding from the CMS-64, we estimate a cost of 
$5,800,000.
    Comment: One commenter was concerned about the cost of responding 
to additional EQR requirements, and the potential for duplication and 
administrative burden to comply with QISMC, the Medicaid rules, and EQR 
rules.
    Response: We do not foresee that there will be any duplication of 
effort between complying with the BBA provisions, including the EQR 
provisions, and QISMC. As we stated previously, QISMC has been 
superseded by the Medicaid managed care final rules that incorporate 
key elements of the QISMC document.

[[Page 3615]]

III. Collection of Information Requirements: November 23, 2001 Federal 
Register Notice: Discussion of Public Comments

    Many of the comments we received in response to the November 23, 
2001 Federal Register notice were issues pertaining to the December 1, 
1999 proposed rule, as opposed to collection of information 
requirements or other issues concerning the protocols. Most of those 
issues were addressed in the previous section that responded to 
comments received on the December 1, 1999 proposed rule. This section 
addresses comments related to the burden estimates and any other aspect 
of the collection of information. We believe that burden estimates 
apply to the following sections of the regulation: EQR protocols (Sec.  
438.352), Nonduplication of mandatory activities (Sec.  438.360), 
Exemption from EQR (Sec.  438.362), and EQR results (Sec.  438.364). We 
first address general comments.

A. General Comments

    Comment: Several commenters did not agree with the methodology we 
used to estimate costs associated with implementing EQR. One commenter 
believes the methodology is flawed and our projected costs may be 
significantly lower than actual costs because our sample was too small 
and the range of estimates is too large for cost averaging. The 
commenter is also concerned that the methodology does not account for 
indirect costs such as rent, transportation, and medical record 
photocopies. The commenter recommended that indirect costs that account 
for geographic variation should be added to accurately predict the cost 
of using the protocols. One commenter stated that our approach did not 
include a determination of whether the function performed by the 
sampled EQROs approximated the functions that would need to be 
conducted in accordance with the protocols. The commenter further noted 
that because we estimated a range of hours for conducting EQR-related 
activities, we have not provided a representative assessment of the 
burden to perform the EQR activities. The commenter recommended we 
develop a more accurate projection of hours and costs associated with 
performing these activities consistent with the protocols.
    Response: While the actual number of EQROs we interviewed was 
relatively small, as stated in our November 23, 2001 Federal Register 
notice, these EQROs had reviewed 16 managed care programs in 8 States 
(Arizona, California, the District of Columbia, Maryland, New Mexico, 
Nevada, Tennessee, and West Virginia). Each of these States contract 
with a different number of MCOs to provide Medicaid services, ranging 
from States contracting with a few MCOs to States with several dozen 
MCOs. So, even though the number of EQROs we interviewed was small, we 
believe we chose EQROs that represented a broad range of experience in 
terms of the number of MCOs they review, as well as representing an 
adequate geographic mix.
    We also recognize that using a broad range of hours given by the 
interviewed EQROs to estimate the average number of hours it will take 
to conduct each activity may overestimate or underestimate the actual 
costs. However, by showing the ranges of costs we averaged, we show the 
variability across States that are inherent when conducting quality 
review activities. As stated above, we believe the interviewed EQROs 
represent an adequate number of MCOs reviewed. In addition, even though 
we did not specifically ask each EQRO about the methodology that they 
used to conduct the EQR activities, the protocols represent generic 
activities and steps that are followed in both the public and private 
sector. We, therefore, believe that the activities for which we 
collected cost information were conducted using a methodology 
consistent with our protocols. Moreover, we have no reason to believe 
that the interviewed EQROs' estimates provided did not include indirect 
costs for conducting EQR activities. Because the commenters did not 
suggest a specific methodology or what other data should be used in 
such a methodology, we retain the methodology used in the November 23, 
2001 Federal Register notice. We have updated the estimates based on 
more current data on the number of MCOs and PIHPs contracting with 
State Medicaid agencies to provide services to Medicaid beneficiaries.
    Comment: One commenter objected to our not including the time 
necessary for MCOs to collect and submit the information necessary to 
perform the functions identified under Sec.  438.358, activities 
related to EQR. The commenter recommended that we interview health 
plans to determine the estimates for this activity and include them in 
our analysis.
    Response: We agree with the commenter and include burden estimates 
in this final rule to address the time and costs associated with MCO 
and PIHP submission of information necessary for the validation of 
performance measures, validation of performance improvement projects, 
and a review for compliance with structural and operational standards. 
The protocols for all three of these activities require that 
documentation be provided by the MCO or PIHP. We do not anticipate, 
however, that new documentation will need to be developed. For example, 
the documentation review activity that occurs when a review for 
compliance with standards is conducted includes a review of reports, 
policies, and surveys that already exist. We believe that it will take 
each MCO or PIHP approximately 4 weeks of one full-time equivalent 
employee to prepare the information to be submitted for the three 
mandatory activities and we have added this estimate under Sec.  
438.352, the EQR protocols.
    Comment: Two commenters believe the protocols will result in 
significant burdens in the areas of data collection, duplication of 
management oversight, and financial costs to the State and its 
contracting MCOs. One commenter estimated the new costs associated with 
the three mandatory activities and the overall EQR will be an 
additional $250,000 per MCO. Another commenter believes the cost per 
MCO would be approximately $424,000 for the three mandatory activities. 
The commenters noted there will be additional indirect cost incurred by 
the State to administer and oversee the EQRO contracts, and by the MCOs 
associated with the annual preparation for the three mandatory 
activities.
    Response: We do not agree that the protocols will cause significant 
financial costs to MCOs and States, cause significant burdens in the 
areas of data collection, or duplicate other oversight activities. Many 
States already require their contracting MCOs and PIHPs to conduct 
performance improvement projects, calculate performance measures, and 
comply with State standards. The three mandatory activities that ensure 
compliance with these requirements are also already conducted by many 
States. However, States may not be contracting with their EQRO for the 
conduct of all these activities. As stated earlier in this preamble, 
the State can conduct these activities itself or contract with an EQRO 
or other entity for the conduct of the EQR-related activities. If the 
State contracts with an EQRO, it will receive the enhanced 75 percent 
FFP. If States are not currently contracting with their EQROs for these 
activities and decide to contract with their EQRO for EQR-related 
activities under this authority, it will decrease their costs related 
to quality activities, as opposed to increasing their costs.

[[Page 3616]]

    We believe that the EQR mandatory activities can easily be 
incorporated into existing State quality assessment systems and will 
not duplicate existing oversight activities. The conduct of EQR and the 
conduct of EQR-related activities is required as part of the quality 
strategy under Sec.  438.204 of the Medicaid managed care final rule 
and MCO quality assessment and performance improvement program 
requirements under Sec.  438.240 of the Medicaid managed care final 
rule. Furthermore, we believe that there will not be additional costs 
incurred by the State to administer and oversee the EQRO contracts 
since this is already an existing requirement on States and MCOs under 
OBRA 1986. Because the commenters did not provide us with an 
alternative methodology to use or evidence to support their statement, 
we retain the approach taken in the November 23, 2001 Federal Register 
notice on the information collection requirements and in the impact 
statement in the December 1, 1999 proposed rule.
    Comment: One commenter disagreed with our assumption that the 
implementation of EQR would not have an increased cost to the Federal 
government. The commenter did not agree that the costs incurred with 
current EQR activities are representative of costs that would be 
incurred under the new requirement. The commenter argued that States 
currently contract with EQROs for a more limited scope of activities.
    Response: Our December 1, 1999 proposed rule acknowledged that 
there is likely to be an increase in Federal expenditures but that we 
did not anticipate this to be a significant increase. We agree with the 
commenter that the scope of work may be different under the BBA EQR 
requirements than it was under the OBRA 1986 requirements. However, we 
do not believe that the cost difference will be significant and it is 
likely that there could be a decrease. By expanding the pool of 
organizations available to conduct EQR, State agencies may be able to 
negotiate savings. We also hope that additional savings will be 
realized through opportunities afforded by this rule to coordinate EQR 
activities with other quality and oversight activities.
    As stated in our December 1, 1999 proposed rule, we expect some 
increase in expenditures since we are applying the EQR requirement to 
PIHPs. We do not expect this to be a significant increase in 
expenditures because States already conduct quality review activities 
on PIHPs and receive a 50 percent FFP. Now States will be able to 
qualify for the enhanced 75 percent FFP.

Section 438.352 EQR Protocols--General Comments

    Comment: One commenter believes the scope of the protocols could 
result in excessive burdens and they should be revised.
    Response: For several reasons, we do not agree that the scope of 
the protocols will result in excessive burdens. First, all protocols 
are based on procedures already in use in the private sector. These 
protocols, therefore, are consistent with common industry practice in 
widespread use today. Second, many States and MCOs and PIHPs are 
already conducting these activities, using methods consistent with or 
more intensive than the activities and steps found in these protocols. 
For example, many State agencies are using the CAHPS surveys. The 
protocols for administering these surveys are consistent with our 
survey protocol, but much more prescriptive. Similarly, many States are 
also requiring validation of performance measures or encounter data 
using approaches consistent with these protocols. Third, the States 
have the option to use the protocols we developed or protocols 
consistent with ours. The protocols also include sample worksheets that 
can be used or modified at the State's discretion. Fourth, we note that 
States are only required to use three of the nine protocols that we 
have developed; the other six protocols are developed for optional 
activities that States can choose to undertake or not, at their 
discretion. For these reasons, we believe the protocols will not be 
excessively burdensome, and we retain the scope of the protocols as 
introduced through the November 23, 2001 Federal Register notice.
    Comment: One commenter recommended that there be a better 
explanation of the use and purpose of the protocols.
    Response: Section 1932 (c)(2)(iii) of the Act required us, in 
coordination with NGA, to contract with an independent quality review 
organization to develop protocols to be used as part of EQR. The 
purpose of the protocols is to provide EQROs with a set of generic 
instructions that ensure that EQR activities are conducted using sound 
methodological principles. To provide ongoing explanation about the use 
of the protocols, we have created a Web site at http://www.cms.hhs.gov/medicaid/managedcare/mceqrhmp.asp that presents the protocols and an 
explanation of their intended use.
    Comment: One commenter recommended that we not base the protocols 
on Federal or industry guidelines and standards, but that we 
incorporate these standards by reference.
    Response: We disagree with the commenter. We purposefully directed 
our contractor to develop the protocols following protocols and quality 
review activities currently used in the managed care and quality 
oversight industries. We believe it is important to take advantage of 
the knowledge and experience that exists in the Medicare program and 
the private sector. Consistency with these approaches will also 
minimize the burden of complying with the protocols.
    Comment: One commenter believes that the activities in this 
protocol will result in the State agency becoming the accrediting 
agency for Medicaid managed care, increasing the scope of prescribing 
and monitoring necessary by the State.
    Response: We disagree with the commenter. The purpose of the three 
mandatory EQR-related activities is to ensure that MCOs and PIHPs are 
in compliance with Sec. Sec.  438.204(g) and 438.240 of the Medicaid 
managed care final rule. However, many States currently conduct these 
activities. States that do not currently monitor for compliance with 
quality standards, monitor MCO and PIHP quality improvement projects or 
require the calculation of performance measures will need to initiate 
these activities. We believe that monitoring for these activities is 
consistent with the intent of the BBA EQR statutory provision to ensure 
that MCOs and PIHPs are providing access to timely and quality 
services.
    Comment: One commenter believes the protocols are very clear in 
describing what information needs to be collected.
    Response: We agree with the commenter and retain the activities and 
steps in the protocols introduced through the November 23, 2001 Federal 
Register notice.
    Comment: One commenter believes that the protocols lack an 
evidenced-based approach to quality improvement. Another commenter 
believes that measuring MCO performance should be oriented to empirical 
performance outcomes and applied against quantifiable baselines and 
benchmarks rather than determining compliance through document reviews 
and interviews.
    Response: We disagree with the first commenter. As we explained 
above, these protocols were developed

[[Page 3617]]

consistent with protocols and quality review activities currently used 
in the managed care and quality oversight industries. Further, the 
protocols addressing performance improvement projects explicitly 
incorporate provisions addressing the use of clinical and nonclinical 
evidence in the selection of quality indicators. We agree with the 
second commenter that MCO and PIHP performance should be oriented 
towards performance outcomes that are measured against baselines and 
benchmarks. This is one reason why the information obtained from the 
validation of performance measures and the validation of performance 
projects is to be included as part of the EQR function. We also believe 
however, that a review of the MCO's and PIHP's compliance with State 
standards is essential for determining whether access to quality and 
timely services is provided. We believe this information used in 
conjunction with the information obtained from the validation of 
performance measures and performance improvement projects provides for 
both a qualitative and quantitative approach to assessing MCO and PIHP 
performance.
    Comment: One commenter recommended that specific clinical areas 
(for example, early and periodic screening, diagnosis, and treatment 
(EPSDT) reporting) be addressed in multiple protocols.
    Response: We believe that a variety of both clinical and 
nonclinical areas of care need to be assessed by the State and MCO or 
PIHP over time. However, we do not specify in regulation or in our 
protocols what those specific clinical and nonclinical areas should be 
because we believe that States should have the discretion to identify 
priority topics based on their knowledge of the public health 
priorities in the State, the health care needs of their beneficiaries, 
and based on discussions with beneficiaries and other stakeholders in 
the State. If we do decide that it is necessary to identify national 
priority topics, Sec.  438.240(a)(2) of the Medicaid managed care final 
rule provides us with the authority to do so in consultation with 
States and other stakeholders.
    Comment: One commenter asked that the protocols reflect our review 
criteria for children with special needs.
    Response: When States require children with special health care 
needs to enroll in a capitated Medicaid managed care program, they must 
follow the review criteria provided in the January 19, 2001 State 
Medicaid Directors' letter. The Medicaid managed care final rule 
includes standards States must comply with when contracting with MCOs 
and PIHPs that enroll Medicaid beneficiaries, including children with 
special health care needs. These standards address the principles on 
which the review criteria are based. This protocol does not put forth 
any new standards, but identifies methods to determine compliance with 
current standards.
    Comment: One commenter suggested that the protocols require the 
validation of performance measures submitted by MCOs, unless the 
measures were validated by a reliable entity using comparable 
standards.
    Response: If performance measures are validated by an entity using 
an approach consistent with our protocol, only the information obtained 
from that review needs to be provided to the EQRO to be used as part of 
the EQR function. The review activity itself need not be duplicated. In 
addition, if the entity qualifies as an EQRO, the State can capture the 
enhanced 75 percent Federal match.
    Comment: One commenter recommended that assessments of quality 
should include multiple sources of information including audits, 
certifications of sufficient networks and systems, and other 
submissions the MCO has provided to the State outside of the review 
process.
    Response: We agree with the commenter that information from 
multiple sources should be included as part of the EQR. We believe we 
have accomplished this through the multipronged approach we have 
provided for in this final rule. The EQR will include information from 
the validation of performance improvement projects, the validation of 
performance measures, and a review for compliance with standards that 
may include plan network adequacy information, service authorization 
procedures, and other documentation that attests to the structural and 
operational components of the MCO or PIHP.

B. Protocol for Determining Compliance With Structural and Operational 
Standards

1. General Comments
    Comment: The commenter believes that because we used a combination 
of private sector protocols in the development of the protocol for 
compliance with structural and operational standards, our protocol is 
likely to be more burdensome than that of any one private sector 
protocol.
    Response: We reviewed a number of private sector protocols in the 
development of the protocol for compliance with structural and 
operational standards. We identified those elements common to all and 
used those as a basis for the protocol. Our protocol is not an additive 
combination of private sector protocols. Conversely, it is a synthesis 
or a streamlining of common elements found in multiple private sector 
protocols. Consequently, we do not believe our protocol is more 
burdensome than any one private sector protocol.
    Comment: One commenter argued that CMS, for Medicare, is changing 
its onsite review process so this will be less frequent and more 
targeted. Medicare is also streamlining its review guide and will be 
reviewing less documentation and including more self-auditing by MCOs. 
The commenter recommended that we adopt a similar approach.
    Response: The process for how this protocol will be used is set 
forth in this final rule, which contains provision for less frequent 
monitoring, and under certain circumstances, for the nonduplication of 
activities conducted under the Medicare program reviews or independent 
accreditation surveys. Through these regulatory provisions, we believe 
we have adopted a streamlined approach to quality review, similar to 
that used by Medicare.
    Comment: One commenter is concerned that this protocol requires 
intensive onsite reviews to determine compliance with the structural 
and operational standards required in the Medicaid managed care final 
rule. The commenter believes that to meet the goals of EQR, it is not 
necessary to include all the areas identified in the monitoring 
protocol and that States should not be required to use this approach. 
One commenter believes that the guidance on the onsite review process 
is prescriptive and it is unlikely that the EQRO will need or use this 
detailed level of guidance. In general, the commenter believes the 
protocol is overly detailed and should be simplified to examine major 
structural and process requirements.
    Response: The degree to which the protocol relies upon onsite 
reviews is consistent with the degree to which onsite review is used by 
private accrediting bodies. Therefore, we do not believe the onsite 
review specified in our protocol is too intensive. In the private 
sector, when an accrediting body has a standard, they monitor for 
compliance with it through a combination of interview activities and 
document review. We have followed this private sector approach and 
intend that all Federal requirements be monitored for compliance. 
Because the protocol contains only ``potential'' interview questions 
and documents for

[[Page 3618]]

``potential'' review, States, in using the protocol, will be able to 
target the reviews as they determine appropriate. We believe the 
protocol provides an appropriate amount of detail needed to reflect the 
scope and depth of the quality review activities to be conducted. We 
note in the protocol that, although the EQR activities must be 
consistent with the protocol, they need not be identical, thus 
providing the option for the States to prescribe a less detailed level 
of activity to the EQRO.
    Comment: One commenter recommended that documents be obtained in 
advance and that multiple fact-finding efforts occur over time before 
conducting the onsite reviews. This allows State staff to be better 
prepared and is less disruptive for MCO staff.
    Response: The EQR protocols are designed for use by EQROs which in 
many circumstances are not likely to be staffed by State personnel. 
However, State staff conducting compliance reviews may also use the 
protocols at their discretion. The protocols specify that documents may 
be obtained in advance, and reviewers, though not directed to do so, 
are not precluded from performing these activities over time.
    Comment: One commenter recommended that the protocol include the 
review of previous monitoring reports and that the MCO's efforts and 
progress in correcting past problems be noted.
    Response: We agree with the commenter. Therefore, in the final 
protocol, we have added that, before the onsite visit, reports on 
previous reviews and subsequent MCO and PIHP corrective actions be 
reviewed to identify areas on which the EQRO might need to focus the 
current monitoring activities.
    Comment: One commenter recommended that the protocol include a 
mechanism for the State to prepare and submit oversight findings to the 
MCO and approaches to follow-up to ensure that corrective action has 
occurred. The commenter also recommended that every onsite review end 
with an exit interview to focus the MCO's attention on those areas the 
State is concerned about and intends to address in the findings and 
recommendations report.
    Response: We agree with the commenter that evaluation results need 
to be reported to the MCO or PIHP. This reporting is common practice 
upon completion of a performance evaluation and a number of strategies 
are available for this reporting. We describe four possible 
alternatives for reporting in the protocol, but States are not 
precluded from selecting other alternatives that might include exit 
interviews with the MCO or PIHP at the conclusion of the onsite review.
    Comment: One commenter recommended simplifying the compliance 
scoring system and placing greater emphasis on objective indicators of 
organizational performance such as performance improvement projects and 
survey results.
    Response: We agree that other sources of information may provide 
information pertaining to MCO/PIHP compliance with the regulatory 
provisions, and we list some of these sources in the protocol under 
Activity 5, ``Collecting Accessory Information.'' In defining 
regulatory compliance, we have indicated that the State Medicaid agency 
will need to identify the level of compliance it requires and what 
rating or scoring system is to be used. In the protocol, we offer 
examples of common approaches, but because there is no evidence that 
one scoring system is better than all others, we allow States the 
discretion to select the scoring system to be used.
    Comment: One commenter believes that of the four alternatives 
listed in the protocol for reporting evaluation results to the State 
Medicaid agency, neither the first nor the fourth alternative is 
acceptable. The commenter claims the first alternative makes 
information vital to the review; that is, the reviewers' analysis, 
unavailable to the State, while the fourth alternative represents a 
complete delegation of the State's monitoring responsibility to the 
EQRO.
    Response: We do not agree with the commenter. In the first 
alternative, analysis is guaranteed based upon the definition of EQR in 
this final rule. According to that definition, EQR requires ``the 
analysis and evaluation of aggregated information.'' In the fourth 
alternative, reporting is accomplished based on pre-established State 
thresholds and guidelines, and therefore does not represent a complete 
delegation of the State's monitoring responsibility to the EQRO. The 
four alternatives listed in the protocol are possible scoring 
strategies; we state in the protocol that other options are available 
for use by States.
    Comment: One commenter recommended that States require EQROs to use 
a standard written reporting tool.
    Response: We agree with the commenter and have included a sample 
document and reporting tool (Appendix C, Attachment C of the final 
protocol) for this purpose. However, we allow States to modify this 
sample tool or develop another standard reporting tool, at their 
discretion.
    Comment: One commenter noted that many questions are broad and not 
well written so the nature of the response being sought is unclear. The 
commenter recommended that the entire section for interviews should be 
reviewed in the context of whether the EQR rule is being exceeded by 
the data required during the interviews. Several commenters recommended 
that the interview section be dramatically shortened by eliminating 
duplicate questions and by deleting questions whose answers cannot be 
evaluated against the State's MCO contract specifications or a specific 
provision in the rule.
    Response: We do not agree that we should more narrowly construct or 
abbreviate the interview questions. We have included a range of 
potential interview questions related to the subject matter of the 
regulatory provisions for reviewer use in prompting discussion. We 
expect, in practice, the reviewers will customize the interviews as 
necessary to clarify issues and confirm document findings. In the 
protocol, we compiled questions related to the regulatory provisions 
for each group of interview participants; for example, MCO or PIHP 
leadership, enrollee services staff. While this format creates some 
redundancy among the interview groups, we believe it facilitates the 
interviews by enabling each interview group's questions to stand alone. 
We also note that it is common practice in private accreditation 
reviews to ask the same or similar questions of different MCO or PIHP 
staff and also to review documents to support information obtained from 
interviews to determine if the information obtained from multiple 
sources converges and reaffirms the EQROs conclusions.
    Comment: One commenter believes the protocols are bureaucratic and 
administratively burdensome and that there is a lack of evidence of the 
success of this type of process-oriented oversight. The commenter 
further stated that the level of detail is excessive to ensure 
conformance with MCO contracts and the BBA rule, and that the purpose 
is not for an accreditation.
    Response: The protocols are based upon the common elements found in 
compliance protocols used by private sector accrediting bodies and the 
Medicare program. Consequently, we do not believe they are overly 
bureaucratic, administratively burdensome, or without a sound 
evidentiary basis. We also have followed the private sector approach in 
specifying that all standards, in this case the Federal

[[Page 3619]]

requirements, be monitored for compliance. We believe the protocol 
provides an appropriate amount of detail needed to reflect the scope 
and depth of the quality review activities to be conducted. We note 
again that the specific interview questions are suggestions only, and 
we expect the questions to be customized for each review.
    Comment: One commenter claimed that some informational items the 
EQRO is to collect from the State Medicaid agency do not exist as 
contract provisions and may not exist as other standard documents. This 
will create additional paperwork. The commenter recommended that the 
EQRO should only verify that the State's managed care contracts require 
compliance with applicable State and Federal laws.
    Response: We do not agree with the commenter. The background 
information that the EQRO will need to collect from the State under 
this protocol includes written documentation of those standards, 
requirements, or decisions pertaining to MCOs and PIHPs that the State 
established to comply with the regulatory requirements that implement 
the BBA provisions governing standards for contracts with MCOs and 
PIHPs. This information is needed to assess MCO or PIHP compliance with 
those regulatory provisions for which the State is required to 
establish certain standards.
    Comment: One commenter claimed that the number and types of 
documents the EQRO is to obtain from the MCO are too extensive and that 
many of the Code of Federal Regulations citations used to justify the 
collection of documentation are incorrect and do not relate to the 
topic. The commenter recommended that the protocol be reviewed for 
incorrect citations and references and that corrections be made.
    Response: We do not agree with the commenter. We believe the 
documents listed are those needed to evaluate MCO or PIHP compliance 
with the Medicaid regulatory provisions. The regulatory provisions 
cited indicate where information obtained from the documents can be 
applied in the review process. For example, although Sec.  438.214 
pertains to credentialing and recredentialing, this provision is 
applicable to oversight of delegated activities, if the MCO or PIHP 
delegates credentialing to another entity.
    Comment: One commenter recommended that Appendix B to this protocol 
have a cross-reference table that summarizes each interview question 
with the respective oversight organization documentation listed.
    Response: We believe the format for the protocol itself is 
generally comparable to the recommended cross-reference table for 
Appendix B (Attachment B of the final protocol). The protocol includes 
a table cross-walking the review documentation with the related 
regulatory provisions. The subsequent interview sections then aggregate 
the interview questions by regulatory provision for each interview 
group.
    Comment: One commenter was concerned that we do not include 
information available from consumers as a source of information to be 
used in this protocol. Several commenters believe this protocol does 
not go far enough to examine actual practices of MCOs' or 
beneficiaries' experience with care; rather, it focuses on policies and 
procedures. One commenter recommended the protocol include interviews 
with State Medicaid personnel and providers, and input from consumers, 
consumer advocates, and people with special health care needs.
    Response: We agree that providers, consumers, and others mentioned 
may offer further information about MCO or PIHP performance; however, 
interviewing these groups requires additional time and substantial 
resources. Therefore, in this protocol, we have made provider and 
contractor interviews optional. However, we have further promulgated a 
separate protocol for the use of provider and consumer surveys as a 
source of information that can be used for EQR at the option of the 
State. We believe that mandating additional surveys as a part of this 
protocol would be burdensome and unnecessary.
    Comment: One commenter believes the MCOs can prepare in advance for 
the review. The commenter recommended reviewers should interview 
providers and beneficiaries not preselected by the MCOs to ensure 
compliance with established policies.
    Response: We agree with the commenter's concern regarding 
preselection. For the reasons previously noted, however, provider 
interviews are an optional part of this protocol. Consumer and provider 
surveys are also specified as a separate, optional EQR-related activity 
for securing input from beneficiaries and providers.
    Comment: One commenter recommended that among document review and 
interviews, we include in our approach extensive file review.
    Response: We are unsure what files the commenter is proposing for 
review. The approach used in the protocol is the same approach used by 
the private sector accrediting bodies and in the Medicare program. If 
the commenter is referring to medical record review, these are included 
and discussed in the protocols for validating and conducting 
performance improvement projects and validating and calculating 
performance measures.
    Comment: One commenter suggested that because a core component of 
quality programs is responsibility for the program at the highest level 
of the organization, we include a discussion of committee structure and 
committee oversight in the overview section.
    Response: We assume the commenter is referring to the MCO or PIHP's 
quality assurance committee and oversight. The protocol addresses 
compliance with the standards required in the Medicaid managed care 
final rule. Because committee structure and committee oversight as a 
core component of quality programs is not included as a standard in the 
Medicaid managed care final rule, it would not be appropriate to 
require it in the protocol.
    Comment: One commenter believes that the pertinent issue in team 
development (p. 6 of the protocol) is the identification of the 
specific functions to be reviewed and the assignment of appropriate 
personnel to the task, not the size of the team.
    Response: We agree that an important consideration in the 
development of the review team is the determination of the types of 
personnel appropriate for the review as related to the functions to be 
reviewed. Therefore, we have specified the desirability of reviewers 
possessing knowledge of Medicaid and managed care, and experience and 
familiarity with the regulatory provisions, the evaluation process, and 
performance expectations.
    Comment: One commenter recommended that we include in the list of 
documents on page 18, committee minutes, vendor oversight committee, 
and committee structure of the quality program.
    Response: The list of documents on page 18 refers to the documents 
used for determining compliance with specific regulatory provisions. 
Because the commenter has not stated what regulatory provisions these 
documents would be used to address, we are unclear as to how to propose 
their use and have not included them in the document list.
2. Provider/Contractor Services
    Comment: One commenter recommended that the review of credentialing 
files by the EQRO be deleted because the criteria for auditing the 
files are inadequate. The commenter

[[Page 3620]]

recommended that the element be simplified to call for the EQRO to 
review MCO credentialing policies and procedures for conformance with 
State contract requirements.
    Response: We disagree with the commenter. We believe that a review 
of policies and procedures alone, when the opportunity exists to review 
documents providing direct evidence of compliance or noncompliance with 
the policies and procedures, is a more effective review mechanism. This 
is consistent with the approach used by private sector accrediting 
bodies and in the Medicare program.
3. Staff Planning/Education/Development
    Comment: One commenter suggested that the requirement for the MCO 
to produce staff handbooks and information about staff training and 
orientation be dropped for lack of specificity or rewritten to make 
clear what criteria the auditors are to use in reviewing the required 
materials.
    Response: We indicate on the list of documents the regulatory 
provisions to which each document applies. In this instance, staff 
handbooks and information about staff training and orientation pertain 
to the requirement that staff be educated about the enrollee's right to 
receive adequate information; for example, information on disenrollment 
rights and hearing and appeals. We have specified interview questions 
for MCO/PIHP leadership, provider and contract services staff, and 
enrollee services staff concerning how appropriate staff are informed 
regarding the enrollee right to information. We believe this provides 
sufficient clarity with respect to the criteria reviewers are looking 
for and we retain the references to the staff handbook, staff training, 
and orientation.
    Comment: One commenter suggested that the interview questions 
include probes to determine how staff are trained to comply with 
Federal and State laws, and how staff advise enrollees of their rights. 
The commenter recommended further that interview questions address the 
content, frequency, and thoroughness of the training to confirm no 
major area of law is overlooked.
    Response: We have specified staff handbooks, and orientation and 
training curriculum, in the list of documents to be reviewed and 
included interview questions to confirm MCO/PIHP compliance with the 
regulatory requirements pertaining to enrollee rights and compliance 
with Federal and State laws. However, if issues arise during the 
document review concerning the adequacy of the staff's training 
regarding these provisions, reviewers are directed to explore them 
during the interviews. We believe this direction affords the reviewers 
the flexibility necessary to appropriately tailor the review activity. 
Further, we do not believe it is possible, given the diversity among 
States and MCO/PIHPs and the scope of the review itself, to include in 
the list of potential interview questions probes to explore all 
applicable State laws.
4. Consumer Protections
    Comment: One commenter recommended that the protocol include the 
monitoring of the Medicaid managed care final rule provisions related 
to consumer protections. The commenter specified for inclusion 
provisions addressing: the free choice of providers for family planning 
services (Sec.  431.51); prohibition on provider discrimination (Sec.  
438.12); availability of out-of-network providers in rural areas (Sec.  
438.52(b)); disenrollment rights as a result of grievance procedures, 
and related notice and appeal rights (Sec.  438.56(d) and (f)); 
enrollee rights regarding treatment, second opinions, and medical 
record access and correction (Sec.  438.100); marketing activities 
(Sec. Sec.  438.104, 438.700(b)); liability for payment beyond what is 
legally allowable (Sec.  438.106); program integrity requirements 
(Sec.  438.608); imposition of sanctions (Sec.  438.700); and multiple 
charges and denial of services for inability to pay cost sharing (Sec.  
447.53).
    Response: We have listed in the protocol documents for review to 
determine compliance with regulatory provisions related to prohibition 
on provider discrimination; disenrollment rights as a result of 
grievance procedures, and related notice and appeal rights (Sec.  
438.56(d)); and enrollee rights regarding treatment, second opinions, 
and medical record access and correction. We further agree with the 
commenter and have amended the protocol to include review of the MCO/
PIHP's relevant policies and procedures to assess compliance with the 
regulatory requirements pertaining to the free choice of providers for 
family planning services; liability for payment beyond what is legally 
allowable; and multiple charges and denial of services for inability to 
pay cost sharing. However, the provisions concerning availability of 
out-of-network providers in rural areas; marketing activities (Sec.  
438.700(b)); program integrity requirements (Sec.  438.608); and 
imposition of sanctions (Sec.  438.700) are responsibilities of the 
State and not the MCO/PIHP and, therefore, we have not included them as 
a focus of this protocol. The regulatory requirements in Sec.  438.104, 
while they pertain to MCO/PIHP marketing activities, are contract 
requirements that do not directly provide information on quality and 
are more particular to a State responsibility. Because the protocol is 
designed to determine MCO/PIHP compliance, we believe it would not be 
appropriate to monitor these latter activities through the protocol.
5. Enrollee Services
    Comment: One commenter believes a State can contract with the MCO 
to provide information to potential enrollees, and recommends the 
protocol monitor the MCO's compliance with these informational 
requirements.
    Response: In the August 20, 2001 Medicaid managed care proposed 
rule, we stated that ``it would be unreasonable to require every MCO/
PIHP to provide the relevant information to all potential enrollees.'' 
We believe the MCO/PIHP should not be contracted by the State to 
undertake this responsibility, and explained in the proposed rule that 
``the State agency is the more appropriate entity to do'' the potential 
enrollee informing. This requirement was, therefore, not included in 
our Medicaid managed care final rule and we are not changing the 
protocol to monitor the MCO's/PIHP's compliance with providing 
information to potential enrollees.
    Comment: One commenter recommended the protocol include a standard 
reflecting the regulatory requirement for the provision to enrollees of 
information on services not provided due to moral or religious 
objections.
    Response: We agree with the commenter. The protocol identifies the 
section of the regulation that requires enrollees to be provided with 
information about services that are not provided by the MCO or PIHP 
because of moral or religious objections. It also identifies relevant 
documents to be reviewed to determine compliance (see pages 22 and 77 
of the protocol). These documents include Medicaid enrollee service 
policies and procedures, statement of enrollee rights, and marketing 
materials.
    Comment: One commenter believes the protocol should include 
guidance on how to measure the adequacy of the MCO's activities to 
inform enrollees. The commenter recommends the protocol include 
additional guidance on the fourth grade reading-level standard for 
materials, and confirmation that written materials are at an 
understandable grade level and in

[[Page 3621]]

alternative forms to accommodate individuals with sight impairments.
    Response: We note that we have provided guidance on this issue in 
the August 2001 proposed Medicaid managed care rule. In the preamble to 
the August 2001 proposed rule, we indicated that materials should be 
understandable to enrollees at a fourth to fifth grade reading level, 
or at another level established by the State agency that adequately 
reflects the potential population to be enrolled. Materials should use 
an easily readable typeface, frequent headings, and should provide 
short, simple explanations of key concepts. Technical or legal language 
should be avoided whenever possible. We proposed further that 
enrollment notices as well as informational and instructional materials 
relating to enrollment take into account the specific needs of 
enrollees and potential enrollees, including furnishing information in 
alternative formats for the visually impaired and for individuals with 
limited reading proficiency. Also, in 1999, we developed and 
distributed to the State Medicaid agencies and made available to others 
a guide entitled, ``Writing and Designing Print Materials for 
Beneficiaries: A Guide for State Medicaid Agencies.'' The guide was 
produced to assist States and MCOs/PIHPs in the creation of materials 
appropriate for their Medicaid populations. We believe the guidance 
that we have provided in the August 2001 proposed rule and through this 
guide is appropriate and reflects the current state-of-the-art. Because 
there is no state-of-the-art standard to apply in measuring the 
adequacy of the MCO's/PIHP's efforts to inform enrollees, we decline to 
do so in this protocol.
    Comment: One commenter recommended that we monitor the States' 
definition of what constitutes a ``significant change'' in certain MCO 
structural and operational features to ensure the State's definition of 
``significant change'' is reasonable and fair to enrollees, and that we 
provide guidance on what parameters a State can use in setting the 
definitional standards.
    Response: The protocol addresses the extent to which an MCO/PIHP, 
as opposed to the State, complies with the requirements in the Medicaid 
managed care final rule. Section 438.10(f)(4) of the Medicaid managed 
care final rule specifies that the definition of ``significant change'' 
is the State's responsibility. It, therefore, would not be appropriate 
to include in the protocol the monitoring of the State's definition. 
Monitoring of States occurs through separate activities conducted by 
our regional offices. Further, as we stated previously, the protocol is 
not intended as a mechanism to impose additional quality standards on 
MCOs/PIHPs or States. Therefore, we do not believe it appropriate to 
provide guidance in the protocol on what parameters a State can use in 
setting the definitional standards.
    Comment: One commenter noted that the interview questions are good 
initial probes, but suggested the protocol include additional guidance 
to more fully probe the MCO's dissemination of enrollee information, 
and require interviews of providers and enrollees regarding the quality 
of the informational materials.
    Response: We specify in the protocol that reviewers should tailor 
the interviews as necessary to clarify and confirm document findings. 
We believe this direction affords the reviewers sufficient flexibility 
to more fully probe areas as appropriate. Further, we do not believe it 
is possible, given the diversity among States and MCOs/PIHPs and the 
scope of the review itself, to include in the list of potential 
interview questions probes to explore every possible problem or issue 
that might arise. Provider interviews are time and resource intensive, 
but because they offer an opportunity to secure additional information 
regarding MCO/PIHP performance, we have included them as an optional 
activity if informational needs warrant them and resources permit. We 
provide for the consideration of enrollee input by including the review 
of the results of Medicaid beneficiary surveys as accessory information 
under Activity 5.
    Comment: One commenter believes the protocol does not adequately 
address linguistic issues. The commenter recommended that the review 
confirm that MCOs collect required language information on enrollees 
and recognize non-English speakers in all transactions. The commenter 
suggested further that the protocol include the review of documentation 
regarding professional translations of written materials, and 
interviews to assess the quality of the written translations and the 
MCO's oral interpretation practices and resources.
    Response: We believe the protocol does adequately address 
linguistic issues. In Appendix B (page 79, Attachment B of the final 
protocol), among the materials to be obtained from the State, we 
include information on the language(s) that the State Medicaid agency 
has determined are prevalent in the MCO's/PIHP's geographic service 
area. On page 85, we direct the reviewer to look at marketing, 
enrollment and other informational and instructional materials relating 
to enrollment, enrollee handbooks, new enrollee materials, statements 
of enrollee rights, and other written materials routinely prepared for 
Medicaid enrollees and potential enrollees to determine whether these 
materials are available in the language(s) that have been identified as 
prevalent within the MCO/PIHP's particular service area. Further, the 
Medicaid managed care final rule at Sec.  438.204(b)(2) requires States 
to identify the primary language spoken by each Medicaid enrollee and 
provide this information to the MCO/PIHP at the time of enrollment. 
Finally, we believe requiring EQRO re-review of translated materials is 
more burdensome than appropriate and therefore have not included it in 
the protocol.
6. Enrollee-Provider Communication
    Comment: One commenter objected to the implication that by contract 
MCOs may place limits on providers' communication with enrollees about 
reproductive health services. The commenter recommended that the 
protocol include document review and interview questions to address 
whether reproductive health services are provided and whether 
restrictions are placed on provider communication. The commenter 
suggested further that for MCOs that exclude any reproductive health 
services the State monitor enrollee access to the full scope of 
services. The commenter noted a potential correlation between 
restricted access to reproductive health care services and poor 
outcomes in other women's health areas, and recommended the State 
monitor related health outcomes and comparison of rates to those of 
MCOs without restrictions.
    Response: Appendix B of the protocol (Attachment B of the final 
protocol) specifies documents for review and interview questions to 
address whether the MCO/PIHP has any moral or religious objection to 
providing, reimbursing for, or providing coverage of, a counseling or 
referral service for a particular Medicaid service or services. This 
would include reproductive health services. For counseling and referral 
services the MCO/PIHP does not cover because of moral or religious 
objections, the Medicaid managed care final rule at Sec.  
438.10(f)(6)(xii) specifies that it is the State's responsibility to 
provide enrollees with information on where and how to obtain the 
service(s). The protocol is designed to address MCO/PIHP compliance 
with the BBA regulatory standards. Consequently, State monitoring of 
enrollee access to

[[Page 3622]]

the full scope of services and State monitoring of health outcomes in 
other women's health areas for enrollees with restricted access to 
reproductive health care services, and comparison of these rates to 
those of MCO/PIHPs without restrictions is beyond the scope of the 
protocol.
7. Emergency Services
    Comment: One commenter suggested that the interview questions 
concerning inappropriate use of emergency rooms emphasize a comparison 
of their inappropriate use with access to routine and urgent care.
    Response: We agree with the commenter and have therefore expanded 
the relevant interview questions in Appendix B of the protocol 
(Attachment B of the final protocol) under Sec.  438.210 that addresses 
coverage and authorization of services to inquire about the potential 
relationship between inappropriate emergency room use and enrollee 
access to routine and urgent care.
8. Delivery Network
    Comment: One commenter recommended that the protocol, in reviewing 
the MCO's/PIHP's network of appropriate providers, consider 
specifically the providers needed to meet the needs of pregnant women, 
children and individuals with special needs, particularly those 
targeted for enrollment.
    Response: In the Medicaid managed care final rule at Sec.  438.206, 
we require the MCO/PIHP to establish a network of appropriate providers 
that considers the ``expected utilization of services, considering 
Medicaid enrollee characteristics and health care needs.'' We intend 
and expect that MCOs and PIHPs that serve pregnant women and 
individuals with special health care needs will consider their 
characteristics and needs. However, we do not explicitly identify them 
in this protocol because they are not explicitly mentioned in the 
regulation in this provision and because not all MCOs and PIHPs may 
serve pregnant women and individuals with special health care needs.
9. Access
    Comment: One commenter suggested that the review address 
transportation services to network providers and out-of-network 
providers for enrollees without access within established time and 
distance standards, and for enrollees with disabilities and special 
needs.
    Response: The regulations do not contain standards for the 
provision of transportation services to network or out-of-network 
providers, or for enrollees with disabilities and special needs. In 
addition, transportation is a service that may or may not be included 
under the MCO/PIHP contract. Therefore, in the protocol's document 
review and interview questions, we include only those transportation 
issues addressed in the regulation.
    Comment: One commenter recommended that the monitoring of access to 
out-of-network providers include a review of the procedures for 
determining when in-plan access is unavailable and out-of-network 
services are appropriate; obtaining access to out-of-network services; 
and for providing in-plan services for enrollees denied out-of-network 
access.
    Response: The protocol specifies a review of the MCO's/PIHP's 
administrative policies and procedures pertaining to the use of out-of-
network providers. Although we reference documents by generic name or 
title, we explain that what is important is the presence or absence of 
evidence to determine compliance with the specified regulatory 
provision. We anticipate reviewers will use the relevant documents to 
determine compliance with all aspects of the regulatory provision 
regarding out-of-network access including those identified by the 
commenter.
    Comment: One commenter suggested that the document review include 
policies, procedures, and criteria for determining that second opinions 
are rendered by qualified providers.
    Response: We agree with the commenter. The protocol specifies a 
review of the MCO's/PIHP's administrative policies and procedures for 
providing enrollees with a second opinion from a qualified health care 
professional. As previously indicated, although the documents are 
referred to by generic name or title, we explain that what is important 
is the presence or absence of evidence to determine compliance with the 
regulatory provision. We anticipate reviewers will use the relevant 
documents to determine compliance with all aspects of the regulatory 
provision requiring that second opinions are rendered by qualified 
providers.
    Comment: One commenter recommended that the document review related 
to direct access to women's health services be expanded to include 
materials produced by the State to inform MCOs and by MCOs to inform 
providers. The commenter suggested further that the review include 
policies and procedures for implementing direct access to these 
services.
    Response: Within the review of enrollee rights, the protocol 
specifies a review of staff and provider orientation, education, and 
training curricula and materials, and other provider and staff 
communication tools for evidence that staff and providers consider, 
among the enrollees' rights, direct access to women's health services. 
We also specify the review of the results of MCO/PIHP monitoring of 
complaints and grievances, enrollee survey or other MCO/PIHP sources of 
enrollee information to detect violations of enrollee rights, including 
the provision of direct access to women's health services. However, we 
do not include in the protocol a review of materials produced by the 
State because the protocol is a review of MCOs or PIHPs, not State 
Medicaid agencies. Review of State compliance with Federal requirements 
is carried out by our regional office staff through a separate process.
    Comment: One commenter recommended that the reviewer monitor the 
time it takes for enrollees to obtain appointments with network 
providers.
    Response: We agree with the commenter. Our protocol directs the 
reviewers to obtain the State Medicaid agency's standards for timely 
access and to review documents showing how the MCO/PIHP ensures 
compliance and continuously monitors its network providers for 
compliance with the timely access standards. The protocol lists some 
acceptable mechanisms the MCO/PIHP may use for monitoring compliance.
    Comment: One commenter suggested that inappropriate use of 
emergency rooms be evaluated according to the ``reasonable lay person'' 
standard. The commenter also recommended that the monitoring of 
emergency room use consider access to nonemergent care and follow-up 
outreach and education for enrollees using emergency rooms for 
nonemergency care.
    Response: The protocol monitors MCO/PIHP application of the prudent 
layperson standard in the regulation at Sec.  438.114. As we indicated 
in our response to a previous comment on emergency room use, we have 
added an interview question to inquire about the potential relationship 
between inappropriate emergency room use and enrollee access to routine 
and urgent care. However, MCO/PIHP follow-up outreach and education for 
enrollees using emergency rooms for nonemergency care is not a 
regulatory requirement, and it would be inappropriate to include it in 
the protocol.

[[Page 3623]]

    Comment: One commenter suggested expanding the protocol's 
activities to include the review of training curricula and materials on 
cultural and linguistic competency, including the scope and depth of 
the training, its frequency, and extent of staff attendance; the 
procedures for the translation and testing of enrollee informational 
materials; and arrangements with community-based organizations 
representing relevant ethnic groups.
    Response: We disagree with the commenter. Our protocol addresses 
the extent to which an MCO/PIHP complies with the regulatory provisions 
that implement the Medicaid managed care sections of the BBA. The 
Medicaid managed care final rule, at Sec.  438.206(c)(2), requires that 
MCOs/PIHPs participate in the State's efforts to promote the culturally 
competent delivery of services. Therefore, the protocol specifies a 
review of documents for evidence of the MCO's/PIHP's participation in 
the relevant State efforts. The inclusion of additional requirements 
not required by regulation within the protocol would be inappropriate.
10. Coordination & Continuity of Care
    Comment: One commenter recommended that the review of coordination 
and continuity of care include interview questions regarding the 
provision of any specialty care services currently not provided in-
network, and MCO efforts to make these services available in-network. 
The commenter also suggested that the interview questions be expanded 
to inquire what proportion of Medicaid enrollees with special health 
care needs have a person or entity formally designated as primarily 
responsible for coordinating their health care services.
    Response: We agree, in part, with the commenter. Consequently, we 
have added an interview question for the organization leaders to 
inquire about the provision of any specialty care services currently 
not provided in-network. We have not added questions about MCO or PIHP 
efforts to make these services available in-network because it is not 
clear whether or not it is always necessary that all specialty services 
be provided by in-network providers. We have added additional potential 
interview questions for enrollee services staff to determine what 
proportion of Medicaid enrollees with special health care needs have a 
person or entity formally designated as primarily responsible for 
coordinating their health care services.
    Comment: One commenter believes the protocol should differentiate 
between gatekeeping activities that are involved with utilization 
control and care coordination and case management functions that are 
related to supporting service access and coordination. The commenter 
believes further that reviewers should consider the MCOs' scope of 
responsibility for EPSDT case management, and how these services are 
provided or referrals are made.
    Response: We agree with the commenter that a State may want to 
differentiate between care coordination models. In so doing, a State 
may decide to explicitly address care coordination for EPSDT care 
management. We specify in the protocol that MCOs/PIHPs may establish 
different coordination mechanisms, and in monitoring for compliance 
with the requirements for care coordination, direct the reviewers to 
obtain the State's requirements for MCO/PIHP care coordination 
programs.
    Comment: One commenter recommended that the interview protocol 
address how and who conducts the MCOs' health screens; how the MCO 
assesses enrollee needs and determines if the provider is qualified to 
perform the assessment; how enrollees access case management services; 
how an enrollee's need for a treatment plan is determined; and how the 
providers are informed of the process. The commenter also suggested 
additional interview questions to address the number of treatment plans 
developed by categories of individuals, the number of denied requests 
for treatment plans and the reason for denial, and the number of 
treatment plans denied.
    Response: The protocol includes interviewer questions for the case 
managers and care coordinators and for the enrollee services staff 
regarding the implementation of health screens, the conduct of health 
assessments for Medicaid enrollees, processes for care coordination, 
and procedures to determine how an enrollee's need for a treatment plan 
is determined. The protocol's interview questions for the provider/
contractor services staff probe how providers are made aware of and are 
involved in procedures for assessments, treatment planning, and care 
coordination. We agree with the commenter regarding the need to explore 
the MCO's/PIHP's treatment planning. We have revised the protocol to 
include a series of questions for the case managers and care 
coordinators concerning the number of treatment plans developed, the 
number of denied requests for treatment plans and the reason for 
denial, and the number of treatment plans denied. However, our revision 
will not include a review of the treatment plans by categories of 
individuals. We do not require specific categories and, therefore, have 
no standard against which to measure the MCO's/PIHP's performance.
11. Prior Authorization
    Comment: One commenter believes the protocol should include a 
review of prior authorization procedures and policies and a 
determination of their reasonableness, reflection of good medical 
practice, and timely application. The commenter suggested reviewers 
monitor the number of and reasons for delayed expedited requests, and 
the health consequences associated with prior authorization delays and 
denials of expedited authorizations. The commenter further believes the 
MCOs' informal communications with providers should be monitored, 
including the handling of provider telephone inquiries, resulting 
changes to the course of treatment, and provision of enrollee notice 
and appeal rights.
    Response: We agree with the commenter regarding the need to 
determine compliance with the requirement for timely prior 
authorization decisions, and therefore have included in the protocol 
document review and interview questions to determine compliance. 
However, the regulations include no standards for the reasonableness of 
the policies and procedures or for their reflection of good medical 
practice; these issues are therefore beyond the scope of the protocol 
that is designed to assess compliance with the Medicaid managed care 
regulatory requirements.
    We also agree with the commenter's suggestion to review the number 
and reasons for delayed expedited requests. We have revised the 
document review for service authorizations to include the review of 
tracking logs or other authorization record-keeping documents to 
address number and reasons for delayed expedited requests.
    We do not agree with the suggestion to monitor health consequences 
associated with prior authorization delays and denials of expedited 
authorizations. We believe that determinations on whether health 
consequences were due to authorization delays or denials, or to the 
normal progression of the enrollees' health condition would be 
subjective. Further, States are required to maintain records of 
grievances and appeals and review this information as part of the State 
quality strategy. If enrollees' health outcomes are adversely affected 
by the MCO's/PIHP's handling of service authorization requests, this 
should become evident to the State through this grievance and appeals 
review.

[[Page 3624]]

Therefore, we have not added this review activity to the protocol. We 
are also not requiring the EQR to review informal communication with 
providers. Informal communications by their nature do not routinely 
involve written documentation, and we believe it would be burdensome to 
require reviewers to monitor verbal exchanges.
    Comment: One commenter recommended that the interview questions 
address the MCO's process and criteria for extensions of the standard 
14 days for regular prior authorization decisions.
    Response: We disagree with the commenter: timeframes for standard 
prior authorization decisions are established by the State. The 
protocol addresses compliance with the standard requirements in the 
Medicaid managed care final rule. Because extensions to State-
established timeframes for standard authorization decisions is not 
included in the regulations addressing enrollee services, it would be 
inappropriate to include it in the protocol.
12. Enrollment & Disenrollment
    Comment: One commenter believes that the protocol should provide 
guidance to reviewers concerning when it is appropriate for enrollees 
to use the MCO's grievance process before the State makes a 
determination on the enrollee's disenrollment request.
    Response: The Medicaid managed care regulation does not specify the 
circumstances under which it is appropriate for enrollees to use the 
MCO's/PIHP's grievance process before the State makes a determination 
on the enrollee's disenrollment request. The protocol is designed to 
address MCO/PIHP compliance with the regulatory provisions and is not 
intended as a vehicle for either specifying additional requirements or 
providing guidance.
    Comment: One commenter recommended the protocol include comparisons 
of MCO disenrollment rates and default or automatic enrollment rates 
because high rates can signify quality or access problems in the former 
instance and information deficits in the latter.
    Response: While we agree with the commenter that disenrollment 
rates and default or automatic enrollment rates may be correlated, we 
do not agree that a comparison of rates alone will suffice. Instead, we 
have revised the protocol to specify that the document review include 
the MCO/PIHP disenrollment rates, and that the review of the 
disenrollment sample determine if a relationship exists between the 
enrollees requesting disenrollment and enrollees enrolled in the MCO/
PIHP automatically or by default.
13. Grievance System
    Comment: One commenter suggested that the protocol include review 
of policies and interview questions to ensure the MCO does not deter 
enrollees from requesting fair hearings. The commenter recommended 
further that the reviewer consider the number of grievances and fair 
hearings versus the population served, and determine whether grievances 
are held in suspense at certain levels of the review process or 
enrollees are deterred from filing or pursuing grievance or fair 
hearing requests. The commenter also suggested the reviewer convene 
focus groups concerning how the grievance system is working.
    Response: We believe the protocol, in the portion addressing review 
of documents related to enrollee grievances, appeals and State fair 
hearings, addresses the MCO/PIHP compliance with the regulatory 
provisions, and in so doing, ensures that the MCO/PIHP does not deter 
enrollees from requesting fair hearings or pursuing grievance or fair 
hearing requests. The protocol specifies a review of logs, registries, 
or other MCO/PIHP documentation of appeals, grievances, and requests 
for State fair hearings made by Medicaid enrollees. Further, States are 
required to maintain records of grievances and appeals and review this 
information as part of the State quality strategy. If grievances are 
held in suspense, this should become evident to the State through this 
grievance and appeals review. We believe that focus groups, like 
provider and consumer interviews, are time and resource intensive. 
Therefore, we include consideration of other accessory information, 
such as beneficiary surveys that may offer information on how the 
grievance system is working but do not require in this protocol that 
the reviewer convene focus groups.
    Comment: One commenter believes that notice of action requirements 
(for denial, reduction or termination of services) apply to all types 
of plans and asked that this be clearly stated in the protocol. The 
commenter further suggested the protocol include interview questions to 
probe the actions that trigger notices required by due process of the 
law, and a review of the MCO's notices to determine that the notices 
comply with the legal requirements for adequate notice of hearing 
rights, assure enrollees the care they receive will not be affected 
because a grievance has been filed, are in languages prevalent in the 
service area, and clearly specify the action the MCO is taking.
    Response: The protocol is designed to specifically determine MCO 
and PIHP compliance with provisions in the Medicaid managed care final 
rule, regardless of whether or not the provisions apply to other types 
of managed care plans. We have, therefore, addressed these two entities 
in assessing compliance with the requirements concerning notice of 
action. We believe a document review is more effective for this issue 
than interview questions as an approach to compliance determination. 
Furthermore, the protocol includes the review of a sample of MCO/PIHP 
notices to determine the extent to which notices include the legal 
requirements for adequate notice of hearing rights and specify the 
action the MCO/PIHP is taking. We agree with the commenter and have 
expanded this review to determine that notices include assurances that 
enrollees will not be treated differentially, and are in languages 
prevalent in the service area. We believe that by reviewing a sample of 
beneficiaries that have been denied services and the reasons for 
denials, reviewers will identify those actions that trigger notices 
required by due process of the law.
    Comment: One commenter believes the protocol fails to ascertain the 
extent to which enrollees have realistic access to the grievance 
process. The commenter recommended that the protocol include interview 
questions concerning the process and frequency by which enrollees are 
informed of the grievance procedures. The commenter also suggested 
reviewers monitor the timeliness of grievance processing, interview 
enrollees regarding the free exercise of their rights, and review the 
MCO's procedures for supplying translation and interpretation services 
during the grievance process.
    Response: As we noted in the prior response, we believe a document 
review is more effective than interview questions in determining 
compliance with these provisions. The protocol includes the review of 
the MCO/PIHP's administrative procedures and policies as well as a 
sample of MCO/PIHP notices. We agree with the commenter that reviewers 
should monitor the timeliness of grievance processing and review the 
MCO's/PIHP's procedures for supplying translation and interpretation 
services during the grievance process. Therefore, we have specified 
that in reviewing the sample of notices, the reviewer should determine 
the timeliness of grievance processing, and have included a review of 
the MCO's/PIHP's procedures for supplying

[[Page 3625]]

translation and interpretation services during the grievance process. 
However, since enrollee interviews are time and resource intensive and 
beneficiary survey results are specified for consideration as accessory 
information, we have not included this activity.
    Comment: One commenter recommended reviewers interview enrollees to 
determine how they are informed of the right to request continuation of 
benefits pending resolution of an appeal or fair hearing, and whether 
continuing benefits were received when requested. The commenter also 
suggested that the reviewers compare the MCO's policies with the 
enrollees' experiences.
    Response: As noted previously, enrollee interviews are time and 
resource intensive and are therefore not a review activity included in 
the protocol. Instead, reviewers are directed to review the results of 
beneficiary surveys as accessory information. The protocol also 
specifies a review of the MCO/PIHP administrative policies and 
procedures, and the review of a sample of notices, to determine the 
extent to which enrollees are informed of their right to request 
continuation of benefits pending resolution of an appeal or fair 
hearing. The findings from the document reviews can then be compared to 
the survey results as suggested by the commenter.
    Comment: One commenter disagreed with the protocol not permitting 
the combination of case manager and care coordinator interviews with 
other interviews. The commenter further recommended the protocol 
include interview questions for case managers and care coordinators on 
the enrollees' process for accessing case management services to ensure 
consistency with MCO policies, the procedures for interfacing with 
carved-out or other services not covered by the MCO, and the ease of 
accessing specialist care.
    Response: The protocol specifies that the case manager's and care 
coordinator's interviews may be combined with the Medical Director 
interview or the Utilization Management interview. This option is 
consistent with the process used by private accrediting bodies and in 
the Medicare program reviews. The protocol specifies potential 
interview questions for case managers and care coordinators to confirm 
MCO/PIHP compliance with the regulatory requirements pertaining to 
enrollee rights, service access, and coordination and continuity of 
care. However, if issues arise during the document review concerning 
the process for accessing case management services, for interfacing 
with carved-out or other services not covered by the MCO, or the ease 
of accessing specialist care, reviewers are directed to explore them 
during the interviews. We believe this direction affords the reviewers 
the flexibility necessary to appropriately tailor the review activity 
to the structure, operations, and circumstances identified for each 
MCO/PIHP. Further, we do not believe it is possible, given the 
diversity among States and MCOs/PIHPs and the scope of the review 
itself, to include in the list of potential interview questions probes 
to explore every possible problem or issue that might arise.
    Comment: One commenter believes that in collecting accessory 
information it is important to consider non-Medicaid enrollee survey 
results and compare these to the Medicaid results to ensure all 
enrollees are receiving the same level of care.
    Response: We believe there are numerous analyses of EQR-related 
activities that can be undertaken. Specifically, the results of 
compliance monitoring, encounter data, and performance measurements can 
all be compared, contrasted, analyzed, and correlated. We do not 
believe the Federal government can or should specify a single set of 
analyses that will yield the most useful information for all States and 
MCOs/PIHPs. We believe that States will choose their EQROs on the basis 
of their demonstrated competence in quality review and analysis, and we 
defer to the State's decisions about the lines of inquiry EQROs should 
pursue regarding all EQR-related data, including surveys of Medicaid 
enrollees and possible comparisons to Medicare enrollees, commercial 
enrollees, and SCHIP enrollees.

C. Protocols for Calculating or Validating Performance Measures

    Comment: One commenter asked that clarification be provided 
regarding the collection and validation of performance measures. The 
commenter is concerned that there is no description of essential EQRO 
activities to ensure that the performance measures being used by the 
State are scientifically sound, meaningful, valid, and reproducible. 
The commenter does not believe that the collection methodology outlined 
in the protocols will ensure valid and reliable measures. The commenter 
recommended that we take steps to ensure that EQROs use only evidence-
based performance measures.
    Response: We disagree with the commenter. The protocols outline a 
methodology to be used in the validation or calculation of performance 
measures to ensure that valid and reliable measures are calculated or 
to determine the extent to which valid and reliable measures have been 
calculated by the MCO/PIHP. The protocols were designed to be 
consistent with approaches used by NCQA and Medicare QIOs but to also 
describe how to validate or calculate measures such as those found in 
HEDIS as well as those developed by States or other groups or 
organizations. We advocate the calculation of measures that have been 
tested and accepted in the private and public sectors but provide 
States with the flexibility to develop measures or use measures 
developed by others that meet their program needs.
    In addition to specifying essential activities to be conducted as 
part of performance measure validation or calculation, we have provided 
an Appendix to this protocol that provides guidance on how to assess an 
MCO's or PIHP's underlying information system (IS) to ensure that valid 
and reliable data are used in the calculation of the performance 
measures. The IS assessment may be conducted as part of this protocol 
by the EQRO validating or calculating the performance measures, or the 
EQRO may review an assessment conducted by another party.
    Comment: One commenter believes that States have already invested 
substantial resources in establishing systems to carry out performance 
measurement activities and that it is not clear how these established 
systems can be adapted easily to meet the requirements of the 
protocols.
    Response: Because the essential components of the protocols are 
accepted practice in both the public and private sector, we expect that 
States will not have to significantly adapt their approaches to 
performance measurement. The performance measures protocols are to be 
used for validating measures calculated by the MCO or PIHP as required 
by the Medicaid managed care final rule or for calculating additional 
measures as directed by the State. State approaches to performance 
measurement might vary but we expect States to require the essential 
components of the protocol for performance measurement activities--
review of MCO/PIHP data management processes, evaluation of compliance 
with specifications for performance measures, and verification of 
performance measurement.
    Comment: One commenter believes this protocol is outdated and 
suggested we reference current industry tools. Another commenter argued 
that the performance measure validation process is heavily biased 
toward proprietary systems entities developed in the

[[Page 3626]]

business of accreditation. The commenter believes this bias limits 
flexibility in the process and promotes a narrow view of performance 
measurement and jeopardizes State's ability to be innovative in 
performance measurement.
    Response: One reason we did not include the protocols in a 
regulation was because we recognize that the protocols will need to be 
updated as the state-of-the-art in quality assessment and improvement 
changes. However, we believe that the activities listed in the protocol 
are still those in current use in the industry. Further, to be in 
compliance with the EQR rule, States only need to ensure that our 
protocols or those consistent with ours are used.
    In addition, we do not agree that the protocol is biased toward 
proprietary systems. We used three sources to develop the performance 
measures protocols (that is, NCQA's HEDIS validation protocol, IPRO 
documents, and documents from the MEDSTAT group). We identified 
activities common to these tools and incorporated those activities to 
ensure valid and reliable methods are used when calculating or 
validating performance measures. Only one of these tools was developed 
by an organization that is in the business of accreditation, and we do 
not agree that the performance measures protocol limits State 
flexibility in the performance measures development process. We provide 
States with the flexibility to use established measures or to develop 
their own measures. We recommend, however, when States choose to 
develop or use measures not widely used in the private and public 
sector, that these measures should be evidenced-based and tested.
    Comment: Several commenters believe the process described for 
validating performance measures is bureaucratic and administratively 
burdensome. The commenters state that they do not understand the value 
of interviewing MCO staff and believe annual onsite review is not 
necessary and is burdensome.
    Response: The process in the protocols for validating performance 
measures is consistent with the process used in the private sector and 
the Medicare program. We drew from established tools in the development 
of these protocols. The protocol includes interviewing MCO and PIHP 
staff in addition to reviewing MCO/PIHP documentation of how 
performance measures are produced. The purpose of interviewing staff is 
not to obtain information that can otherwise be obtained from 
documentation. It is to supplement and confirm information as needed. 
In the protocol, interviews of MCO/PIHP personnel are identified as an 
effective mechanism to understanding an MCO's/PIHP's IS and its 
application to performance measurement. While much information can be 
obtained by reviewing an MCOs/PIHPs internal documents describing its 
IS, we believe that interviews with MCO/PIHP staff can be a helpful 
adjunct to the review of IS documents in understanding the issues the 
MCO/PIHP has with respect to ISs and how it affects the MCO's/PIHP's 
production of performance measures.
    Comment: One commenter argued that some States calculate and report 
MCO-level performance measures and therefore, much of what is contained 
in the calculating performance measures protocol is not applicable to 
MCOs, but is applicable to the State.
    Response: We recognize that States may have MCOs and PIHPs submit 
encounter data to them instead of performance measures and, therefore, 
the State may be the entity calculating the performance measure. We 
have allowed for this in the quality assessment and performance 
improvement program requirements specified in Sec.  438.240 of the 
Medicaid managed care final rule. However, regardless of who calculates 
the performance measures, MCO and PIHP-level performance measures must 
be calculated as required by the Medicaid managed care final rule and, 
if calculated by the MCO/PIHP, must be validated to provide information 
for the EQR function. We have added clarifying language under Sec.  
438.358(b)(2) to recognize that States may be calculating the MCO/PIHP 
performance measures and in this circumstance the State would provide 
the information obtained from this activity to the EQRO for the EQR 
function.
    Comment: One commenter suggested combining the validating 
performance measures protocol and the calculating performance measures 
protocol to reduce the length and complexity of the two protocols.
    Response: We purposefully provided separate protocols for each EQR-
related activity. Even though some of the protocols are variations on a 
theme (for example, validating performance measures and calculating 
measures) we wanted to provide stand-alone documents for each activity. 
In addition, though the protocols are variations on a theme, the 
activities do differ somewhat and we believe the clearest way to 
present the information is in separate documents.
    Comment: One commenter argued that the 30 sample medical record 
review recommended in the protocol for performance measures not 
calculated with administrative data only will add tremendous cost, is 
needlessly intrusive, and is very time consuming.
    Response: This aspect of the protocol illustrates what we mean when 
we say that States must use protocols that are consistent with (but not 
identical to) our protocols. In this protocol, onsite Activity 4 is the 
``Assessment of Processes to Produce Numerators.'' To be consistent 
with our protocol, the EQRO must perform this activity (that is, assess 
the MCOs' or PIHPs' processes to produce the performance measure 
numerator). In our description of Activity 4, we describe how this 
activity is to be conducted and state that this activity should include 
a review of a sample of the medical records used to determine the 
numerator. Thirty medical records is the number that was included in 
the private sector protocols we reviewed. However, EQROs may use 
another sample size and still be consistent with our protocol. Our 
protocol endorses the policies found in private sector protocols, that 
require a sufficient number of medical records be reviewed to validate 
a reported numerator for a given performance measure. As stated 
previously, however, activities used to provide information for the EQR 
must be conducted ``consistent with'' our protocols. ``Consistent 
with'' means that the protocols used contain all of the activities and 
steps included in our protocols. How EQROs and States implement the 
activities and steps is left to their discretion.
    Comment: One commenter suggested we add lab data as a data source 
to calculating performance measures numerators (page 8, item 4).
    Response: We agree with the commenter and have added laboratory 
data as a possible data source for calculating performance measures.
    Comment: One commenter suggested some editorial changes.
    Response: We have made editorial changes that were recommended 
where we thought appropriate and helpful.
    Comment: One commenter suggested on page 15 we add ``place of 
service'' to the list of claims and encounter data elements to be 
assessed when assessing the integrity of the MCO's/PIHP's IS.
    Response: We agree with the commenter and have added place of 
service to the list of claims and encounter data elements that may be 
used to conduct performance measurement.

[[Page 3627]]

D. Protocols for Conducting or Validating Performance Improvement 
Projects and Conducting Focused Studies

    Comment: One commenter believes all the activities in this protocol 
are reasonable.
    Response: We agree and retain the activities in the protocol.
    Comment: One commenter asked for clarification of why the protocol 
for conducting performance improvement projects was developed. The 
commenter questioned the value of this protocol since the EQRO is not 
affiliated with any MCO and has no way to implement performance 
improvement initiatives affecting the actual delivery of care. The 
commenter recommended eliminating this protocol.
    Response: This protocol was developed to provide EQROs and States 
guidance on the activities required when conducting performance 
projects as an optional EQR-related activity that qualifies for 75 
percent FFP. A State may itself, through another State contractor, or 
through the EQRO, have additional performance improvement projects 
conducted other than those required to be conducted by the MCO/PIHP 
under Sec.  438.240(b)(1) of the Medicaid managed care final rule and 
Sec.  438.358(b)(1) of this rule. As long as the project is conducted 
consistent with the protocol, the information can be provided to the 
EQRO and be included as part of the EQR function. If the State itself 
or other State contractor conducts the activity, the State would not 
qualify for the 75 percent enhance match. If the EQRO conducts the 
performance improvement project, the State could claim the enhanced 
match. We developed separate protocols for the conduct of performance 
improvement projects and the validation of performance improvement 
projects to have stand-alone documents.
    Comment: One commenter recommended that the focused study protocol 
be combined with the validating performance improvement projects 
protocol. The resulting protocol should be an optional protocol to be 
used at the State's discretion. One commenter recommended that the 
validating performance improvement projects and conducting performance 
improvement projects protocols be combined.
    Response: We have developed separate protocols for validating and 
conducting performance improvement projects and for conducting a 
focused study of health care quality in order to provide stand-alone 
documents for each of the EQR-related activities. The focused study 
protocol and the conducting performance improvement projects protocol 
are to be used at the State's discretion if it decides to include 
information from these optional EQR-related activities as part of the 
EQR. In contrast, validating performance improvement projects conducted 
by MCOs/PIHPs is a mandatory activity. Although these protocols have 
much in common, there are some differences and we believe it is more 
helpful to the readers and users of the protocols to present these 
similar, but different activities in separate documents.
    Comment: One commenter argued that the focused study protocol is 
biased towards proprietary measurement systems, that we advocate the 
use of indicators that are generally used in the public health 
community such as those developed by NCQA and the Foundation for 
Accountability (FACCT). The commenter recommended that the protocol be 
neutral in tone and approach the topic of performance measure selection 
from the perspective of State preferences and existing or evolving 
State-specified systems.
    Response: We agree with the commenter that we advocate the use of 
performance indicators that are generally used in the public health and 
managed care industry. This is because these measures have been tested 
for validity and reliability and are widely accepted in the public and 
private sectors. However, we also, in the performance measures (both 
conducting and validating) and focused study protocols state that other 
indicators may be used. We recommend that these indicators be developed 
on the basis of current clinical practice guidelines or clinical 
literature derived from health services research or findings of expert 
or consensus panels.
    Comment: One commenter suggested we add appointment availability 
studies, network assessment studies, open-closed panel reports, member 
and provider satisfaction survey data, and provider language reports as 
potential sources of information for selecting study topic for 
performance improvement projects or focused studies of health care 
quality.
    Response: We agree with the commenter and have revised the 
potential sources of supporting information section, under Activity 
``Selecting the Study Topic,'' in the performance improvement projects 
(conducting and validating) and focused studies protocols to include 
the following: data on appointments and provider networks such as 
access, open and closed panels, and provider language spoken. Data from 
surveys was already included in this section in each protocol.
    Comment: One commenter suggested we add a discussion of service 
needs for special needs populations to the list of methods for 
selecting the study topic.
    Response: We recommend in this section that topics should reflect 
high-volume or high-risk conditions of populations served, including 
populations with special health care needs such as children in foster 
care, adults with disabilities, and the homeless. We further state that 
although these populations may be small, their special health care 
needs place them at high risk. We believe these provisions address the 
commenter's concerns and that no change is needed.
    Comment: One commenter believes that our rationale for reliable 
data collection only addresses clinical data collection. The commenter 
suggested we add a section for service studies such as appointment 
availability and that methods to implement this include review of 
appointment books, and ``secret shopper'' techniques when someone calls 
to make an appointment. These kinds of indicators require scripts and 
very clear definitions of items such as acute care, emergent care, and 
routine care.
    Response: We agree with the commenter that we did not include a 
discussion on data collection issues when using nonclinical data. We 
have added a paragraph in the performance improvement projects (both 
conducting and validating) and focused studies protocols to address 
this issue.

E. Protocol for Validating Encounter Data

    Comment: One commenter stated that the protocol does not allow for 
the fact that encounter data may be used for risk adjusted payment and/
or other utilization data analysis purposes.
    Response: Accurate and reliable encounter data is crucial to 
performing any analysis of utilization data, and in particular to the 
development of capitated payments which are based on utilization data. 
This protocol specifies processes for assessing the completeness and 
accuracy of the encounter data MCOs and PIHPs submit to the State. We 
believe this protocol for validation of encounter data accommodates the 
multiple purposes for which encounter data are used.
    Comment: One commenter stated that this protocol is long, detailed, 
needlessly prescriptive and biased toward the MEDSTAT and HEDIS models. 
The commenter also stated that since States generally have encounter 
data validation processes in place, this

[[Page 3628]]

protocol will be redundant and should therefore be dropped, reformatted 
as technical assistance or combined with other protocols to reduce the 
length and complexity of the protocols.
    Response: In developing this protocol (as with all the protocols) 
we instructed our contractor to draw from existing protocols that have 
been tested and used in the public and private sectors, and that are 
consistent with current industry practice. The elements contained in 
the MEDSTAT and HEDIS tools are consistent with other validation 
processes reviewed, and contain generic activities and steps that 
include the essential components of a methodologically sound review of 
encounter data. By requiring protocols that are ``consistent with,'' 
rather than ``identical,'' we believe that we have allowed for State 
flexibility while ensuring a minimum standard of quality. Since the 
validation of encounter data is an optional EQR-related activity, 
States have the option to conduct this activity or not. Consequently, 
we do not believe this protocol is redundant, needlessly prescriptive, 
or biased.
    Comment: One commenter believes this protocol should address State 
data issues and improvements that may impede the ability of MCOs and 
PHPs to improve their data quality. These issues include the inability 
of the State to receive MCO and PHP data, unclear data specifications 
to MCOs and PHPs, and State policies and procedures.
    Response: Section 4705(a)(2) of the BBA specifies that EQR be a 
review of MCOs. Therefore, these protocols focus on MCOs and PIHPs, not 
on the State. State Medicaid agencies have available to them a variety 
of approaches that use contractors to strengthen their Medicaid 
Management Information System (MMIS). Additionally, we have funding 
opportunities that assist States with improvements to their MMIS. We, 
therefore, are not modifying this protocol to address State Medicaid 
agency data issues.
    Comment: One commenter asked for clarification about the purpose of 
the chart on page 11, including how the categories were decided upon, 
and who will calculate the elements.
    Response: The ``Acceptable Error Rates Specifications and 
Identified Areas of Concern Form,'' is meant to serve as an example of 
a tool that an EQRO can use when assessing rates of accuracy and 
completeness for each data field. This tool can be used at the State's 
or EQRO's discretion. It may be adapted to meet individual State 
standards, or a State or EQRO may decide to develop a similar tool. Its 
purpose is to illustrate that States need to specify what error rate 
they will determine to be acceptable for the various types of encounter 
data to be submitted to them. The categories of ``encounter type'' were 
determined by the subcontractor that developed this protocol based on 
its extensive experience as a contractor to us and State Medicaid 
agencies on the production, assessment, and improvement of encounter 
data. The acceptable error rates should be specified by the State.
    Comment: One commenter recommended against an analysis of mandatory 
fields (page 16) because these items are generally mandatory and an 
MCO's submission would not be accepted if any of the fields were not 
complete.
    Response: We do not agree that an MCO's/PIHP's submission would not 
be accepted if any of the fields were not complete. State Medicaid 
agencies determine the acceptable levels of missing, surplus, or 
erroneous data. States also determine the standards for encounter data 
accuracy and completeness, to which encounter data submitted by MCOs 
and PIHPs will be compared. This protocol recommends that the encounter 
data validation process analyze and interpret the data in submitted 
fields to determine if the information is of the type that was 
requested by the State Medicaid agency, and if the values are valid and 
reasonable.
    Comment: One commenter believes that because an MCO does not 
participate in or control the process of documenting the service in the 
medical record and subsequent billing that is based upon the medical 
record, there is no possibility for payor misbehavior.
    Response: This protocol specifies processes for assessing the 
completeness and accuracy of encounter data MCOs/PIHPs submit. The 
protocol references reviews of medical records as an activity that is 
conducted to verify the accuracy of the automated data submitted, using 
the medical record as the point of reference. Payor misbehavior is not 
the issue. The issue addressed by this protocol is the accuracy of the 
information a provider submits, through the MCO/PIHP to the State, and 
the extent to which the MCO/PIHP has procedures in place to promote the 
accuracy and completeness of the data submitted by their providers.
    Comment: One commenter believes the acceptable error rates form 
(page 5) is not information that can be assessed during an onsite 
visit.
    Response: The Acceptable Error Rate form is a tool that can be used 
by the State or EQRO to document whether the MCO/PIHP has exceeded the 
acceptable error rate for each encounter type, and whether any concerns 
have been raised that trigger the need for further investigation. The 
protocol does not specify at what location (State Medicaid agency 
offices, MCO or PIHP offices, or EQRO offices) compliance with 
acceptable error rates is to be determined. The location where this 
form is to be constructed or used is to be determined by the State.
    Comment: One commenter suggested that the protocol address rejected 
data.
    Response: Activity 3, ``Analyze Electronic Encounter Data for 
Completeness and Accuracy,'' represents the core of the process the 
EQRO will use to test the validity of the encounter data. Activity 3 is 
designed to yield information about the general magnitude of missing 
encounter data, and should identify problems in the MCO's/PIHP's 
process for compiling and submitting encounter data. Rejected data 
should be included in the evidence of and reasons for an MCO's/PIHP's 
inability to submit encounter data. Additionally, Appendix Z 
(Information Systems Capabilities Assessment) asks what happens to the 
encounter if one or more required fields are missing, incomplete, or 
invalid.
    Comment: One commenter suggested that the protocol address 
additional significant issues in performing data accuracy assessments. 
The commenter further recommended that it be clear before proceeding if 
the data are pre- or post-edits and whether they are from the MCO, the 
State, or from the State's data warehouse.
    Response: We do not understand what the commenter is referring to 
when suggesting that the protocol address additional significant issues 
in performing accuracy assessments. In response to the second comment, 
the data that the protocol addresses is MCO/PIHP level data, and where 
the data resides is unique to each State. The protocol addresses 
encounter data submitted by the MCO/PIHP to the State. Therefore, the 
data would include any edits made by the MCO/PIHP. The State will need 
to identify to the EQRO the extent to which it has performed any edits 
of the data submitted by the MCO/PIHP.
    Comment: One commenter suggested that the protocol address 
benchmark data that can be used to help determine data completeness.
    Response: The use of benchmarks is discussed in a number of the 
Steps in Activities 2 and 3. The protocol does not specify exact 
benchmarks that are to be used because benchmarks should be

[[Page 3629]]

tailored to each State's status with respect to the accuracy and 
completeness of its encounter data. The protocol instead discusses how 
the EQRO should use benchmarks for testing the quality of data. 
Additionally, the protocol indicates the source for some benchmarks, 
and in some cases, provides instructions for EQROs to develop certain 
benchmarks.
    Comment: One commenter suggested that the protocol address 
incorporation of vendor data in reporting to the State.
    Response: We agree that vendor data should be included when 
reporting to the State. That is why we reference the importance of 
vendor data when assessing the MCO's/PIHP's capability to produce 
accurate and complete encounter data in Activity 2. Activity 2 directs 
the EQRO to conduct an IS assessment that is consistent with the 
process described in Appendix Z. Appendix Z includes as elements that 
impact the accuracy and completeness of encounter data, the MCO's/
PIHP's data submission policies, and the contract requirements for 
vendors and contractors.

F. Information Systems Capabilities Assessment (Appendix Z)

    Comment: One commenter believes the level of detail required in the 
information systems capabilities assessment (ISCA) tool is excessive. 
The commenter does not believe that the reviewer should have the option 
of asking for the source code for a variety of computer and report 
programs. Moreover, the commenter stated that MCOs do not necessarily 
have the source code because that information may be proprietary and 
may be the property of a vendor.
    Response: We do not agree that the ISCA tool requires an excessive 
level of detail. A number of public and private sector protocols and 
tools were examined to promote consistency between this assessment and 
similar public and private sector activities. We also disagree with the 
comment that the reviewer does not need the source codes used to 
perform various calculations, and because these codes are proprietary 
the MCO/PIHP would not have access to this documentation. The source 
codes referred to in the protocol are codes used in the programs 
written by MCO/PIHP staff or by their contractors to calculate 
continuous enrollment or other calculations using MCO/PIHP 
administrative data. Consequently, whenever the accuracy of 
calculations performed by the MCO/PIHP impact on other aspects of the 
quality measurement; for example, performance measures, the EQRO will 
require source codes to validate the accuracy of those calculations. 
These source codes should, therefore, be available to the MCO/PIHP.
    Comment: One commenter believes the onsite activities under this 
Appendix probe policies and procedures not subject to regulation and 
that they are not relevant to the State MCO contract.
    Response: We disagree with the premise that the policies and 
procedures related to the MCO/PHP ISCA are not subject to regulation. 
This Appendix relates to three different regulatory provisions. Under 
Sec.  438.242 of the Medicaid managed care final rule, the State must 
ensure, through its contracts, that each MCO/PIHP maintains an IS that 
accurately and completely collects, analyzes, integrates, and reports 
data on utilization, enrollment and disenrollment. Additionally, Sec.  
438.240 stipulates that the State must require MCOs/PIHPs to have an 
ongoing quality assessment and improvement program for which accurate 
and complete data is an essential element. Further, in Sec.  438.350 of 
this final rule, each State is required to provide its EQRO information 
obtained through methods consistent with these protocols. In our 
contractor's review of private sector industry and Medicare practices, 
it was determined that an assessment of an MCO's/PIHP's IS is an 
essential component of validation of encounter data and performance 
measurement.
    Comment: One commenter believes that this Appendix is outdated and 
suggested the encounter data protocol should reference current industry 
available tools.
    Response: When we started developing the protocols we used the most 
recent version of the public and private sector tools referenced. These 
private and public sector tools have since been updated. However, 
because we developed the protocols as generic activities and steps to 
be used in the conduct of the EQR-related activities, we do not agree 
that the protocols are outdated. Furthermore, in this final rule we 
allow for use of other protocols, as long as they are consistent (that 
is, contain the activities and steps identified in these protocols) 
with those we have developed.
    Comment: One commenter believes that States may routinely assess 
MCO IS capabilities and in these cases this protocol is of limited 
applicability.
    Response: To avoid duplication, in all the protocols calling for an 
ISCA, we state that the EQRO may use information about the MCO/PIHP 
ISCA obtained from an ISCA conducted by another party as part of 
another review such as the validation of performance measures, 
validation of encounter data, or a review for compliance with 
standards. If the ISCA was performed by another party as part of 
another review, the State or EQRO should obtain a copy of the 
assessment, review it to determine if the findings are current, 
consistent with this Appendix, and where appropriate, seek more recent 
or additional information. If a recent assessment has not been 
conducted, an ISCA that is consistent with this Appendix should be 
conducted.

G. Protocols for Administering or Validating Surveys

    Comment: One commenter argued that the protocol for administering a 
survey is very prescriptive and the value of such a detailed protocol 
is questionable particularly when States choose to follow the 
recommended CAHPS survey method. The commenter asked us to clarify how 
much latitude there was to follow the CAHPS methodology.
    Response: The administration of validation of consumer or provider 
surveys of quality of care are optional EQR-related activities. If a 
State elects to have its EQR perform these activities and to qualify 
for the 75 percent enhanced match, our protocol or a protocol 
consistent with ours must be used. Our protocol includes generally 
accepted practices of survey design and implementation. We relied upon, 
but condensed, generally accepted principles of survey design and 
administration discussed in textbooks and other health services 
publications. Although many States use CAHPS surveys (and the CAHPS 
survey methodology would meet the requirements of this protocol) it was 
necessary to put forth this protocol to cover those instances when 
States desired to use a survey other than a CAHPS survey.
    Comment: One commenter asked us to clarify the distinctions between 
the two survey protocols.
    Response: The first protocol applies to the situation in which the 
State or its agent administers a survey, that is, designs and/or 
conducts a survey. Administration of a survey may include the design 
and implementation of a new survey or the modification of an existing 
survey and its implementation.
    The second protocol applies to the situation in which the State or 
its agent validates the use of a survey administered or conducted by 
another party. The process of validation is necessary to ensure that 
the survey results are both reliable and valid. In

[[Page 3630]]

this protocol, survey validation is limited to a review of the survey 
procedures. The validation process does not include collecting survey 
data anew from respondents to verify their responses.
    Comment: One commenter believes that beta testing all surveys and 
the additional questions to members and providers would be time 
consuming and cost prohibitive.
    Response: The protocols do not suggest beta testing of all surveys. 
Instead, they acknowledge the commitment of time and resources and the 
demands on survey respondents that make such an activity infeasible. 
The protocol suggests that survey validation be limited to a review of 
survey procedures.

H. Other Appendices (Attachments to Final Protocols)

    Comment: One commenter recommended that we explain the obligations 
of the State or the EQRO with regard to the documents included in the 
appendices (for example, what is the role of the documents and how the 
documents are to be used).
    Response: With the exception of Appendix Z, ISCA for MCOs and 
PIHPs, the appendices (Attachments to the final protocols) provide 
additional guidance to States and EQROs on how to implement the EQR-
related activities. The information contained in the appendices 
(Attachments to the final protocols) are to be used at the discretion 
of the State or EQRO based on the particular circumstances of the 
activity being conducted and other means of obtaining needed 
information.

I. Section 438.360 (Nonduplication of Mandatory Activities)

    Comment: One commenter believes the estimates of the time necessary 
to collect the information under this provision are too low. In 
addition, the commenter believes that this function needs to be 
performed by both professional staff and clerical staff and that a 
blend of the hourly costs should be used to determine the estimated 
costs.
    Response: As we stated earlier, because we received several 
comments indicating that this estimate is low but commenters did not 
provide us with what they believe the correct estimate to be, we have 
increased the burden hours by 100 percent to 8 hours. We have taken the 
commenters recommendation and blended the hourly costs to reflect that 
both professional and clerical staff will partake in this effort.

J. Section 438.362 (Exemption From EQR)

    No comments were received on this section.

K. Section 438.364 (EQR Results)

    No comments were received on this section.

IV. Provisions of the Final Regulation

    For the most part, this final rule adopts the provisions of the 
December 1, 1999 proposed rule. In response to public comments, we have 
made clarifying wording changes. Those provisions of this final rule 
that differ from the provisions of the December 1, 1999 proposed rule 
follow.

Section 438.310--Basis, Scope, and Applicability

    We have revised this section to reference the applicability of this 
rule to PIHPs. We have added the reference to PIHPs throughout the rule 
as appropriate.

Section 438.320--Definitions

    We have revised this section by adding clarifying language to the 
definitions for the terms ``EQR'' and ``EQRO'' and adding a definition 
for the term ``financial relationship.'' The definition of EQR has been 
revised to clarify that this rule applies to the care provided to 
Medicaid beneficiaries that receive health care services furnished by 
MCO and PIHP subcontractors as well as MCOs and PIHPs. This definition 
has also been revised to clarify that EQR-related activities are not 
considered part of the EQR function. We have revised the definition of 
EQRO to mean an organization that conducts the EQR function as well as 
EQR-related activities. EQR-related activities had not previously been 
included in the EQRO definition. As a result of this clarifying 
language, how we use the terms EQR, EQR-related activities, and EQRO 
needed to be changed in several sections of this rule.

Section 438.350--State Responsibilities

    We have revised this section to add clarifying language that the 
information provided to the EQRO is consistent with the information we 
require as part the EQR results; for each EQR-related activity that 
provides information for the EQR, the EQRO must have the objectives of 
the activity, the methods of data collection and analysis, a 
description of the data obtained, and the conclusions drawn.

Section 438.352--External Quality Review Protocols

    We have revised this section to add clarifying language at 
paragraph (c) of this section to explain what we meant by each protocol 
must specify the ``detailed procedures'' to be followed in collecting 
the data to promote its accuracy, validity, and reliability. We have 
changed the wording of ``detailed procedures'' to ``activities and 
steps'' to be consistent with how the EQR protocols have been designed.

Section 438.354--Qualifications of External Quality Review 
Organizations

    We have revised this section to add at paragraph (b)(1) that the 
EQRO must have ``demonstrated experience'' as well as knowledge of the 
Medicaid recipients, policies, data systems, and processes; managed 
care delivery systems, organizations, and financing; quality assessment 
and improvement methods, and research design and methodology.
    We have revised paragraph (c) of this section to require that all 
EQROs, as opposed to only State entities that qualify as EQROs, may not 
deliver any health care services to Medicaid beneficiaries, or conduct 
on the State's behalf ongoing Medicaid managed care program operations 
related to the oversight of MCO or PIHP quality of services. This later 
provision has been revised to apply only to Medicaid managed care 
operations as opposed to all Medicaid program operations. This provides 
States the opportunity to contract with a broader group of entities 
than was provided for in the December 1, 1999 proposed rule.
    We have also revised paragraph (c) of this section to add 
clarifying language to explain how ``control'' is defined in 48 CFR 
19.101. In addition, we have added a provision that prohibits an entity 
from qualifying as an EQRO if it has a financial relationship with an 
MCO or PIHP that it will review as an EQRO.

Section 438.356--State Contract Options

    We have revised paragraph (a) of this section to clarify that 
States may only contract with one entity for EQR alone or EQR and other 
EQR-related activities, but may contract with multiple entities to 
conduct additional EQR-related activities.

Section 438.358--Activities Related to External Review

    We have revised this section by adding cross-references to the 
Medicaid managed care final rule. We have made these cross-references 
throughout this rule where appropriate. We had not

[[Page 3631]]

included these cross-references in the December 1, 1999 proposed rule 
as the Medicaid managed care final rule had not yet been published.
    We have added a general rule under paragraph (a) to clarify that 
the mandatory and optional EQR related activities can be conducted by 
the State, the State's agent that is not an MCO or PIHP, or an EQRO.
    We have revised paragraph (b)(1) to clarify that information from 
the validation of performance improvement projects that are underway, 
as opposed to those being performed, must be obtained from the MCO or 
PIHP. We have revised paragraph (b)(2) to clarify that information on 
performance measures can be obtained from either those calculated by 
the MCO/PIHP and validated by the State or its agent, or those 
calculated by the State on behalf of the MCO/PIHP. We have also revised 
(b)(3) by eliminating the reference to specific State standards. These 
are now referenced in the aggregate by our cross-reference to the 
Medicaid managed care final rule provision. We have also revised 
paragraph (c) to clarify that information from optional activities must 
be from information derived within the preceding 12 months.

Section 438.360--Nonduplication of Mandatory Activities

    We have revised this section by removing the word ``exempt.'' Using 
this word caused confusion with the ``exemption of EQR requirements'' 
under Sec.  438.362. In its place, we provide language that explains 
that the nonduplication provisions allow States to use information from 
either a Medicare or accreditation review for certain standards and 
activities in place of a Medicaid review.
    We have also revised this section to allow States to apply this 
provision to MCOs and PIHPs that provide health care services to 
commercial consumers of health care as well as Medicare beneficiaries. 
We have further revised this section to clarify that national 
accrediting organizations are those organizations that have been 
approved and recognized for M+C deeming. We have made this 
clarification throughout the rule as appropriate.
    We have restructured this section by revising paragraph (b) so it 
applies to both M+C and MCOs and PIHPs that provide services to 
commercial consumers and have revised paragraph (c) to address 
additional provisions for those MCOs and PIHPs providing services to 
dually eligible beneficiaries only. Under paragraph (b) and (c), we 
have added a provision that requires the State in its quality strategy 
to identify those standards and activities for which it will substitute 
the Medicare or accreditation review for the Medicaid review. In 
addition, we require the State to explain the rationale for why the 
State considers the standards or activities duplicative.

Section 438.362--Exemption From External Quality Review

    We have revised paragraph (a)(2) to clarify that the Medicare and 
Medicaid contract must overlap geographically within the State when it 
exempts the MCO or PIHP from EQR. The December 1, 1999 proposed rule 
did not require that the overlap be within the State.
    We have revised (b)(1) to clarify that information from Medicare 
reviews is to be obtained by the State from the MCO or PIHP. The 
language in the December 1, 1999 proposed rule could have been 
misinterpreted to mean that the State had to obtain the information 
from CMS or its agent. We have also revised paragraph (b)(2) to clarify 
that the MCO or PIHP must provide the State a copy of the accreditation 
review findings as opposed to ensuring the State receives a copy.

Section 438.364--External Quality Review Results

    We have revised paragraph (a)(1) to clarify that in the detailed 
report, conclusions are drawn as to the timeliness of and access to 
care as well as the quality of care. We have revised paragraph 
(a)(1)(iii) to clarify that the detailed report should include a 
``description'' of the data obtained for each EQR-related activity as 
opposed to the data obtained. We did not intend for the raw data to be 
provided as part of the EQR results. We have also revised paragraph 
(a)(2) to require an assessment of the MCO's and PIHP's strengths and 
weaknesses be addressed as opposed to a ``detailed'' assessment of the 
MCO's and PIHP's strengths and weaknesses.
    We have revised paragraph (b) to require that the EQR results, upon 
request, be made available in alternative formats for persons with 
sensory impairments and that the EQR results be made available through 
electronic as well as printed copies.

Section 438.370--Federal Financial Participation

    We have revised (a) to clarify that 75 percent FFP is also 
available for the production of the EQR results.

V. Collection of Information Requirements

    Under the Paperwork Reduction Act of 1995 (PRA), we are required to 
provide a 30-day notice in the Federal Register and solicit public 
comment before a collection of information requirement is submitted to 
the Office of Management and Budget (OMB) for review and approval. In 
order to fairly evaluate whether an information collection should be 
approved by OMB, section 3506(c)(2)(A) of the PRA requires that we 
solicit comment on the following issues:
    [sbull] The need for the information collection and its usefulness 
in carrying out the proper functions of our agency.
    [sbull] The accuracy of our estimate of the information collection 
burden.
    [sbull] The quality, utility, and clarity of the information to be 
collected.
    [sbull] Recommendations to minimize the information collection 
burden on the affected public, including automated collection 
techniques.
    Therefore, we are soliciting public comment on each of these issues 
for Sec. Sec.  438.352, 438.360, 438.362 and 438.364 of this document 
that contain information collection requirements.
    We published a notice in the Federal Register on November 23, 2001, 
to give the public a 60-day period in which to comment. The basic 
purpose was to afford the public an opportunity to comment on the 
protocols. We have addressed the comments received in response to this 
Federal Register notice in section III. above.
    For purposes of this requirement, we incorporated Medicaid managed 
care data from the 2001 Medicaid enrollment report. As of June 2001, 
there were 329 MCOs (this includes 5 HIOs that must adhere to the EQR 
requirements of this regulation), and 129 mental health and substance 
abuse PIHPs.
    Sec.  438.358 (Activities related to EQR)--For each MCO and PIHP, 
the EQR must use information from the following activities:
    (1) Validation of performance improvement projects required by the 
State to comply with requirements set forth in Sec.  438.240(b)(1) and 
that were under way during the preceding 12 months.
    (2) Validation of MCO or PIHP performance measures reported (as 
required by the State) or MCO or PIHP performance measure calculated by 
the State during the preceding 12 months to comply with requirements 
set forth in Sec.  438.240(b)(2).
    (3) A review, conducted within the previous 3-year period, to 
determine the MCO's or PIHP's compliance with standards (except with 
respect to standards under Sec. Sec.  438.240(b)(1) and (2), for the 
conduct of performance improvement projects and calculation of 
performance measures, respectively)

[[Page 3632]]

established by the State to comply with the requirements of Sec.  
438.204(g).
    In addition, if a State, at its option, wishes to provide 
additional information to its EQRO, and to have CMS provide 75 percent 
FFP in the costs of producing this information, then the additional 
information must be produced through activities identified as optional 
activities in this final rule and also must be produced in a manner 
consistent with (as opposed to identical to) the protocols for these 
six optional activities. These six optional activities are (1) 
validation of client level data such as claims and encounters, (2) 
administration or validation of a survey, (3) calculation of 
performance measures, (4) conduct of performance improvement projects, 
and (5) conduct of focused studies of quality of care.
    The burden associated with this requirement is the time and effort 
for a State, EQRO, or other State contractor, to conduct and document 
the findings of the three mandatory activities--the validation of 
performance improvement projects conducted by the MCO/PIHP, the 
validation of performance measures calculated by the MCO/PIHP, and a 
review of MCO/PIHP compliance with structural and operational 
standards. Each of these activities will need to be conducted on the 
329 MCOs and 129 PIHPs that we estimate are currently providing 
Medicaid services. The types of services provided by these managed care 
entities and the number of performance improvement projects conducted 
and performance measures calculated will vary.
    We interviewed four EQROs who in 2000 reviewed MCOs/PIHPs in 16 
mandatory or voluntary managed care programs in eight States. Based on 
the information provided by the four EQROs, we confirmed that the hours 
and costs to conduct these activities vary. The information provided 
includes: (1) It takes 25 to 138 hours at a cost of $2,000 to $10,000 
to validate a performance improvement project conducted by an MCO/PIHP; 
(2) it takes 12 to 202 hours at a cost of $1,200 to $7,000 to validate 
a performance measure calculated by an MCO/PIHP; and it takes 200 to 
800 hours at a cost of $11,000 to $49,000 to review for MCO/PIHP 
compliance with structural and operational standards. Based on the 
submitted information, it takes an average of 65, 53, and 361 hours, 
respectively, to conduct the above mandatory EQR activities. Therefore, 
the average total burden associated with this requirement is 479 hours 
x 458 entities (329 MCOs + 129 PIHPs). Assuming wages of $63 per hour 
for professionals to comply with the requirement, the cost is 
$13,821,066.
    For the optional EQR activities--validation of client level data 
(such as claims and encounters), administration or validation of 
consumer or provider surveys, calculation of performance measures, 
conduct of performance improvement projects, and conduct of focused 
studies--we have no data to estimate the hours associated with how long 
it will take to conduct these activities. We, therefore, estimate that 
it will take 350 hours to validate client level data and 50 hours to 
validate consumer or provider surveys. We estimate it will take three 
times as long to calculate performance measures as it takes on average 
to validate (159 hours) and three times as long to conduct performance 
improvement projects and focused studies as it takes on average to 
validate performance improvement projects (195 hours). We also estimate 
that it will take three times as long to administer a consumer or 
provider survey than it takes to validate a survey (150 hours).
    Based on 2001 State reported data, we know that of the 42 States 
that had capitated programs (MCOs or PIHPs) in 2001, 29 (69 percent) 
had their EQROs validate MCO/PIHP encounter data, 18 (43 percent) had 
their EQRO administer or validate consumer or provider surveys, 12 (29 
percent) had their EQRO calculate performance measures, 16 (38 percent) 
had their EQRO conduct performance improvement projects, and 32 (76 
percent) had their EQRO conduct focused studies. Using the 
aforementioned percentages and applying them to the number of MCOs and 
PIHPs, we estimate that States will contract with their EQROs to 
validate the encounter data of 316 MCOs/PIHPs, administer or validate 
consumer or provider surveys of 197 MCOs/PIHPs, calculate performance 
measures of 133 MCOs/PIHPs, conduct performance improvement projects of 
174 MCOs/PIHPs, and conduct focused studies of 348 MCOs/PIHPs.
    We, therefore, estimate the average total burden associated with 
conducting each optional EQR activity as follows:
    [sbull] Validating client level data 350 hours x 316 MCOs/PIHPs = 
110,600 hours.
    [sbull] Validating consumer or provider surveys 50 hours x 98 MCOs/
PIHPs (\1/2\ of 197 MCO/PIHPs that administered or validated surveys) = 
4,900 hours.
    [sbull] Administering consumer or provider surveys 150 hours x 99 
MCOs/PIHPs (\1/2\ of 197 MCO/PIHPs that administered or validated 
surveys) = 14,850 hours.
    [sbull] Calculating performance measures 159 hours x 133 MCOs/PIHPs 
= 21,147 hours.
    [sbull] Conducting performance improvement projects 195 hours x 174 
MCOs/PIHPs = 33,930 hours.
    [sbull] Conducting focused studies 159 hours x 348 = 55,332 hours.
    Assuming a wage of $63 per hour for professionals to comply with 
the requirement, the cost of conducting the optional EQR activities is 
(240,759 hours x $63) $15,167,817. We solicit comments specifically on 
this issue because we had no data on which to base the estimated hours 
for the conduct of each of the optional EQR activities.
    The burden estimate associated with this requirement also includes 
the time and effort for an MCO/PIHP to prepare the information 
necessary for the EQRO or other State contractor to conduct the three 
mandatory activities--the validation of performance improvement 
projects conducted by the MCO/PIHP, the validation of performance 
measures calculated by the MCO/PIHP, and a review of MCO/PIHP 
compliance with structural and operational standards. We estimate that 
it will take each MCO and PIHP 160 hours to prepare this documentation. 
We believe one-half of the time preparing the information will be done 
by professional staff at $63 per hour and the other one-half of the 
time preparing the information will be done using clerical staff at $12 
per hour. Therefore, to comply with the requirement, the cost of 
compiling the necessary information is (458 MCOs/PIHPs x (80 hours x 
$63 + 80 hours x $12) $2,748,000.
    Sec.  438.360 (Nonduplication of mandatory activities)--In order to 
avoid duplication, the State agency may allow the MCO/PIHP to 
substitute information from a Medicare or accreditation review for the 
Medicaid review if specified conditions are met. To demonstrate 
compliance with these requirements an MCO/PIHP must provide to the 
State agency reports, findings, and other results of the Medicare or 
private accreditation review. The burden associated with these 
requirements is the time and effort for an MCO/PIHP to disclose the 
reports, findings, and other results of the Medicare or private 
accreditation review to the State agency. Of the 329 MCOs and 129 PIHPs 
providing Medicaid services, approximately 122 are Medicaid-only MCOs. 
We believe that there is the potential for States to allow the 
remaining 336 MCOs/PIHPs to take advantage of the nonduplication 
provision and that these MCOs/PIHPs will be required to disclose the 
necessary information to each State

[[Page 3633]]

agency. We estimate that it will take each MCO 8 hours to disclose the 
necessary documentation to the State, 4 hours of professional time and 
4 hours of clerical time. Therefore, the total burden associated with 
this requirement is 336 MCOs/PIHPs x 8 hours = 2688 annual burden 
hours. At $37.50 per hour ($12 + $63/2), the cost will be $100,800.
    This section also requires that a State agency provide the reports, 
findings, and other results of the Medicare or private accreditation 
review to the appropriate EQRO. We estimate that it will take, on 
average, 8 hours for a State to disclose the necessary documentation to 
the appropriate EQRO. The total annual burden associated with this 
requirement is 2688 hours ($37.50 per hour) and $100,800.
    This section also requires a State to include in its quality 
strategy information concerning the activities or standards for which 
it is obtaining information from Medicare or an accrediting 
organization. We believe that the burden for this information 
collection requirement is included in the burden addressed in the 
Medicaid managed care rule and approved under OMB number 0938.
    Sec.  438.362 (Exemption from EQR)--Each year, exempted MCOs/PIHPs 
must provide to the State agency the most recent Medicare review 
findings reported to the MCO/PIHP. This information must include (1) 
all data, correspondence, information, and findings pertaining to the 
MCO's/PIHP's compliance with Medicare standards for access, quality 
assessment and performance improvement, health services, or delegation 
of these activities; (2) all measures of the MCO's/PIHP's performance; 
and (3) the findings and results of all performance improvement 
projects pertaining to Medicare enrollees.
    If an exempted MCO/PIHP has been reviewed by a private accrediting 
organization and the survey results have been used to either fulfill 
certain requirements for Medicare external review under 42 CFR part 
422, subpart D or to deem compliance with Medicare requirements as 
provided in Sec.  422.156, the MCO/PIHP must submit a copy of all 
findings pertaining to its most recent accreditation review to the 
State agency. These findings must include accreditation survey results 
of evaluation of compliance with individual accreditation standards, 
noted deficiencies, corrective action plans, and summaries of unmet 
accreditation requirements.
    The burden associated with these requirements is not applicable for 
2 years following the final publication of this regulation. After 2 
years, the time and effort for an exempted MCO/PIHP to disclose the 
findings of its most recent Medicare or private accreditation review to 
the State agency will be the burden associated with these requirements. 
We estimate, of the approximately 202 MCOs that potentially may provide 
Medicare services in addition to Medicaid services, State agencies will 
allow for approximately 10 percent of the MCOs to be exempt from the 
EQR requirement. We further estimate that it will take each MCO 8 hours 
to prepare and submit the necessary documentation to the State agency. 
Therefore, the total burden associated with this requirement is 10 
percent of 202 MCOs x 8 hours = 160 annual burden hours. At a cost of 
$37.50 ($12 + $63/2) per hour, we assume a total cost of $6,000.
    Sec.  438.364 (EQR results)--The EQRO responsible for the EQR 
function will be required to provide to the State agency a detailed 
technical report that describes for each mandatory and optional 
activity undertaken for the EQR, the objectives, technical methods of 
data collection and analysis, a description of the data obtained, 
conclusions drawn from the data, and the manner in which the 
conclusions were drawn as to the quality of the care furnished by the 
MCO/PIHP. In addition, the report must include: (1) An assessment of 
each MCO's/PIHP's strengths and weaknesses with respect to the quality, 
timeliness, and access to health care services furnished to Medicaid 
beneficiaries; (2) recommendations for improving the quality of health 
care services furnished by each MCO/PIHP; (3) as the State agency 
determines methodologically appropriate, comparative information about 
all MCOs/PIHPs, and (4) an assessment of the degree to which each MCO/
PIHP has addressed effectively the recommendations for quality 
improvement, as made by the EQRO during the previous year's EQR.
    The burden associated with this requirement is the time and effort 
for an EQRO to submit to a State agency a detailed technical report for 
each EQR conducted. We estimate that it will take an EQRO 200 hours to 
prepare and submit the necessary documentation to the State agency. 
Therefore, the total burden associated with this requirement is 458 
technical reports (329 MCOs + 129 PIHPs) x 200 hours = 91,600 annual 
burden hours. Assuming wages of $63 per hour for professionals to 
comply with this requirement, the cost is $5,770,800.
    This section also requires each State agency to provide copies of 
technical reports, upon request, to interested parties such as 
participating health care providers, enrollees and potential enrollees 
of the MCO/PIHP, beneficiary advocate groups, and members of the 
general public.
    The burden associated with this requirement is the time and effort 
for a State agency to disclose copies of a given technical report to 
interested parties. We estimate that on average, it will take a State 
agency 8 hours to disclose the required information. Therefore, the 
total burden associated with this requirement is 329 MCOs + 129 PIHPs x 
25 requests per MCO or PIHP x 8 hours = 91,600 annual burden hours and 
a cost ($12 per hour) of $1,099,200.
    The information collection requirements contained in this final 
rule will be submitted to OMB for review. In accordance with the 
Paperwork Reduction Act, these requirements will not go into effect 
until approved by OMB.
    If you comment on any of these information collection and record 
keeping requirements, please mail 3 copies directly to the following:

Centers for Medicare & Medicaid Services, Office of Information 
Services, Security and Standards Group, Division of CMS Enterprise 
Standards, Room N2-14-26, 7500 Security Boulevard, Baltimore, MD 21244-
1850; Attn: Julie Brown, HCFA-2015-F; and
Office of Information and Regulatory Affairs, Office of Management and 
Budget, Room 10235, New Executive Office Building, Washington, DC 
20503, Attn: Brenda Aguilar, CMS Desk Officer.

VI. Regulatory Impact Analysis

A. Overall Impact

    We have examined the impacts of this rule as required by Executive 
Order 12866 (September 1993, Regulatory Planning and Review) the 
Regulatory Flexibility Act (RFA) (September 19, 1980, Pub. L. 96-354), 
section 1102(b) of the Social Security Act, the Unfunded Mandates 
Reform Act of 1995 (Pub. L. 104-4), and Executive Order 13132. 
Executive Order 12866 directs agencies to assess all costs and benefits 
of available regulatory alternatives and, when regulation is necessary, 
to select regulatory approaches that maximize net benefits, including 
potential economic, environmental, public health and safety effects, 
distributive impacts, and equity. A regulatory impact analysis (RIA) 
must be prepared for major rules with economically significant effects 
($100 million or more in any one year).

[[Page 3634]]

    The RFA requires agencies to analyze options for regulatory relief 
of small entities. For purposes of the RFA, we prepare a regulatory 
flexibility analysis unless we certify that a rule will not have a 
significant economic impact on a substantial number of small entities. 
Small entities include small businesses, nonprofit organizations, and 
governmental agencies. Most hospitals and other providers and suppliers 
are small entities, either by nonprofit status or by having revenues of 
$5 to $25 million or less annually. Individuals and States are not 
included in the definition of a small entity.
    Section 1102(b) of the Act requires us to prepare a regulatory 
impact analysis for any proposed rule that may have a significant 
impact on the operations of a substantial number of small rural 
hospitals. This analysis must conform to the provisions of section 604 
of the RFA. For purposes of section 1102(b) of the Act, we define a 
small rural hospital as a hospital that is located outside a 
Metropolitan Statistical Area and has fewer than 100 beds.
    The Unfunded Mandates Reform Act (Pub. L. 104-4) requires that 
agencies prepare an assessment of anticipated costs and benefits before 
proposing any rule that may result in an annual expenditure by State, 
local, and tribal governments, in the aggregate, or by the private 
sector, of $110 million or more. This rule does not impose any mandates 
on State, local, or tribal governments, or the private sector that will 
result in an annual expenditure of $110 million or more.
    Under Executive Order 13132, we are required to adhere to certain 
criteria regarding Federalism in developing regulations. We have 
determined that this regulation will not significantly affect States 
rights, roles, and responsibilities. Section 1903(a)(30)(C) of the Act 
currently requires an EQR for each contract a State has with a section 
1903(m) organization. In accordance with section 4705 of the BBA, this 
rule will establish requirements and procedures for EQR of Medicaid 
MCOs. We require States to ensure that an annual EQR is performed by a 
qualified EQRO for each contracting MCO, the EQRO has adequate 
information to carry out the review, and that the results of the 
reviews are made available to interested parties such as participating 
health care providers, enrollees, advocate groups, and the general 
public. We also require that these EQR provisions apply to PIHPs and 
certain entities with comprehensive risk contracts that have been 
exempted from the requirements of section 1903(m) of the Act. We 
believe this is consistent with the intent of the Congress in enacting 
the quality provisions of the BBA. This rule would not require State 
agencies to dismantle EQR mechanisms that they have used to meet 
section 1902 (a)(30)(C) of the Act and which they have found to be 
effective and efficient. Rather, this rule would provide States greater 
flexibility in the types of entities they may use to conduct EQR.
    We worked closely with States in developing this regulation. 
Specifically, in accordance with section 1932(c)(2)(A)(ii) of the Act, 
which requires the Secretary to consult with States to establish a 
method for identifying entities qualified to conduct EQR, we met with 
States and other stakeholders under the auspices of the NASHP to 
establish a criteria to identify qualified entities. Most of the 
recommendations made at this meeting have been incorporated into this 
rule. For recommendations not accepted, an explanation was provided in 
the December 1, 1999 proposed rule.
    In addition, section 1932(c)(2)(A)(iii) of the Act requires the 
Secretary to coordinate with the NGA in contracting with an independent 
quality review organization to develop protocols to be used in EQR. To 
meet this requirement, we issued a request for proposal for one or more 
contractors to develop a set of review protocols for EQROs to use in 
the conduct of EQRs. Two State representatives selected by the NGA were 
members of the panel that reviewed and rated responding proposals. 
Moreover, part of the development of the EQR protocols includes 
convening an expert panel for review and comment of the protocols. 
State representatives were included in this process.

B. Anticipated Effects

    In publishing this final rule, we considered two main alternatives. 
The first was to allow this final rule to be published, incorporating 
public comments on the proposed rule. The second alternative was to 
implement the provisions of the BBA as written, without expanding the 
regulations beyond the statutory language. We believe this final rule 
as written was the appropriate alternative to choose. Used in 
conjunction with the Medicaid Managed Care final rule published June 
14, 2002, this final rule is a necessary tool for States to use to 
create and maintain strong, viable Medicaid managed care programs that 
deliver high quality health care in their State marketplaces and health 
care delivery systems. Further, we felt this final rule was necessary 
to implement the Congress' directive to the Secretary to establish a 
method for identifying entities qualified to conduct EQR.
    We do not anticipate that the provisions in this final rule will 
have a substantial economic impact on most hospitals, including small 
rural hospitals. The BBA provisions include some new requirements on 
State agencies and MCOs, but not directly on individual hospitals. The 
impact on individual hospitals will vary according to each hospital's 
current and future contractual relationships with MCOs. Furthermore, 
the impact will also vary according to each hospital's current 
procedures and level of compliance with existing law and regulation 
pertaining to Medicaid managed care. For these reasons, this final rule 
will not have a significant impact on the operations of a substantial 
number of hospitals. The only other small entity affected by these 
regulations would be the EQROs. However, this rule does not impose 
additional burdens on them. Instead, the rule offers these 
organizations the benefit of opportunities for additional revenues. 
Thus we certify that this rule will not have a significant economic 
impact on a substantial number of small entities.
    We do not anticipate a significant increase in Medicaid 
expenditures as a result of the publication of these regulations for 
the following reasons. First, approximately 42 States are currently 
obtaining 75 percent enhanced FFP for EQR activities carried out by 
QIOs and organizations that meet the requirements to contract with 
Medicare as a QIO. Permitting these State agencies to claim 75 percent 
matching for EQR activities conducted by the additional types of 
entities allowed by these regulations would therefore not result in 
increased costs to the extent that State agencies switch from QIO or 
organizations that meet the requirements to contract with Medicare as a 
QIO to these other entities. Moreover, we believe that, by expanding 
the pool of organizations available to conduct EQR, State agencies may 
be able to negotiate savings compared to current costs of dealing with 
PRO and PRO-like organizations. Additional savings may be realized 
through opportunities afforded by the final rule to coordinate EQR 
activities with quality reviews conducted for other purposes. 
Additional costs may arise where State agencies currently conduct 
quality review activities at 50 percent Federal matching rate that 
would now qualify for 75 percent, and from new EQR activities 
undertaken as a result of the BBA requirements.
    In addition, even though we extend this requirement to PIHPs, again 
we do

[[Page 3635]]

not expect this to significantly increase Medicaid expenditures. PIHP 
costs account for approximately 5 percent of the payments we make to 
capitated arrangements. Furthermore, State agencies currently conduct 
quality review activities on PIHPs at a 50 percent Federal matching 
rate. Additional costs may arise for States' quality review activities 
that would now qualify for 75 percent and for new quality review 
activities undertaken as a result of the activities required in this 
rule.
    Although we cannot quantify these various cost and savings effects, 
we believe that their net impact would be well below the $100 million 
threshold for a major rule, and therefore that a regulatory impact 
analysis is not required. We do not believe that this final rule will 
cause MCOs to devote significantly more time to collect, organize and 
prepare for EQR than is already required by States. While the scope of 
work for EQR may be different under this final rule, we do not believe 
that the cost difference will be significant and States may actually be 
able to achieve savings since we are expanding the pool of 
organizations available to conduct EQR. Further, additional savings may 
also be realized through opportunities afforded by this rule to 
coordinate EQR activities with other quality and oversight activities. 
We acknowledge with the increased opportunity to contract with other 
qualified entities to conduct EQR, more States may avail themselves the 
75 percent match for EQR activities. However, we do not believe this 
would represent a significant cost impact.

C. Conclusion

    For these reasons, we are not preparing analyses for either the RFA 
or section 1102(b) of the Act because we have determined, and we 
certify, that this rule will not have a significant economic impact on 
a substantial number of small entities or a significant impact on the 
operations of a substantial number of small rural hospitals.
    In accordance with the provisions of Executive Order 12866, this 
regulation was reviewed by the Office of Management and Budget.

List of Subjects

42 CFR Part 433

    Administrative practice and procedure, Child support, Claims, Grant 
programs-health, Medicaid, Reporting and record keeping requirements.

42 CFR Part 438

    Grant Programs--health, Managed care entities, Medicaid, Quality 
assurance, Reporting and record keeping requirements.

    For the reasons set forth in the preamble, the Centers for Medicare 
& Medicaid Services amends 42 CFR chapter IV as set forth below.

PART 433--STATE FISCAL ADMINISTRATION

    A. Amend part 433 as set forth below.
    1. The authority citation for part 433 continues to read as 
follows:

    Authority: Sec. 1102 of the Social Security Act (42 U.S.C. 
1302).


    2. In Sec.  433.15, add a new paragraph (b)(10) to read as follows:


Sec.  433.15  Rates of FFP for administration.

* * * * *
    (b) * * *
    (10) Funds expended for the performance of external quality review 
or the related activities described in Sec.  438.358 of this chapter 
when they are performed by an external quality review organization as 
defined in Sec.  438.320 of this chapter: 75 percent.

    B. Add a new subpart E to part 438 to read as set forth below.

PART 438--MANAGED CARE

Subpart E--External Quality Review
Sec.
438.310 Basis, scope, and applicability.
438.320 Definitions.
438.350 State responsibilities.
438.352 External quality review protocols.
438.354 Qualifications of external quality review organizations.
438.356 State contract options.
438.358 Activities related to external quality review.
438.360 Nonduplication of mandatory activities.
438.362 Exemption from external quality review.
438.364 External quality review results.
438.370 Federal financial participation.

    Authority: Sec. 1102 of the Social Security Act (42 U.S.C. 
1302).

Subpart E--External Quality Review


Sec.  438.310  Basis, scope, and applicability.

    (a) Statutory basis. This subpart is based on sections 1932(c)(2), 
1903(a)(3)(C)(ii), and 1902(a)(4) of the Act.
    (b) Scope. This subpart sets forth requirements for annual external 
quality reviews of each contracting managed care organization (MCO) and 
prepaid inpatient health plan (PIHP), including--
    (1) Criteria that States must use in selecting entities to perform 
the reviews;
    (2) Specifications for the activities related to external quality 
review;
    (3) Circumstances under which external quality review may use the 
results of Medicare quality reviews or private accreditation reviews; 
and
    (4) Standards for making available the results of the reviews.
    (c) Applicability. The provisions of this subpart apply to MCOs, 
PIHPs, and to health insuring organizations (HIOs) that began on or 
after January 1, 1986 that the statute does not explicitly exempt from 
requirements in section 1903(m) of the Act.


Sec.  438.320  Definitions.

    As used in this subpart--
    EQR stands for external quality review.
    EQRO stands for external quality review organization.
    External quality review means the analysis and evaluation by an 
EQRO, of aggregated information on quality, timeliness, and access to 
the health care services that an MCO or PIHP, or their contractors 
furnish to Medicaid recipients.
    External quality review organization means an organization that 
meets the competence and independence requirements set forth in Sec.  
438.354, and performs external quality review, other EQR-related 
activities as set forth in Sec.  438.358, or both.
    Financial relationship means--
    (1) A direct or indirect ownership or investment interest 
(including an option or nonvested interest) in any entity. This direct 
or indirect interest may be in the form of equity, debt, or other means 
and includes any indirect ownership or investment interest no matter 
how many levels removed from a direct interest; or
    (2) A compensation arrangement with an entity.
    Quality, as it pertains to external quality review, means the 
degree to which an MCO or PIHP increases the likelihood of desired 
health outcomes of its enrollees through its structural and operational 
characteristics and through the provision of health services that are 
consistent with current professional knowledge.
    Validation means the review of information, data, and procedures to 
determine the extent to which they are accurate, reliable, free from 
bias, and in accord with standards for data collection and analysis.


Sec.  438.350  State responsibilities.

    Each State that contracts with MCOs or PIHPs must ensure that--
    (a) Except as provided in Sec.  438.362, a qualified EQRO performs 
an annual EQR for each contracting MCO or PIHP;

[[Page 3636]]

    (b) The EQRO has sufficient information to use in performing the 
review;
    (c) The information used to carry out the review must be obtained 
from the EQR-related activities described in Sec.  438.358.
    (d) For each EQR-related activity, the information must include the 
elements described in Sec.  438.364(a)(1)(i) through (a)(1)(iv);
    (e) The information provided to the EQRO in accordance with 
paragraph (c) of this section is obtained through methods consistent 
with the protocols established under Sec.  438.352; and
    (f) The results of the reviews are made available as specified in 
Sec.  438.364.


Sec.  438.352  External quality review protocols.

    Each protocol must specify--
    (a) The data to be gathered;
    (b) The sources of the data;
    (c) The activities and steps to be followed in collecting the data 
to promote its accuracy, validity, and reliability;
    (d) The proposed method or methods for validly analyzing and 
interpreting the data once obtained; and
    (e) Instructions, guidelines, worksheets, and other documents or 
tools necessary for implementing the protocol.


Sec.  438.354  Qualifications of external quality review organizations.

    (a) General rule. The State must ensure that an EQRO meets the 
requirements of this section.
    (b) Competence. The EQRO must have at a minimum the following:
    (1) Staff with demonstrated experience and knowledge of--
    (i) Medicaid recipients, policies, data systems, and processes;
    (ii) Managed care delivery systems, organizations, and financing;
    (iii) Quality assessment and improvement methods; and
    (iv) Research design and methodology, including statistical 
analysis.
    (2) Sufficient physical, technological, and financial resources to 
conduct EQR or EQR-related activities.
    (3) Other clinical and nonclinical skills necessary to carry out 
EQR or EQR-related activities and to oversee the work of any 
subcontractors.
    (c) Independence. The EQRO and its subcontractors are independent 
from the State Medicaid agency and from the MCOs or PIHPs that they 
review. To qualify as ``independent''--
    (1) A State agency, department, university, or other State entity 
may not have Medicaid purchasing or managed care licensing authority; 
and
    (2) A State agency, department, university, or other State entity 
must be governed by a Board or similar body the majority of whose 
members are not government employees.
    (3) An EQRO may not--
    (i) Review a particular MCO or PIHP if either the EQRO or the MCO 
or PIHP exerts control over the other (as used in this paragraph, 
``control'' has the meaning given the term in 48 CFR 19.101) through--
    (A) Stock ownership;
    (B) Stock options and convertible debentures;
    (C) Voting trusts;
    (D) Common management, including interlocking management; and
    (E) Contractual relationships.
    (ii) Deliver any health care services to Medicaid recipients;
    (iii) Conduct, on the State's behalf, ongoing Medicaid managed care 
program operations related to oversight of the quality of MCO or PIHP 
services, except for the related activities specified in Sec.  438.358; 
or
    (iv) Have a present, or known future, direct or indirect financial 
relationship with an MCO or PIHP that it will review as an EQRO.


Sec.  438.356  State contract options.

    (a) The State--
    (1) Must contract with one EQRO to conduct either EQR alone or EQR 
and other EQR-related activities; and
    (2) May contract with additional EQROs to conduct EQR-related 
activities as set forth in Sec.  438.358.
    (b) Each EQRO must meet the competence requirements as specified in 
Sec.  438.354(b).
    (c) Each EQRO is permitted to use subcontractors. The EQRO is 
accountable for, and must oversee, all subcontractor functions.
    (d) Each EQRO and its subcontractors performing EQR or EQR-related 
activities must meet the requirements for independence, as specified in 
Sec.  438.354(c).
    (e) For each contract, the State must follow an open, competitive 
procurement process that is in accordance with State law and 
regulations and consistent with 45 CFR part 74 as it applies to State 
procurement of Medicaid services.


Sec.  438.358  Activities related to external quality review.

    (a) General rule. The State, its agent that is not an MCO or PIHP, 
or an EQRO may perform the mandatory and optional EQR-related 
activities in this section.
    (b) Mandatory activities. For each MCO and PIHP, the EQR must use 
information from the following activities:
    (1) Validation of performance improvement projects required by the 
State to comply with requirements set forth in Sec.  438.240(b)(1) and 
that were underway during the preceding 12 months.
    (2) Validation of MCO or PIHP performance measures reported (as 
required by the State) or MCO or PIHP performance measure calculated by 
the State during the preceding 12 months to comply with requirements 
set forth in Sec.  438.240(b)(2).
    (3) A review, conducted within the previous 3-year period, to 
determine the MCO's or PIHP's compliance with standards (except with 
respect to standards under Sec. Sec.  438.240(b)(1) and (2), for the 
conduct of performance improvement projects and calculation of 
performance measures respectively) established by the State to comply 
with the requirements of Sec.  438.204(g).
    (c) Optional activities. The EQR may also use information derived 
during the preceding 12 months from the following optional activities:
    (1) Validation of encounter data reported by an MCO or PIHP.
    (2) Administration or validation of consumer or provider surveys of 
quality of care.
    (3) Calculation of performance measures in addition to those 
reported by an MCO or PIHP and validated by an EQRO.
    (4) Conduct of performance improvement projects in addition to 
those conducted by an MCO or PIHP and validated by an EQRO.
    (5) Conduct of studies on quality that focus on a particular aspect 
of clinical or nonclinical services at a point in time.
    (d) Technical assistance. The EQRO may, at the State's direction, 
provide technical guidance to groups of MCOs or PIHPs to assist them in 
conducting activities related to the mandatory and optional activities 
that provide information for the EQR.


Sec.  438.360  Nonduplication of mandatory activities.

    (a) General rule. To avoid duplication, the State may use, in place 
of a Medicaid review by the State, its agent, or EQRO, information 
about the MCO or PIHP obtained from a Medicare or private accreditation 
review to provide information otherwise obtained from the mandatory 
activities specified in Sec.  438.358 if the conditions of paragraph 
(b) or paragraph (c) of this section are met.
    (b) MCOs or PIHPs reviewed by Medicare or private accrediting

[[Page 3637]]

organizations. For information about an MCO's or PIHP's compliance with 
one or more standards required under Sec.  438.204(g), (except with 
respect to standards under Sec. Sec.  438.240(b)(1) and (2), for the 
conduct of performance improvement projects and calculation of 
performance measures respectively) the following conditions must be 
met:
    (1) The MCO or PIHP is in compliance with standards established by 
CMS for Medicare+Choice or a national accrediting organization. The CMS 
or national accreditation standards are comparable to standards 
established by the State to comply with Sec.  438.204(g) and the EQR-
related activity under Sec.  438.358(b)(3).
    (2) Compliance with the standards is determined either by--
    (i) CMS or its contractor for Medicare; or
    (ii) A private national accrediting organization that CMS has 
approved as applying standards at least as stringent as Medicare under 
the procedures in Sec.  422.158.
    (3) The MCO or PIHP provides to the State all the reports, 
findings, and other results of the Medicare or private accreditation 
review applicable to the standards provided for in Sec.  438.204(g); 
and the State provides the information to the EQRO.
    (4) In its quality strategy, the State identifies the standards for 
which the EQR will use information from Medicare or private 
accreditation reviews, and explains its rationale for why the standards 
are duplicative.
    (c) Additional provisions for MCOs or PIHPs serving only dually 
eligibles. The State may use information obtained from the Medicare 
program in place of information produced by the State, its agent, or 
EQRO with respect to the mandatory activities specified in Sec.  
438.358 (b)(1) and (b)(2) if the following conditions are met:
    (1) The MCO or PIHP serves only individuals who receive both 
Medicare and Medicaid benefits.
    (2) The Medicare review activities are substantially comparable to 
the State-specified mandatory activities in Sec.  438.358(b)(1) and 
(b)(2).
    (3) The MCO or PIHP provides to the State all the reports, 
findings, and other results of the Medicare review from the activities 
specified under Sec.  438.358(b)(1) and (b)(2) and the State provides 
the information to the EQRO.
    (4) In its quality strategy, the State identifies the mandatory 
activities for which it has exercised this option and explains its 
rationale for why these activities are duplicative.


Sec.  438.362  Exemption from external quality review.

    (a) Basis for exemption. The State may exempt an MCO or PIHP from 
EQR if the following conditions are met:
    (1) The MCO or PIHP has a current Medicare contract under part C of 
title XVIII or under section 1876 of the Act, and a current Medicaid 
contract under section 1903(m) of the Act.
    (2) The two contracts cover all or part of the same geographic area 
within the State.
    (3) The Medicaid contract has been in effect for at least 2 
consecutive years before the effective date of the exemption and during 
those 2 years the MCO or PIHP has been subject to EQR under this part, 
and found to be performing acceptably with respect to the quality, 
timeliness, and access to health care services it provides to Medicaid 
recipients.
    (b) Information on exempted MCOs or PIHPs. When the State exercises 
this option, the State must obtain either of the following:
    (1) Information on Medicare review findings. Each year, the State 
must obtain from each MCO or PIHP that it exempts from EQR the most 
recent Medicare review findings reported on the MCO or PIHP including--
    (i) All data, correspondence, information, and findings pertaining 
to the MCO's or PIHP's compliance with Medicare standards for access, 
quality assessment and performance improvement, health services, or 
delegation of these activities;
    (ii) All measures of the MCO's or PIHP's performance; and
    (iii) The findings and results of all performance improvement 
projects pertaining to Medicare enrollees.
    (2) Medicare information from a private, national accrediting 
organization that CMS approves and recognizes for Medicare+Choice 
deeming.
    (i) If an exempted MCO or PIHP has been reviewed by a private 
accrediting organization, the State must require the MCO or PIHP to 
provide the State with a copy of all findings pertaining to its most 
recent accreditation review if that review has been used for either of 
the following purposes:
    (A) To fulfill certain requirements for Medicare external review 
under subpart D of part 422 of this chapter.
    (B) To deem compliance with Medicare requirements, as provided in 
Sec.  422.156 of this chapter.
    (ii) These findings must include, but need not be limited to, 
accreditation review results of evaluation of compliance with 
individual accreditation standards, noted deficiencies, corrective 
action plans, and summaries of unmet accreditation requirements.


Sec.  438.364  External quality review results.

    (a) Information that must be produced. The State must ensure that 
the EQR produces at least the following information:
    (1) A detailed technical report that describes the manner in which 
the data from all activities conducted in accordance with Sec.  438.358 
were aggregated and analyzed, and conclusions were drawn as to the 
quality, timeliness, and access to the care furnished by the MCO or 
PIHP. The report must also include the following for each activity 
conducted in accordance with Sec.  438.358:
    (i) Objectives.
    (ii) Technical methods of data collection and analysis.
    (iii) Description of data obtained.
    (iv) Conclusions drawn from the data.
    (2) An assessment of each MCO's or PIHP's strengths and weaknesses 
with respect to the quality, timeliness, and access to health care 
services furnished to Medicaid recipients.
    (3) Recommendations for improving the quality of health care 
services furnished by each MCO or PIHP.
    (4) As the State determines, methodologically appropriate, 
comparative information about all MCOs and PIHPs.
    (5) An assessment of the degree to which each MCO or PIHP has 
addressed effectively the recommendations for quality improvement made 
by the EQRO during the previous year's EQR.
    (b) Availability of information. The State must provide copies of 
the information specified in paragraph (a) of this section, upon 
request, through print or electronic media, to interested parties such 
as participating health care providers, enrollees and potential 
enrollees of the MCO or PIHP, recipient advocacy groups, and members of 
the general public. The State must make this information available in 
alternative formats for persons with sensory impairments, when 
requested.
    (c) Safeguarding patient identity. The information released under 
paragraph (b) of this section may not disclose the identity of any 
patient.


Sec.  438.370  Federal financial participation.

    (a) FFP at the 75 percent rate is available in expenditures for EQR 
(including the production of EQR results) and EQR-related activities 
set forth in Sec.  438.358 conducted by EQROs and their subcontractors.
    (b) FFP at the 50 percent rate is available in expenditures for 
EQR-

[[Page 3638]]

related activities conducted by any entity that does not qualify as an 
EQRO.

(Catalog of Federal Domestic Assistance Program No. 93.778, Medical 
Assistance)

    Dated: August 6, 2002.
Thomas A. Scully,
Administrator, Centers for Medicare & Medicaid Services.
    Approved: October 3, 2003.
Tommy G. Thompson,
Secretary.
[FR Doc. 03-1294 Filed 1-23-03; 8:45 am]
BILLING CODE 4120-01-P