[Federal Register Volume 85, Number 58 (Wednesday, March 25, 2020)]
[Notices]
[Pages 16943-16945]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 2020-06208]


=======================================================================
-----------------------------------------------------------------------

DEPARTMENT OF HEALTH AND HUMAN SERVICES

Agency for Healthcare Research and Quality


Agency Information Collection Activities: Proposed Collection; 
Comment Request

AGENCY: Agency for Healthcare Research and Quality, HHS.

ACTION: Notice.

-----------------------------------------------------------------------

SUMMARY: This notice announces the intention of the Agency for 
Healthcare Research and Quality (AHRQ) to request that the Office of 
Management and Budget (OMB) approve the proposed information collection 
project ``Patient-Centered Outcomes Research Clinical Decision Support: 
Current State and Future Directions.''

DATES: Comments on this notice must be received by 60 days after date 
of publication of this notice in the Federal Register.

ADDRESSES: Written comments should be submitted to: Doris Lefkowitz, 
Reports Clearance Officer, AHRQ, by email at 
[email protected].
    Copies of the proposed collection plans, data collection 
instruments, and specific details on the estimated burden can be 
obtained from the AHRQ Reports Clearance Officer.

FOR FURTHER INFORMATION CONTACT: Doris Lefkowitz, AHRQ Reports 
Clearance Officer, (301) 427-1477, or by emails at 
[email protected].

SUPPLEMENTARY INFORMATION:

Proposed Project

``Patient-Centered Outcomes Research Clinical Decision Support: Current 
State and Future Directions''

    Research has shown that health care quality in the U.S. varies 
significantly and only half of adults receive evidence-based, 
recommended care. Individuals with multiple chronic conditions (42% of 
adults) and older adults are at particular risk for negative health 
outcomes. Current evidence shows that clinical decision support (CDS) 
systems improve adherence to evidence-based practices by analyzing 
patient data and making appropriate information available to the 
physician at the time they need it. CDS systems are usually electronic 
health record (EHR)-based, encompassing tools like alerts, clinical 
guidelines, patient reports and dashboards, diagnostic support, and 
workflow tools. These tools help reduce clinical errors and allow for 
customization to patient needs, improving quality of care and patient 
outcomes.
    The AHRQ Patient-Centered (PC) CDS Learning Network (PC CDS LN) 
defines PC CDS as: ``CDS that supports individual patients and their 
approved care givers and/or care teams in health-related decisions and 
actions by leveraging information from PCOR findings and/or patient-
specific information (e.g., patient-generated health data).'' Through 
PC CDS, AHRQ seeks to accelerate the movement of patient-centered 
outcomes research (PCOR) evidence into practice and to make CDS more 
shareable, standards-based, and publicly available.
    Traditionally, CDS initiatives have focused on provider-directed 
guidelines and increasing the shareability of CDS artifacts; however, 
PC CDS targets both patients (and/or caregivers) and providers.
    AHRQ's effort to support PC CDS has included efforts such as the PC 
CDS LN, CDS Connect, and other related grants and contracts. In this 
project, AHRQ seeks to conduct a comprehensive evaluation to assess the 
impact of AHRQ's PCOR CDS Initiative (the Initiative) on understanding 
of the current state of PC CDS and to identify gaps to guide AHRQ's 
future research.
    This research has the following goal:
    To assess the accomplishments and opportunities for the Initiative 
as a whole, and each of its four individual components: The PC CDS 
Learning Network, CDS Connect, Quantifying Efficiencies, and the U18 
CDS Resource Grants.
    This study is being conducted by AHRQ through its contractor, NORC 
at the University of Chicago, pursuant to AHRQ's statutory authority to 
disseminate government-funded research relevant to comparative clinical 
effectiveness research. 42 U.S.C. 299b-37(a)-(c).

Method of Collection

    To achieve these goals, the evaluation team will use key informant 
interviews and a web-based survey to gather information about the 
programs from stakeholders, contributors, and users of the CDS 
Initiative programs.
    Key Informant Interviews: The evaluation team will conduct semi-
structured interviews with people involved in the Initiative's 
components, including representatives from academia, industry, health 
systems, and government. Key informants will include the following 
groups:
    Leaders: Includes AHRQ project officers, contractor's senior staff, 
and senior consultants to Initiative components. Leaders are expected 
to have set the direction of the components or activities and to be 
familiar with the activities, the processes of implementation, and 
their outputs in their entirety.
    Contributors: Includes lead authors or content developers for a 
product or output of a component, and may overlap with leaders. 
Examples of contributors from the PC CDS LN include lead authors of the 
Trust Framework, Opioid Action Plan, or Patient Blogs; examples from 
the CDS Connect include individuals who contributed CDS artifacts to 
the repository.
    Participants: Includes individuals who participated in workgroups 
of either the PC CDS LN or CDS Connect, or participated in the 
development of one of the products.
    Consumers: Includes individuals who have used a product developed 
by the Initiative, including artifacts found on the CDS Connect 
repository and the CDS Connect Authoring Tool in particular. 
Individuals will be identified from interviews with leaders, 
contributors, and participants, and through literature review for 
authors making references to Initiative products (i.e., reports or 
artifacts).
    AHRQ and the evaluation contractor will create a list of eligible 
key informants that reflect the appropriate mix of roles and depth of 
experience to ensure comprehensive evaluation. Key informants will 
receive invitational emails that explain the scope and allow candidates 
to ask questions before declining or accepting the invitation. We will 
include clinical staff in our sample of participants in the Quantifying 
Efficiencies grant program, the U18 grants and the two opioid-related 
CDS projects. Involving staff at clinical sites will also be critical 
to understanding the value of PC CDS in the context of provider 
workflows and burdens.
    Web Survey: The purpose of the web survey is to understand more 
about who the users of CDS Connect resources are, their reasons for 
using the resources, how they use these resources, and their 
perceptions about their value. The CDS Connect resources of interest 
include the CDS Authoring Tool, artifacts in the CDS Connect Repository 
and open-source CDS Connect resources available

[[Page 16944]]

on Github, a platform for developing and sharing software. Respondents 
will be identified through a chain-referral methodology. The first set 
of survey invitations will be sent to a list of email addresses of 
known contributors or users of CDS Connect as well as a group of 
potential users of CDS Connect. At the end of the survey, each 
respondent will be asked to provide names and email addresses for up to 
four other users of CDS Connect resources. After the list of names from 
all referrals is deduplicated, a survey invitation will be sent to 
these referrals.
    The survey instrument includes multiple choice questions that 
capture important data points about use of CDS Connect resources, 
specifically the CDS Authoring tool, GitHub resources, and artifacts 
from the CDS Repository. Respondents will only be presented with more 
detailed questions about CDS Connect resource usage based on their 
responses to initial screening questions. The survey will take ten 
minutes on average to complete based on in-house testing.
    This mixed methods evaluation seeks to answer the following 
research questions about the Initiative as a whole:
    1. To what extent has the Initiative promoted the dissemination and 
implementation of PCOR findings through sharable, standards-based, and 
publicly available CDS and how?
    2. What activities carried out through each component (e.g., 
webinars, workgroups, in-person meetings, repositories, CDS artifacts 
and development tools, final reports or plans) were found to be most 
successful in furthering the various goals of the Initiative?
    3. What do stakeholders perceive to be the impacts of the 
Initiative to date, including reflection on their own involvement in 
it, and current or potential achievements, such as the development of a 
common definition of PC CDS and growth of interest in and capacity for 
developing these types of CDS among stakeholders?
    4. How does the Initiative address federal policies for the 
dissemination and implementation of evidence-based research funded by 
the PCOR Trust Fund, and how do they interact with other federal policy 
initiatives designed to promote widespread use, interoperability and 
patient access to information from EHRs with advanced CDS.
    5. What can AHRQ learn from the Initiative that is relevant to 
other initiatives aimed at disseminating and implementing clinical 
evidence and evidence-based practices? How can the lessons learned here 
inform future research, implementation, and dissemination initiatives?
    Information collected by the study will inform strategies to 
promote the adoption of PCOR evidence into practice through CDS 
developed by AHRQ and other Department of Health and Human Services 
agencies, including the Centers for Medicare & Medicaid Services (CMS) 
and the Office of the National Coordinator for Health IT, as well as 
state and local governments and private health care organizations. 
Findings from the evaluation can help identify and shape strategies to 
promote more effective implementation of PCOR CDS in order to 
accelerate the movement of evidence into clinical practice and support 
patient-centered decision making by clinicians with their patients.

Estimated Annual Respondent Burden

    Key Informant Interviews. Key informant interviews will be 
conducted with up to 147 key informants across a variety of 
organizations involved in each component of the Initiative. NORC will 
use one of 14 interview protocols based on the component the key 
informant is involved in and their role in that component. As shown in 
Exhibit 1, the interview form names include the type of role of the key 
informant in the project. All interviews are expected to last one hour. 
Some key informants may serve multiple roles or work on multiple 
projects. In these cases, the relevant protocols will be combined and 
streamlined so that the informant only completes one interview. Some of 
the key informant interviews for the sites or Opioid-related grants may 
be conducted during the course of site visits at the implementation 
sites, either with individuals or small groups of respondents.
    Web Survey. For the web survey, it is estimated that 453 CDS 
Connect users will respond to the 10-minute survey. The total annual 
burden hours for the key informant interviews and surveys is estimated 
to be 224 hours as shown in Exhibit 1.

                                  Exhibit 1--Estimated Annualized Burden Hours
----------------------------------------------------------------------------------------------------------------
                                                                     Number of       Hours per     Total burden
                            Form name                               respondents      response          hours
----------------------------------------------------------------------------------------------------------------
PC CDS Learning Network--Leader.................................               7               1               7
PC CDS Learning Network--Governance/Non-Executive Steering                     3               1               3
 Committee......................................................
PC CDS Learning Network--Contributor............................               8               1               8
CDS Connect--Leader.............................................               5               1               5
CDS Connect--Contributor........................................              20               1              20
CDS Connect--Consumer/Patient...................................              25               1              25
CDS Connect--Participant........................................              10               1              10
Quantifying Efficiencies--Leader................................               5               1               5
Quantifying Efficiencies--Informaticist.........................               4               1               4
Quantifying Efficiencies--Clinician.............................               8               1               8
PC CDS Projects -Site Leader....................................              18               1              18
PC CDS Projects--Informaticist..................................              10               1              10
PC CDS Projects--Clinician......................................              20               1              20
PC CDS Projects--Patient........................................               4               1               4
Web Survey of CDS Connect Users.................................             453             .17              77
rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr
    Total.......................................................             600  ..............             224
----------------------------------------------------------------------------------------------------------------

    Exhibit 2 shows the estimated annual cost burden associated with 
the respondents' time to participate in this information collection, 
which comes to $14,371.85.

[[Page 16945]]



                                   Exhibit 2--Estimated Annualized Cost Burden
----------------------------------------------------------------------------------------------------------------
                                                     Number of     Total burden   Average hourly    Total cost
                    Form name                      interviews *        hours       wage rate **       burden
----------------------------------------------------------------------------------------------------------------
PC CDS Learning Network--Leader.................               7               7      \1\ $59.54         $416.78
PC CDS Learning Network--Governance/Non-                       3               3       \1\ 59.54          178.62
 Executive Steering Committee...................
PC CDS Learning Network--Contributor............               8               8       \1\ 59.54          476.33
CDS Connect--Leader.............................               5               5       \1\ 59.54          297.71
CDS Connect--Contributor........................              20              20       \1\ 59.54        1,190.82
CDS Connect--Consumer...........................              25              25       \1\ 59.54        1,488.53
CDS Connect--Participant........................              10              10       \1\ 59.54          595.41
Quantifying Efficiencies--Leader................               5               5       \1\ 59.54          297.71
Quantifying Efficiencies--Informaticist.........               4               4       \1\ 59.54          238.16
Quantifying Efficiencies--Clinician.............               8               8      \1\ 101.43          811.46
PC CDS Projects -Site Leader....................              18              18       \1\ 59.54        1,071.74
PC CDS Projects--Informaticist..................              10              10       \1\ 59.54          595.40
PC CDS Projects--Clinician......................              20              20      \2\ 101.43        2,028.60
PC CDS Projects--Patient........................               4               4       \1\ 24.98           99.93
Web Survey of CDS Connect Users.................             453              77       \2\ 59.54        4,584.66
rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr
    Total.......................................             600             224  ..............       14,371.85
----------------------------------------------------------------------------------------------------------------
** Wage rates were calculated using the mean hourly wage from the U.S. Department of Labor, Bureau of Labor
  Statistics, May 2018 National Occupational Employment and Wage Estimates for the United States, https://www.bls.gov/oes/current/oes_nat.htm.
\1\ Average rate for Computer Information and Research Scientists.
\2\ Average rate for Physicians and Surgeons.
\3\ Average rate for All Occupations.

Request for Comments

    In accordance with the Paperwork Reduction Act, comments on AHRQ's 
information collection are requested with regard to any of the 
following: (a) Whether the proposed collection of information is 
necessary for the proper performance of AHRQ's health care research and 
health care information dissemination functions, including whether the 
information will have practical utility; (b) the accuracy of AHRQ's 
estimate of burden (including hours and costs) of the proposed 
collection(s) of information; (c) ways to enhance the quality, utility 
and clarity of the information to be collected; and (d) ways to 
minimize the burden of the collection of information upon the 
respondents, including the use of automated collection techniques or 
other forms of information technology.
    Comments submitted in response to this notice will be summarized 
and included in the Agency's subsequent request for OMB approval of the 
proposed information collection. All comments will become a matter of 
public record.

    Dated: March 19, 2020.
Virginia L. Mackay-Smith,
Associate Director.
[FR Doc. 2020-06208 Filed 3-24-20; 8:45 am]
 BILLING CODE 4160-90-P