[Federal Register Volume 85, Number 122 (Wednesday, June 24, 2020)]
[Notices]
[Pages 37947-37949]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 2020-13574]


=======================================================================
-----------------------------------------------------------------------

DEPARTMENT OF HEALTH AND HUMAN SERVICES

Agency for Healthcare Research and Quality


Agency Information Collection Activities: Proposed Collection; 
Comment Request

AGENCY: Agency for Healthcare Research and Quality, HHS.

ACTION: Notice.

-----------------------------------------------------------------------

SUMMARY: This notice announces the intention of the Agency for 
Healthcare Research and Quality (AHRQ) to request that the Office of 
Management and Budget (OMB) approve the proposed information collection 
project: ``Patient-Centered Outcomes Research Clinical Decision 
Support: Current State and Future Directions.'' This proposed 
information collection was previously published in the Federal Register 
on March 25, 2020. AHRQ received no comments from members of the 
public. The purpose of this notice is to allow an additional 30 days 
for public comment.

DATES: Comments on this notice must be received by 30 days after date 
of publication of this notice.

ADDRESSES: Written comments and recommendations for the proposed 
information collection should be sent within 30 days of publication of 
this notice to www.reginfo.gov/public/do/PRAMain. Find this particular 
information collection by selecting ``Currently under 30-day Review--
Open for Public Comments'' or by using the search function.

FOR FURTHER INFORMATION CONTACT: Doris Lefkowitz, AHRQ Reports 
Clearance Officer, (301) 427-1477, or by email at 
[email protected].

SUPPLEMENTARY INFORMATION:

Proposed Project

``Patient-Centered Outcomes Research Clinical Decision Support: Current 
State and Future Directions''

    Research has shown that health care quality in the U.S. varies 
significantly and only half of adults receive evidence-based, 
recommended care. Individuals with multiple chronic conditions (42% of 
adults) and older adults are at particular risk for negative health 
outcomes. Current evidence shows that clinical decision support (CDS) 
systems improve adherence to evidence-based practices by analyzing 
patient data and making appropriate information available to the 
physician at the time they need it. CDS systems are usually electronic 
health record (EHR)-based, encompassing tools like alerts, clinical 
guidelines, patient reports and dashboards, diagnostic support, and 
workflow tools. These tools help reduce clinical errors and allow for 
customization to patient needs, improving quality of care and patient 
outcomes.
    The AHRQ Patient-Centered (PC) CDS Learning Network (PC CDS LN) 
defines PC CDS as: ``CDS that supports individual patients and their 
approved care givers and/or care teams in health-related decisions and 
actions by leveraging information from PCOR findings and/or patient-
specific information (e.g., patient-generated health data).'' Through 
PC CDS, AHRQ seeks to accelerate the movement of patient-centered 
outcomes research (PCOR) evidence into practice and to make CDS more 
shareable, standards-based, and publicly available.
    Traditionally, CDS initiatives have focused on provider-directed 
guidelines and increasing the shareability of CDS artifacts; however, 
PC CDS targets both patients (and/or caregivers) and providers.
    AHRQ's effort to support PC CDS has included efforts such as the PC 
CDS LN, CDS Connect, and other related grants and contracts. In this 
project, AHRQ seeks to conduct a comprehensive evaluation to assess the 
impact of AHRQ's PCOR CDS Initiative (the Initiative) on understanding 
of the current state of PC CDS and to identify gaps to guide AHRQ's 
future research.
    This research has the following goal:
    To assess the accomplishments and opportunities for the Initiative 
as a whole, and each of its four individual components: The PC CDS 
Learning Network, CDS Connect, Quantifying Efficiencies, and the U18 
CDS Resource Grants.
    This study is being conducted by AHRQ through its contractor, NORC 
at the University of Chicago, pursuant to AHRQ's statutory authority to 
disseminate government-funded research relevant to comparative clinical 
effectiveness research. 42 U.S.C. 299b-37(a)-(c).

Method of Collection

    To achieve these goals, the evaluation team will use key informant 
interviews and a web-based survey to gather information about the 
programs from stakeholders, contributors, and users of the CDS 
Initiative programs.
    Key Informant Interviews: The evaluation team will conduct semi-
structured interviews with people involved in the Initiative's 
components, including representatives from academia, industry, health 
systems, and government. Key informants will include the following 
groups:
    Leaders: Includes AHRQ project officers, contractor's senior staff, 
and senior consultants to Initiative components. Leaders are expected 
to have set the direction of the components or activities and to be 
familiar with the activities, the processes of implementation, and 
their outputs in their entirety.
    Contributors: Includes lead authors or content developers for a 
product or output of a component, and may overlap with leaders. 
Examples of contributors from the PC CDS LN include lead authors of the 
Trust Framework, Opioid Action Plan, or Patient Blogs; examples from 
the CDS Connect include individuals who contributed CDS artifacts to 
the repository.
    Participants: Includes individuals who participated in workgroups 
of either the PC CDS LN or CDS Connect, or participated in the 
development of one of the products.
    Consumers: Includes individuals who have used a product developed 
by the Initiative, including artifacts found on

[[Page 37948]]

the CDS Connect repository and the CDS Connect Authoring Tool in 
particular. Individuals will be identified from interviews with 
leaders, contributors, and participants, and through literature review 
for authors making references to Initiative products (i.e., reports or 
artifacts).
    AHRQ and the evaluation contractor will create a list of eligible 
key informants that reflect the appropriate mix of roles and depth of 
experience to ensure comprehensive evaluation. Key informants will 
receive invitational emails that explain the scope and allow candidates 
to ask questions before declining or accepting the invitation. We will 
include clinical staff in our sample of participants in the Quantifying 
Efficiencies grant program, the U18 grants and the two opioid-related 
CDS projects. Involving staff at clinical sites will also be critical 
to understanding the value of PC CDS in the context of provider 
workflows and burdens.
    Web Survey: The purpose of the web survey is to understand more 
about who the users of CDS Connect resources are, their reasons for 
using the resources, how they use these resources, and their 
perceptions about their value. The CDS Connect resources of interest 
include the CDS Authoring Tool, artifacts in the CDS Connect Repository 
and open-source CDS Connect resources available on Github, a platform 
for developing and sharing software. Respondents will be identified 
through a chain-referral methodology. The first set of survey 
invitations will be sent to a list of email addresses of known 
contributors or users of CDS Connect as well as a group of potential 
users of CDS Connect. At the end of the survey, each respondent will be 
asked to provide names and email addresses for up to four other users 
of CDS Connect resources. After the list of names from all referrals is 
deduplicated, a survey invitation will be sent to these referrals.
    The survey instrument includes multiple choice questions that 
capture important data points about use of CDS Connect resources, 
specifically the CDS Authoring tool, GitHub resources, and artifacts 
from the CDS Repository. Respondents will only be presented with more 
detailed questions about CDS Connect resource usage based on their 
responses to initial screening questions. The survey will take ten 
minutes on average to complete based on in-house testing.
    This mixed methods evaluation seeks to answer the following 
research questions about the Initiative as a whole:
    1. To what extent has the Initiative promoted the dissemination and 
implementation of PCOR findings through sharable, standards-based, and 
publicly available CDS and how?
    2. What activities carried out through each component (e.g., 
webinars, workgroups, in-person meetings, repositories, CDS artifacts 
and development tools, final reports or plans) were found to be most 
successful in furthering the various goals of the Initiative?
    3. What do stakeholders perceive to be the impacts of the 
Initiative to date, including reflection on their own involvement in 
it, and current or potential achievements, such as the development of a 
common definition of PC CDS and growth of interest in and capacity for 
developing these types of CDS among stakeholders?
    4. How does the Initiative address federal policies for the 
dissemination and implementation of evidence-based research funded by 
the PCOR Trust Fund, and how do they interact with other federal policy 
initiatives designed to promote widespread use, interoperability and 
patient access to information from EHRs with advanced CDS.
    5. What can AHRQ learn from the Initiative that is relevant to 
other initiatives aimed at disseminating and implementing clinical 
evidence and evidence-based practices? How can the lessons learned here 
inform future research, implementation, and dissemination initiatives?
    Information collected by the study will inform strategies to 
promote the adoption of PCOR evidence into practice through CDS 
developed by AHRQ and other Department of Health and Human Services 
agencies, including the Centers for Medicare & Medicaid Services (CMS) 
and the Office of the National Coordinator for Health IT, as well as 
state and local governments and private health care organizations. 
Findings from the evaluation can help identify and shape strategies to 
promote more effective implementation of PCOR CDS in order to 
accelerate the movement of evidence into clinical practice and support 
patient-centered decision making by clinicians with their patients.

Estimated Annual Respondent Burden

    Key Informant Interviews. Key informant interviews will be 
conducted with up to 147 key informants across a variety of 
organizations involved in each component of the Initiative. NORC will 
use one of 14 interview protocols based on the component the key 
informant is involved in and their role in that component. As shown in 
Exhibit 1, the interview form names include the type of role of the key 
informant in the project. All interviews are expected to last one hour. 
Some key informants may serve multiple roles or work on multiple 
projects. In these cases, the relevant protocols will be combined and 
streamlined so that the informant only completes one interview. Some of 
the key informant interviews for the sites or Opioid-related grants may 
be conducted during the course of site visits at the implementation 
sites, either with individuals or small groups of respondents.
    Web Survey. For the web survey, it is estimated that 453 CDS 
Connect users will respond to the 10-minute survey. The total annual 
burden hours for the key informant interviews and surveys is estimated 
to be 224 hours as shown in Exhibit 1.

                                  Exhibit 1--Estimated Annualized Burden Hours
----------------------------------------------------------------------------------------------------------------
                                                                     Number of       Hours per     Total burden
                            Form name                               respondents      response          hours
----------------------------------------------------------------------------------------------------------------
PC CDS Learning Network--Leader.................................               7               1               7
PC CDS Learning Network--Governance/Non-Executive Steering                     3               1               3
 Committee......................................................
PC CDS Learning Network--Contributor............................               8               1               8
CDS Connect--Leader.............................................               5               1               5
CDS Connect--Contributor........................................              20               1              20
CDS Connect--Consumer/Patient...................................              25               1              25
CDS Connect--Participant........................................              10               1              10
Quantifying Efficiencies--Leader................................               5               1               5
Quantifying Efficiencies--Informaticist.........................               4               1               4
Quantifying Efficiencies--Clinician.............................               8               1               8

[[Page 37949]]

 
PC CDS Projects--Site Leader....................................              18               1              18
PC CDS Projects--Informaticist..................................              10               1              10
PC CDS Projects--Clinician......................................              20               1              20
PC CDS Projects--Patient........................................               4               1               4
Web Survey of CDS Connect Users.................................             453             .17              77
                                                                 -----------------------------------------------
    Total.......................................................             600  ..............             224
----------------------------------------------------------------------------------------------------------------

    Exhibit 2 shows the estimated annual cost burden associated with 
the respondents' time to participate in this information collection, 
which comes to $14,371.85.

                                   Exhibit 2--Estimated Annualized Cost Burden
----------------------------------------------------------------------------------------------------------------
                                                                                      Average
                    Form name                        Number of     Total burden     hourly wage     Total cost
                                                    interviews*        hours          rate**          burden
----------------------------------------------------------------------------------------------------------------
PC CDS Learning Network--Leader.................               7               7      \1\ $59.54         $416.78
PC CDS Learning Network--Governance/Non-                       3               3       \1\ 59.54          178.62
 Executive Steering Committee...................
PC CDS Learning Network--Contributor............               8               8       \1\ 59.54          476.33
CDS Connect--Leader.............................               5               5       \1\ 59.54          297.71
CDS Connect--Contributor........................              20              20       \1\ 59.54        1,190.82
CDS Connect--Consumer...........................              25              25       \1\ 59.54        1,488.53
CDS Connect--Participant........................              10              10       \1\ 59.54          595.41
Quantifying Efficiencies--Leader................               5               5       \1\ 59.54          297.71
Quantifying Efficiencies--Informaticist.........               4               4       \1\ 59.54          238.16
Quantifying Efficiencies--Clinician.............               8               8      \2\ 101.43          811.46
PC CDS Projects--Site Leader....................              18              18       \1\ 59.54        1,071.74
PC CDS Projects--Informaticist..................              10              10       \1\ 59.54          595.40
PC CDS Projects--Clinician......................              20              20      \2\ 101.43        2,028.60
PC CDS Projects--Patient........................               4               4       \3\ 24.98           99.93
Web Survey of CDS Connect Users.................             453              77       \1\ 59.54        4,584.66
                                                 ---------------------------------------------------------------
    Total.......................................             600             224  ..............       14,371.85
----------------------------------------------------------------------------------------------------------------
** Wage rates were calculated using the mean hourly wage from the U.S. Department of Labor, Bureau of Labor
  Statistics, May 2018 National Occupational Employment and Wage Estimates for the United States, https://www.bls.gov/oes/current/oes_nat.htm.
\1\ Average rate for Computer Information and Research Scientists
\2\ Average rate for Physicians and Surgeons
\3\ Average rate for All Occupations

Request for Comments

    In accordance with the Paperwork Reduction Act, comments on AHRQ's 
information collection are requested with regard to any of the 
following: (a) Whether the proposed collection of information is 
necessary for the proper performance of AHRQ's health care research and 
health care information dissemination functions, including whether the 
information will have practical utility; (b) the accuracy of AHRQ's 
estimate of burden (including hours and costs) of the proposed 
collection(s) of information; (c) ways to enhance the quality, utility 
and clarity of the information to be collected; and (d) ways to 
minimize the burden of the collection of information upon the 
respondents, including the use of automated collection techniques or 
other forms of information technology.
    Comments submitted in response to this notice will be summarized 
and included in the Agency's subsequent request for OMB approval of the 
proposed information collection. All comments will become a matter of 
public record.

    Dated: June 18, 2020.
Virginia L. Mackay-Smith,
Associate Director.
[FR Doc. 2020-13574 Filed 6-23-20; 8:45 am]
BILLING CODE 4160-90-P