[Federal Register Volume 85, Number 49 (Thursday, March 12, 2020)]
[Notices]
[Pages 14487-14489]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 2020-05027]


=======================================================================
-----------------------------------------------------------------------

DEPARTMENT OF HEALTH AND HUMAN SERVICES

Agency for Healthcare Research and Quality


Agency Information Collection Activities: Proposed Collection; 
Comment Request

AGENCY: Agency for Healthcare Research and Quality, HHS.

ACTION: Notice.

-----------------------------------------------------------------------

SUMMARY: This notice announces the intention of the Agency for 
Healthcare Research and Quality (AHRQ) to request that the Office of 
Management and Budget (OMB) approve the proposed information collection 
project ``Evaluation of Learning Health Systems K12 Training Program.''

DATES: Comments on this notice must be received by 60 days after date 
of publication.

ADDRESSES: Written comments should be submitted to: Doris Lefkowitz, 
Reports Clearance Officer, AHRQ, by email at 
[email protected].
    Copies of the proposed collection plans, data collection 
instruments, and specific details on the estimated burden can be 
obtained from the AHRQ Reports Clearance Officer.

FOR FURTHER INFORMATION CONTACT: Doris Lefkowitz, AHRQ Reports 
Clearance Officer, (301) 427-1477, or by email at 
[email protected].

SUPPLEMENTARY INFORMATION: 

Proposed Project: Evaluation of Learning Health Systems K12 Training 
Program

    AHRQ, in partnership with the Patient-Centered Outcomes Research 
Institute (PCORI), supports an innovative institutional mentored career 
development program (K12) to train clinician and research scientists to 
conduct patient-centered outcomes research within learning health 
systems (LHSs). LHSs provide an environment where science generated 
from health services research, patient-centered outcomes research 
(PCOR), and clinical research; informatics; incentives; and culture are 
aligned for continuous improvement and innovation. In addition, in an 
LHS, best practices are seamlessly embedded in the care process, in 
which stakeholders (i.e., providers, patients, and families) are active 
participants in all elements, and new knowledge is captured as an 
integral by-product of the care experience. The following are the LHS 
K12 training program objectives:

 Develop and implement a training program that includes both 
didactic and experiential learning and embeds the scholars in training 
at the interface of research, informatics, and clinical operations 
within LHSs
 Identify, recruit, and train clinician and research scientists 
who are committed to conducting PCOR in healthcare settings that 
generate new evidence to facilitate rapid implementation of practices 
that will improve quality of care and patient outcomes
 Establish Centers of Excellence (COEs) in LHS Research 
Training, focusing on the application and mastery of the newly 
developed core LHS researcher competencies
 Promote cross-institutional scholar-mentor interactions, 
cooperation on multisite projects, dissemination of project findings, 
methodological advances, and development of a shared curriculum

    The purpose of this evaluation is to assess the overall achievement 
of the LHS K12 training program's objectives, outcomes, and impact, as 
well as the program's value to its stakeholders. The information 
collected through this data collection will allow AHRQ to improve the 
LHS K12 program and identify whether results correspond to intentional 
changes in program strategy and implementation.
    This study is being conducted by AHRQ through its contractor, 2M 
Research, pursuant to AHRQ's statutory authority to ``build capacity 
for comparative clinical effectiveness research by establishing a grant 
program that provides for the training of researchers in the methods 
used to conduct such research.'' 42 U.S.C. 299b-37(e).

Method of Collection

    The evaluation will include two types of data collection: (1) Semi-
structured interviews with scholars who are close to completing the LHS 
K12 training program, their health system advisors, and program 
directors of each of the 11 institutions; and (2) surveys with health 
system advisors. The proposed data collection spans three years (2020-
2023).
    To achieve the goals of this project the following data collections 
will be implemented.

[[Page 14488]]

    1. Scholar Interview: Interviews with LHS K12 scholars assess the 
degree of scholar embeddedness in their respective health systems and 
understand which aspects of the training program were most and least 
successful. Telephone interviews will be conducted one time with 
scholars who are currently enrolled but close to (within 2 to 3 months 
of) completing the LHS K12 training program. The total number of 
scholars interviewed will be approximately up to 137 (or approximately 
46 scholars annually).
    2. Health System Advisor Interview: Interviews with scholars' 
health system advisors assess the perceived value of the LHS K12 
training program to the health system and the role of health system 
advisors in supporting the research conducted by LHS K12 scholars. One 
health system advisor from each scholar's advisory committee will be 
interviewed by telephone. Health system advisors selected for 
interviews will include those with direct involvement with or knowledge 
of the LHS K12 scholars' research projects. Health system advisors will 
be interviewed once around the same time that the scholar is 
interviewed. The total number of health system advisors interviewed 
will be approximately up to 137 (or approximately 46 health system 
advisors annually).
    3. Program Director Interview: Interviews with LHS K12 program 
directors assess the perceived value of the LHS K12 training program to 
the health system and the role of health system advisors in supporting 
the LHS K12 training program. The program director of each of the 11 
grantee institutions participating in the LHS K12 program will be 
interviewed by telephone in the final year of the LHS K12 program. The 
total number of program directors interviewed will be 11 (or 
approximately 4 program directors annually).
    4. Health System Advisor Survey: Pre-post surveys with scholars' 
health system advisors measure change in attitudes toward the role of 
health systems research and the importance of patient, family, and 
other stakeholder engagement in research. A brief survey will be 
administered electronically to health system advisors at two time 
points: Once at the beginning and conclusion of their respective 
scholar's training. The total number of health system advisors surveyed 
will be approximately up to 237 (or approximately 79 health system 
directors annually).
    AHRQ will use the information collected through this Information 
Collection Request to assess the program progress of the LHS K12 
training program, and impact to its LHS stakeholders in a prospective 
manner. The information collected will facilitate program planning.

Estimated Annual Respondent Burden

    Table 1 shows the estimated annualized burden hours for the 
respondents' time to participate in this evaluation. Interviews will be 
conducted with a total of 285 respondents (137 scholars, 137 health 
system advisors, and 11 program directors), which is approximately 95 
respondents interviewed each year (46 scholars, 46 health system 
advisors, and 4 program directors). Each interview is expected to be 
approximately 60 minutes. Surveys will be conducted with a total of 237 
health system advisors (or approximately 79 health system advisors each 
year). The survey is expected to take approximately 10 minutes. The 
total hour burden is expected to be 328.29 hours (or approximately 
109.43 hours each year) for this participant data collection effort.

                                   Table 1--Estimated Annualized Burden Hours
----------------------------------------------------------------------------------------------------------------
                                                                                                  Total annual
             Instrument                Estimated number     Frequency of     Average time per   burden estimate
                                        of respondents        response       response (hours)       (hours)
----------------------------------------------------------------------------------------------------------------
Scholar Interviews..................                 46                  1               1.00              46.00
Health System Advisor Interviews....                 46                  1               1.00              46.00
Program Director Interviews.........                  4                  1               1.00               4.00
Health System Advisor Surveys.......                 79                  1               0.17              13.43
                                     ---------------------------------------------------------------------------
    Estimated Annual Total..........                175  .................  .................             109.43
----------------------------------------------------------------------------------------------------------------

    Table 2 shows the estimated annualized cost burden based on the 
respondents' time to participate in this project. This cost was 
calculated using average hourly earnings for May 2018, obtained from 
the Bureau of Labor Statistics' estimates for occupational employment 
wages. The total estimated annualized cost burden for this data 
collection is $7,649.07. The following hourly wages were used in the 
annualized cost calculations: $37.38 per hour for a scholar, $96.22 per 
hour for a health system advisor, and $52.81 per hour for a program 
director.

                                    Table 2--Estimated Annualized Cost Burden
----------------------------------------------------------------------------------------------------------------
                                                            Total annual
             Instrument                Estimated number   burden estimate      Hourly rate         Total cost
                                        of respondents        (hours)
----------------------------------------------------------------------------------------------------------------
Scholar Interviews *................                 46              46.00             $37.38          $1,719.48
Health System Advisor Interviews **.                 46              46.00              96.22           4,426.12
Program Director Interviews ***.....                  4               4.00              52.81             211.24
Health System Advisor Surveys **....                 79              13.43              96.22           1,292.23
                                     ---------------------------------------------------------------------------
    Estimated Annual Total..........                175             109.43  .................           7,649.07
----------------------------------------------------------------------------------------------------------------
Bureau of Labor Statistics (BLS), U.S. Department of Labor. (2018). Occupational employment statistics May 2018
  national wages. https://www.bls.gov/oes/home.htm.
* The hourly wage for scholars varies depending on the scholar's degree. AHRQ averaged hourly wages using the
  following occupations code to develop an estimate that represents the mix of medical and academic degrees: 29-
  0000, 29-1000, 21-0000.
** AHRQ anticipates that many health system advisors will be C-suite leaders. The hourly wage for BLS's
  occupation code 11-1010 (chief executive) was used for this estimate.

[[Page 14489]]

 
*** Program directors hold various roles and responsibilities and, therefore, have varied salaries. For the
  purpose of this estimate, the hourly wages for the following managerial and post-secondary occupational codes
  were averaged: 11-3131,11-1021,11-9030,11-9033,11-9039, and 11-9199.

Request for Comments

    In accordance with the Paperwork Reduction Act, comments on AHRQ's 
information collection are requested with regard to any of the 
following: (a) Whether the proposed collection of information is 
necessary for the proper performance of AHRQ's health care research and 
health care information dissemination functions, including whether the 
information will have practical utility; (b) the accuracy of AHRQ's 
estimate of burden (including hours and costs) of the proposed 
collection(s) of information; (c) ways to enhance the quality, utility 
and clarity of the information to be collected; and (d) ways to 
minimize the burden of the collection of information upon the 
respondents, including the use of automated collection techniques or 
other forms of information technology.
    Comments submitted in response to this notice will be summarized 
and included in the Agency's subsequent request for OMB approval of the 
proposed information collection. All comments will become a matter of 
public record.

    Dated: March 6, 2020.
Virginia L. Mackay-Smith,
Associate Director.
[FR Doc. 2020-05027 Filed 3-11-20; 8:45 am]
BILLING CODE 4160-90-P