[Federal Register Volume 80, Number 129 (Tuesday, July 7, 2015)]
[Notices]
[Pages 38716-38718]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 2015-16573]
-----------------------------------------------------------------------
DEPARTMENT OF HEALTH AND HUMAN SERVICES
Substance Abuse and Mental Health Services Administration
National Registry of Evidence-Based Programs and Practices
AGENCY: Substance Abuse and Mental Health Services Administration
(SAMHSA), HHS.
ACTION: Notice regarding SAMHSA's NREPP: Redesign of NREPP.
-----------------------------------------------------------------------
SUMMARY: The mission of SAMHSA is to reduce the impact of substance
abuse and mental illness on America's communities. Established in 1992,
the agency was directed by Congress to target effective substance abuse
and mental health services to the people most in need, and to translate
research in these areas more effectively and more rapidly into the
general health care system. NREPP is a key public resource SAMHSA has
developed to help meet this directive. This notice announces the
redesign of NREPP to better align the registry with the standards and
processes of other evidence-based repositories. A re-launch of the Web
site with revised content is anticipated in late fall/early winter
2015.
The notice explains the changes in how programs and practices will
be identified for NREPP, how submissions will be screened and reviewed,
and provides guidance on accessing updated information on the NREPP
site. Potential applicants should be aware that this notice includes
updated information relating to the eligibility of interventions for
inclusion in NREPP and changes in the program and practice review
process that supersedes guidance provided in earlier Federal Register
notices.
FOR FURTHER INFORMATION CONTACT: Carter Roeber, Ph.D., Social Science
Analyst, Center for Behavioral Health Statistics and Quality, SAMHSA, 1
Choke Cherry Road, Room 2-1050, Rockville, MD 20857, telephone 240-276-
1488.
SUPPLEMENTARY INFORMATION:
Advancing Evidence-Based Programs and Practices Through Improved
Decision Support Tools: Reconceptualizing NREPP
Introduction
SAMHSA's NREPP is an evidence-based repository and review system
designed to provide the public with reliable information about
behavioral health interventions in the areas of mental health and
substance abuse. Programs and practices that are accepted for inclusion
in NREPP undergo a review process that provides information on the
quality of research and the magnitude and direction of program or
practice impact on individual outcomes. Materials for dissemination are
reviewed to determine the type and extent of information available to
support implementation. The results of these reviews are published on
the NREPP Web site (http://nrepp.samhsa.gov).
It should be noted that inclusion in NREPP indicates that some, but
not necessarily all, of the evidence for a program or practice has been
reviewed. In some cases, the quality of the research supporting the
program or practice may have been determined to be poor or insufficient
to earn a rating. Inclusion in NREPP does not constitute endorsement of
an intervention as effective by SAMHSA. Moreover, since NREPP has not
reviewed all behavioral health interventions, the use of NREPP as an
exclusive or exhaustive list of interventions is not appropriate.
Policymakers and funders in particular are discouraged from limiting
providers and/or potential grantees to selecting exclusively from among
NREPP interventions and from funding NREPP interventions regardless of
the ratings the interventions receive.
This notice announces changes to (1) the process for identifying
new programs and practices for NREPP review, (2) the process for
announcing open submission periods, (3) the minimum requirements to be
considered for NREPP review, and (4) the review process. This notice
also announces the intent to re-review currently posted NREPP programs
and practices to comport with new review criteria and ratings. The re-
review of programs and practices currently posted will take place over
the course of the next several years, depending on available resources.
A re-launch of the NREPP Web site will take place in phases and the
first phase is planned for late fall/early winter of 2015.
Identifying New Programs and Practices for NREPP
Open submissions periods, during which applicants may submit
materials for review, will continue to be used to identify new programs
and practices for review (see below). Programs and practices addressing
specific SAMHSA priorities may also be identified by SAMHSA or through
environmental scans (including literature reviews, focus groups, public
input, and interviews), as time and resources permit. Programs and
practices related to priority areas may be reviewed before programs and
practices identified through the open submission period. SAMHSA will be
consulting with subject matter experts and leadership in underserved
groups and populations, including American Indian/Alaska Native Tribes,
regarding ways to incorporate traditional and culturally-specific
interventions into NREPP, in order to better meet the needs of groups
whose efforts to promote behavioral health may not have not been
routinely evaluated. Innovative, but perhaps less rigorously tested,
programs and
[[Page 38717]]
practices will be considered if specific to SAMHSA priority areas, as a
parallel contribution to the standard review procedures.
Stand-alone pharmacologic treatments are not eligible for review
and should not be submitted to NREPP. The evidence base for
pharmacologic treatments is reviewed and approved through the U.S. Food
and Drug Administration (FDA). FDA approved pharmacotherapy
interventions (on-label use) are considered for NREPP review only when
combined with one or more behavioral or psychosocial treatments.
SAMHSA reserves the right to reject for review programs and
practices whose goals or activities are determined to be inconsistent
with SAMHSA's mission, which is ``to reduce the impact of substance
abuse and mental illness on American communities.''
Open Submission Periods
SAMHSA accepts new applications for review during open submission
periods. SAMHSA generally holds one open submission period a year but,
depending on the number of reviews in progress and resources available
may hold more or fewer within a calendar year. All future open
submission periods will be announced on the NREPP Web site. Emails will
also be sent announcing the open submission period to those on the
NREPP listserv. Anyone wishing to be notified of future open submission
periods can join the NREPP listserv by sending a request to
[email protected].
Applications can be submitted at any point during an open period.
Program and practice developers, researchers, and others interested in
submitting an intervention should read below for information about the
new minimum requirements to be considered for an NREPP review.
Additional future changes to the review process and criteria will be
posted on the NREPP Web site as they are implemented. Therefore, before
submitting a program or practice for NREPP review, applicants should
examine the most recent information posted on the NREPP Web site about
the review process and criteria and the most recent guidance for
preparing an intervention for submission (see http://www.nrepp.samhsa.gov/ReviewSubmission.aspx). This guidance will be
periodically updated to reflect the redesign of NREPP during the re-
launch.
The selection of interventions will take place after the closing of
the open submission period, and applicants will be notified whether
they have been accepted after an initial screening to ensure that the
application meets minimum requirements. The number of reviews conducted
will depend on the availability of funds, with the timing of reviews to
be determined by SAMHSA. In submitting an intervention, applicants
should understand that if interventions are selected for review, the
results of NREPP reviews are considered public information and will be
posted on the NREPP Web site. Once a review is completed, the applicant
will be provided with a summary document (``the program or practice
profile'') that presents results of the review, ratings of program
effectiveness, and descriptive information about the intervention. The
applicant will have the opportunity to comment on the profile before it
is posted but they will not have the option to refuse posting.
Minimum Requirements
To be considered for review, interventions must meet three minimum
requirements:
1. Research or evaluation of the intervention has assessed mental
health or substance use outcomes among individuals, communities, or
populations OR other behavioral health-related outcomes on individuals,
communities, or populations with or at risk of mental health issues or
substance use problems.
2. Evidence of these outcomes has been demonstrated in at least one
study using an experimental or quasi-experimental design. Experimental
designs require random assignment, a control or comparison group, and
pre- and post-intervention outcome assessments. Quasi-experimental
designs do not require random assignment, but do require a comparison
or control group and pre- and post-intervention outcome assessments.
Comparison/control groups must be a no-treatment control group, a wait-
list control group, a treatment-as-usual comparison group, or an
intervention that is presumed to be ineffective or substantially less
effective than the intervention (e.g., a ``placebo'' control or, in
cases in which providing no treatment might be considered unethical,
less effective treatments, even if not treatment-as-usual, such as
``supportive therapy''). Studies with single-group, pretest-posttest
designs or single-group, longitudinal/multiple time series do not meet
this requirement, but will be considered to identify emerging programs
and practices for consideration in the Learning Center. Comparative
effectiveness trials, in which two interventions, both presumed to be
equally effective, are compared, and studies in which the effects of
the same intervention on various subpopulations are compared or in
which various doses or components of the same intervention are compared
will be reviewed, but only for purposes of providing readers in the
Learning Center the opportunity to determine whether a particular
program or practice may have use or show promise in different settings
or circumstances.
3. The results of these studies have been published in a peer-
reviewed journal or other professional publication, or documented in a
comprehensive evaluation report, published within the previous 25
years. Comprehensive evaluation reports must include a review of the
literature, theoretical framework, purpose, methodology, findings/
results with statistical analysis and p values for significant
outcomes, discussion, and conclusions.
Applicants are required to provide full-text documents at the time
of submission that demonstrate the intervention meets these minimum
requirements. Other research articles, published or unpublished
evaluation reports, grant final reports, and replication studies may be
submitted as additional supporting documentation. Note: Abstracts or
links to partial articles are regarded as incomplete and will not be
considered.
NREPP will no longer require programs and practices to have
developed implementation materials, training and support resources, and
quality assurance procedures. However, programs and practices with such
dissemination and/or implementation resources will be considered for
prioritized review if within the priorities established by the SAMHSA
review process, and the materials, along with the location of their
availability, will be listed in the program or practice profile.
Applicants submitting dissemination and implementation resources
should include a brief narrative description of the materials that are
being submitted. These materials may include, but are not limited to,
treatment manuals, information for administrators, information for
direct service staff, tested training curricula, mechanisms for ongoing
supervision and consultation, protocols for gathering process and
outcome data, ongoing monitoring of intervention fidelity, and
processes for gathering feedback. Applicants should also provide the
location of where the materials can be obtained.
[[Page 38718]]
Selection of Interventions for Review
SAMHSA will select interventions for review from among submissions
meeting the minimum requirements. In selecting interventions for
review, SAMHSA may give special consideration to interventions that
meet one or more of the following conditions:
More than one research study or evaluation has been
conducted on the same or a similar target population that meets the
minimum requirements.
The intervention targets underserved populations (e.g.,
minority populations, tribal communities or American Indian/Alaska
Native populations, elderly individuals, young adults, individuals who
are incarcerated, etc.).
Dissemination and implementation materials (e.g., program
or practice manuals, training guides, measurement instruments,
implementation fidelity tools) are available. Lower costs and no-cost
materials may be prioritized.
The intervention contributes to a content area in which
few evidence-based interventions have been previously identified.
Interventions that are not selected for review may be resubmitted
by the applicant in a future open submission period.
The Review Process
The review process has been revised to improve the quality of the
reviews and utility of information that NREPP can provide its users. In
addition to articles and reports submitted by NREPP applicants,
additional studies, articles, and evaluation reports regarding the
interventions will be identified through literature searches. Studies
and outcomes to be reviewed will be determined through the systematic
application of standardized screening criteria, and the number of
studies and outcomes to be reviewed will be expanded to more
comprehensively represent the evidence base for the program or
practice. Inclusion of studies and outcomes will no longer be limited
to positive significant outcomes; all studies and outcomes that meet
the standardized screening criteria will be reviewed, including those
with negative and non-significant effects. Programs and practices will
be assessed on the basis of evaluation studies of program or practice
impact, information related to conceptual framework (that is, program
or practice goals, theory of change, and program or practice
components), and information about implementation fidelity (that is,
whether a study employs quality assurance measures to declare that the
program or practice is delivered as intended to the program's or
practice's target population).
The methodological rigor (that is, internal validity, statistical
validity, and measurement validity) of the research for each program or
practice will be reviewed, as it pertains to each outcome examined,
along with the magnitude and direction of the program's or practice's
effect on each outcome. Based on this information, the program's or
practice's effectiveness for each outcome will be rated, along with the
rigor of the research examining the program or practice, and the
ratings will be displayed on the NREPP Web site.
In general, each NREPP evidence review will be conducted by two
trained and certified reviewers. However, based on funding and
available resources, SAMHSA use one reviewer for programs and practices
being re-reviewed. When necessary, NREPP may conduct author queries to
confirm or gather additional information needed for the review. Program
and practice profiles will be developed on the basis of the information
gathered. Applicants will have the opportunity to review the program or
practice profile before it is posted on the NREPP site, but they will
not have the option to refuse posting.
Dissemination and implementation materials will no longer be rated
as they were historically. Instead, descriptions of available materials
for each program or practice, highlighting information that may be of
most interest to NREPP users, will be included in the program or
practice profile, along with information documenting the extent to
which materials are available.
Programs and practices currently posted on NREPP will be re-
reviewed as time and resources permit but the re-reviews of currently
posted programs and practices will take place over the next few years.
Detailed information about the revised review process will be
available at http://www.nrepp.samhsa.gov after the re-launch of the new
NREPP Web site.
Enhancing the Learning Center
NREPP's Learning Center is a developing and underutilized component
of the NREPP Web site. With the evolution and enhancement of the
registry, SAMHSA seeks to bring greater recognition to both rigorously
evaluated behavioral health interventions and those interventions that
have been implemented, demonstrate promise, but have not necessarily
been evaluated in a rigorous manner. To that end, the Learning Center
is being significantly revamped to support stakeholder engagement and
to become a shared learning environment for all stakeholders. SAMHSA
recognizes that the successful promotion and dissemination of evidence-
based programs and practices requires an environment that promotes
community assessment, program and practice planning and evaluation, as
well as guidance on the selection and implementation of programs and
practices listed on NREPP. There are useful types of evaluation
research, often conducted among underserved populations, which provide
valuable insights for practitioners, but do not meet the minimum
criteria required for experimental or quasi-experimental design. SAMHSA
intends the Learning Center to be a forum for presenting research on
emerging programs and practices, and exploring ways that pre-
experimental and qualitative research can complement and enrich
findings from experimental and quasi-experimental research designs. An
inventory of such programs and practices will be compiled and
maintained within the Learning Center and will operate in parallel to
the listing of reviewed programs and practices with experimental and
quasi-experimental designs. In this way, SAMHSA intends to support
programs and practices researched with the most rigorous approaches
while also supporting the development of practice-based evidence,
especially for certain populations and emerging practices that are
critical to learning and improving behavioral health outcomes for
persons with or at risk of developing behavioral health issues.
Summer King,
Statistician.
[FR Doc. 2015-16573 Filed 7-6-15; 8:45 am]
BILLING CODE 4162-20-P