[Federal Register Volume 89, Number 44 (Tuesday, March 5, 2024)]
[Notices]
[Pages 15875-15876]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 2024-04526]


-----------------------------------------------------------------------

DEPARTMENT OF HEALTH AND HUMAN SERVICES

Food and Drug Administration

[Docket No. FDA-2023-N-0894]


Agency Information Collection Activities; Submission for Office 
of Management and Budget Review; Comment Request; The Real Cost Monthly 
Implementation Assessment

AGENCY: Food and Drug Administration, HHS.

ACTION: Notice.

-----------------------------------------------------------------------

SUMMARY: The Food and Drug Administration (FDA or we) is announcing 
that a proposed collection of information has been submitted to the 
Office of Management and Budget (OMB) for review and clearance under 
the Paperwork Reduction Act of 1995.

DATES: Submit written comments (including recommendations) on the 
collection of information by April 4, 2024.

ADDRESSES: To ensure that comments on the information collection are 
received, OMB recommends that written comments be submitted to https://www.reginfo.gov/public/do/PRAMain. Find this particular information 
collection by selecting ``Currently under Review--Open for Public 
Comments'' or by using the search function. The title of this 
information collection is ``The Real Cost Monthly Implementation 
Assessment.'' Also include the FDA docket number found in brackets in 
the heading of this document.

FOR FURTHER INFORMATION CONTACT: JonnaLynn Capezzuto, Office of 
Operations, Food and Drug Administration, Three White Flint North, 10A-
12M, 11601 Landsdown St., North Bethesda, MD 20852, 301-796-3794, 
[email protected].

SUPPLEMENTARY INFORMATION: In compliance with 44 U.S.C. 3507, FDA has 
submitted the following proposed collection of information to OMB for 
review and clearance.

The Real Cost Monthly Implementation Assessment

OMB Control Number 0910-NEW

    This information collection supports the development and 
implementation of FDA public education campaigns related to tobacco 
use. To reduce the public health burden of tobacco use in the United 
States and educate the public--especially young people--about the 
dangers of tobacco use, the FDA Center for Tobacco Products (CTP) is 
developing and implementing multiple public education campaigns.
    FDA launched ``The Real Cost'' in February 2014, seeking to reduce 
tobacco use among at-risk teens ages 12-17 in the United States who are 
open to smoking cigarettes and/or using electronic nicotine delivery 
systems (ENDS) products, or who have already experimented with 
cigarettes and/or ENDS products. Complementary evaluation studies, 
including the ``Evaluation of FDA's Public Education Campaign on Teen 
Tobacco,'' were implemented to measure awareness of ``The Real Cost'' 
paid media campaign among teens ages 12-17 in the United States, and to 
understand how awareness is related to change in key outcomes.
    Although outcome evaluation studies of ``The Real Cost'' have and 
continue to assess the impact of awareness on outcomes, no studies have 
sought to assess the implementation of ``The Real Cost.'' As FDA 
continues to increase the presence of ``The Real Cost'' on digital 
channels (e.g., Hulu, YouTube, Instagram), the need for an 
implementation evaluation has become clear as these messages are 
received by the target audience on digital channels differently 
compared to how the messages are received on broadcast channels. Before 
the migration of campaign ads to digital channels, ads from ``The Real 
Cost'' were primarily aired on broadcast TV. In the broadcast space, 
for people to avoid receiving the message, they needed to be proactive 
(e.g., finding the remote to change the channel or leaving the room). 
In the digital space, however, people need to be proactive to watch the 
full message, like stopping scrolling on social media or watching the 
full ad on YouTube. Assessment of this information is integral to 
understanding self-reported ad awareness levels, as well as how our 
audience experiences and processes the ads as they are airing in a 
digital setting.
    Therefore, we propose a study to help us understand, in a digital 
setting, how teens experience the messages, how they engage with 
messages, the extent to which teens report being exposed to messages, 
and how teens process the messages. Data gathered from this assessment 
will also provide the necessary and timely information to optimize 
campaign messages, the digital media buy (i.e., where, how, and when 
ads are shown), and creative rotations (i.e., which ads are shown).
    ``The Real Cost'' Monthly Implementation Assessment (MIA) is a 
repeated cross-sectional survey that will be conducted using web-based 
surveys that are self-administered on personal computers or web enabled 
mobile devices to collect rapid data on ``The Real Cost'' stimuli. Data 
from up to 2,000 teens in the United States will be collected each 
month for up to 24 months. To be eligible, participants must be between 
the ages of 12-20 and have not taken the MIA survey within the past 3 
months. We will use an Ipsos Knowledge Panel to collect data on ``The 
Real Cost'' stimuli. This design offers flexibility to assess new 
stimuli messages, as they air across various digital platforms, examine 
their performance over time, as well as the ability to pivot and add 
new survey measures as necessary. Monthly data will also allow us to 
obtain timely information on stimuli awareness, perceived 
effectiveness, as well as on teen attention and processing of the 
stimuli.
    The purpose of FDA's ``The Real Cost'' is to evaluate the following 
key components about ``The Real Cost'' stimuli:
     Awareness of ``The Real Cost'' stimuli.
     Attention behaviors when seeing ``The Real Cost'' stimuli.
     Processing of ``The Real Cost'' stimuli, including:
    [cir] Engagement with the stimuli.
    [cir] Main message comprehension.
    [cir] Acceptance and/or rejection of the stimuli.
     Perceived effectiveness of ``The Real Cost'' stimuli.
     Potential unintended consequences of viewing ``The Real 
Cost'' stimuli.
    In addition to the above components, the survey will ask 
participants to report on tobacco use and other psychographic and 
demographic items. The time frame that the survey items will ask about 
for stimuli awareness (i.e., past 30 days or past week) will depend on 
several factors, including how long the stimuli was on air. The survey 
will take an average of approximately 25 minutes to complete per 
participant. As the survey items are tested, any irrelevant items will 
be cut as necessary. Stimuli creative for both vaping and cigarette 
products will be assessed; therefore, two similar surveys (one on ENDS-
focused

[[Page 15876]]

stimuli and one on cigarette-focused stimuli) will be fielded as 
appropriate, but not within the same month.
    In support of the provisions of the Tobacco Control Act that 
require FDA to protect the public health and to reduce tobacco use by 
minors, FDA requests OMB approval to collect information to evaluate 
CTP's public education campaign ``The Real Cost'' through the MIA.
    In the Federal Register of April 27, 2023 (88 FR 25660), FDA 
published a 60-day notice requesting public comment on the proposed 
collection of information. No comments were received.
    FDA estimates the burden of this collection of information as 
follows:

                                                     Table 1--Estimated Annual Reporting Burden \1\
--------------------------------------------------------------------------------------------------------------------------------------------------------
                                                                  Number of
         Type of respondent/activity              Number of     responses per       Total             Average burden per response           Total hours
                                                 respondents     respondent       responses
--------------------------------------------------------------------------------------------------------------------------------------------------------
Parent Screener..............................       2,338,560               1       2,338,560  0.05 (3 minutes).........................         116,928
Parent Permission............................       1,753,920               1       1,753,920  0.05 (3 minutes).........................          87,696
Invitation Emails (Respondents ages 18-20)...          54,096               1          54,096  0.02 (1 minute)..........................           1,082
Youth Assent.................................          27,936               1          27,936  0.05 (3 minutes).........................           1,397
Young Adult Consent..........................          20,064               1          20,064  0.05 (3 minutes).........................           1,003
Online Survey................................          48,000               1          48,000  0.42 (25 minutes)........................          20,160
Reminder Emails..............................          48,000               1          48,000  0.20 (12 minutes)........................           9,600
                                              ----------------------------------------------------------------------------------------------------------
    Total....................................  ..............  ..............  ..............  .........................................         237,866
--------------------------------------------------------------------------------------------------------------------------------------------------------
\1\ There are no capital costs or operating and maintenance costs associated with this collection of information.

    Data collection for the MIA will consist of administering a monthly 
survey to participants ages 12-20 over the course of 2 years (24 
months). We expect the screening process (3 minutes per response) to 
yield an approximate 2.3 to 1 ratio of eligible participants. We will 
need to screen approximately 97,440 potential parents each month 
(resulting in 2,338,560 screeners) over the study period. Since the 
eligible age for data collection is 12 to 20 years old, we intend to 
screen parents of eligible youth and young adults. Parents of the youth 
participants determined to be eligible through the screener will 
provide parent permission (3 minutes per response). We estimate that 
1,753,920 of the parents who complete the screener will provide their 
permission for their youth to complete the online survey (approximately 
75 percent of the 2,338,560 screened). In addition to recruiting 
respondents through parents, we will send direct invitations to young 
adult panel members (18 to 20 years old). We anticipate that 50 percent 
of young adults will agree to participate. We will send 508 direct 
invitations a month to young adult panel members (18 to 20 years old). 
Eligible youth (1,753,920) will provide their assent (3 minutes per 
response) to participate in the online survey (25 minutes per 
response). Participants who are 18 to 20 years old (19 to 20 years old 
in Alabama and Nebraska in accordance with state law) will provide 
their consent (3 minutes per response) to participate in the online 
survey. We estimate that approximately 42 percent of the 48,000 
completed surveys will come from young adults aged 18 to 20 (aged 19 to 
20 in Alabama and Nebraska).
    Over the course of the study period, we intend to survey 
approximately 2,000 teens ages 12-20 per month for 24 months. From 
completed screeners, we estimate that we will obtain data from 
approximately 27,936 youth and 20,064 young adults. This will give us a 
total of 48,000 participants for the study. The survey will be repeated 
with a new cross-sectional sample approximately every month over a 
period of 24 months; however, some participants will complete more than 
one wave. These 48,000 respondents will receive an invitation email 
with a link to take the survey (4 minutes), 6 reminder emails (3 
minutes each), and a thank you email (3 minutes) upon completion of the 
study for a total of 25 minutes for respondents to read and respond to 
the emails.
    Several changes have been made to this information collection 
request since the 60-day notice was published in the Federal Register. 
These changes include (a) editing to clarify that the ad campaign is 
intended for ``teens'' not just ``youth;'' (b) removing the focus on 
video ads since the campaign may use other forms of communication to 
deliver its message and replacing the term ``ad'' with ``stimuli;'' (c) 
removing the youth screener from the burden table because parents 
determine the eligibility of their youth aged 12-17 (18-20 in Alabama 
and Nebraska in accordance with state law); (d) removing the young 
adult screener from the burden table, which will not be needed because 
young adult panel members (18-20 years old) will only receive an email 
invitation to complete the survey; (e) updating the burden table to 
reflect that we will send direct invitations to young adult panel 
members (18-20 years old); (f) updating the permission, assent, and 
consents because of updated information on the expected sample 
breakdown from the sample vendor for the distribution of the sample who 
are 12-17 years old and 18-20 years old; and (g) removing the thank you 
email since that will not be a part of the data collection procedures. 
In addition to the implementation evaluation described above, we will 
also assess perceptions to proposed stimuli and potential unintended 
consequences in order to inform the development of future messaging.

    Dated: February 28, 2024.
Lauren K. Roth,
Associate Commissioner for Policy.
[FR Doc. 2024-04526 Filed 3-4-24; 8:45 am]
BILLING CODE 4164-01-P