SS-Part A HMTS

Health Message Testing System (HMTS)

OMB No. 0920-0572

Expiration Date: August 31, 2021

Part A

Project Manager: Dawn B. Griffin

Health Communication Specialist

Office of the Associate Director for Communication (OADC)

Phone: (404) 498-0129

Fax: (404) 498-2221

Email address: [email protected]

Centers for Disease Control and Prevention

2500 Century Blvd.

Atlanta GA 30345

May 10, 2021

Table of Contents

Section

Page

A. Justification 3

A.1. Circumstances Making the Collection of Information Necessary 3

A.2. Purposes and Use of Information Collection 5

A.3. Use of Improved Information Technology and Burden Reduction 5

A.4. Efforts to Identify Duplication and Use of Similar Information 7

A.5. Impact on Small Businesses or Other Small Entities 7

A.6. Consequences of Collecting the Information Less Frequently 8

A.7. Special Circumstances 8

A.8. Comments in Response to the FR Notice and Consultation 8

A.8.a. Federal Register Notice 8

A.8.b Outside Consultation 8

A.9. Explanation of Any Payment or Gift to Respondents 9

A.10. Assurance of Confidentiality Provided to Respondents 11

A.10.1 Privacy Impact Assessment Information……………………………. 11

A.11. Justification for Sensitive Questions 13

A.12. Estimates of Annualized Burden Hours and Cost 14

A.13. Estimates of Annualized Respondent Capital and

Maintenance Costs 15

A.14. Estimates of Annualized Cost to the Federal Government 15

A.15. Explanation for Program Changes or Adjustments 15

A.16. Plans for Tabulation and Publication and Project Time Schedule 15

A.17. Reason(s) Display of OMB Expiration Date is Inappropriate 16

A.18. Exceptions to the Certification Statement 16

List of Tables

Table A12A: Estimated Annualized Burden Hours 14

Table A12B: Estimated Annualized Burden Costs 14

List of Attachments 17

The goal of the Health Message Testing System (HMTS) is to enable CDC programs to test health messages in a timely manner for clarity, salience, appeal and persuasiveness to target audiences.

Message testing questions will focus on issues such as comprehension, impressions, personal relevance, content and wording, efficacy of response, channels, and spokesperson/sponsor. Such information will enable message developers to enhance the effectiveness of messages for intended audiences.

Data collection methods proposed for HMTS includes intercept interviews, telephone interviews, focus groups, online surveys, and cognitive interviews. In almost all instances, data will be collected by outside organizations under contract with CDC.

In most cases, the results of tests of messages and materials will not be published; instead, the information will be used to inform health promotion activities across CDC.

A. Justification

A.1. Circumstances Making the Collection of Information Necessary

Background

This is an extension of a previously approved data collection, Health Message Testing System (HMTS), 0920-0572, which will expire on August 31,2021. CDC is resubmitting its generic clearance of the HMTS and is requesting approval for an additional three years. Upon re-approval, CDC program staff would submit requests to OMB for review and approval. Each submission would consist of an approved template containing relevant information about the planned data collection and copies of the data collection instruments.

Before CDC disseminates a health message to the public, the message always undergoes scientific review. However, reflecting the current state of scientific knowledge accurately provides no guarantee that the public will understand a health message or that the message will move people to take recommended action.¹ Communication theorists and researchers agree for health messages to be as clear and influential as possible, target audience members or representatives must be involved in developing the messages,² and provisional versions of the messages must be tested with members of the target audience.^{3, 4}

Increasingly there are circumstances when CDC must move swiftly to protect life, prevent disease, or calm public anxiety. Health message testing is even more important in these instances, because of the critical nature of the information need.

In the interest of timely health message dissemination, many programs forgo the important step of testing messages on dimensions such as clarity, salience, appeal, and persuasiveness (i.e., the ability to influence behavioral intention). Skipping this step avoids the delay involved in the standard OMB review process, but at a high potential cost. Untested messages can waste communication resources and opportunities because the messages can be perceived as unclear or irrelevant.⁵ Untested messages can also have unintended consequences, such as jeopardizing the credibility of Federal health officials.⁶

It is very important to extend the HMTS package to:

• Encourage health communication program directors to test the clarity and effectiveness of urgent health messages.

• Maintain the credibility of the nation’s public health communication and bioterrorism preparedness efforts.

For many years CDC programs have used HMTS to test and refine message concepts and test draft materials for clarity, salience, appeal, and persuasiveness to target audiences. Having this generic clearance available has enabled them to test their information and get critical health information out to the public quickly. Over the last three years, more than 32 messages have been tested using this clearance. For example: CDC Older Adult Injury Prevention Creative Campaign – Survey. This health communication campaign aimed to support and expand upon CDC's older adult injury prevention efforts and to raise awareness among older adults and their caregivers about preventable injuries that disproportionately impact them, steps to reduce their risk of injuries, and increase education about risk factors. Information collected can assist in the most effective use of CDC communication resources and opportunities by assessing clarity, appeal, persuasiveness and effectiveness of campaign material and advertisements (e.g. poster or video advertisement).

The Division of Tuberculosis Elimination (DTBE) obtained OMB approval through HMTS for Health Communications Testing for Latent Tuberculosis Infections Campaign for CDC’s National Center for HIV/AIDS, Viral Hepatitis, STD, and TB Prevention. This formative information collection will be sued to inform NCHHSTP DTBE’s future public service campaign efforts targeted to consumers at high-risk for LTBI and the providers who serve them. This information collection activity is essential because it will provide CDC with effective messages for communicating about this disease and infection to motivate at-risk consumers to get preventive screening and, if infected, treatment, and to motivate healthcare providers to encourage testing and early detection.

The Division of Diabetes Translation obtained OMB approval through HMTS for Message Testing for Diabetes Self-Management Education and Support (DSMES) Marketing Support: Card Sort Activity. Findings from this message testing effort were used by the Centers for Disease Control and Prevention’s (CDC) Division of Diabetes Translation (DDT) to inform how best to communicate with key audiences about DSMES services. Specifically, information about which attributes of DSMES services are most important to each audience will be identified and will serve as the basis for messages developed to promote DSMES services. This work will help increase the likelihood that messages will resonate and be understood as intended.

Over 27,696 respondents were queried and over 6,100 burden hours used during this time period. Because the availability of this ICR has been so critical to programs in disseminating their materials and information to the public in a timely manner, OADC is requesting a three year approval of this information collection.

The CDC is authorized to conduct research with the public under the Public Health Service Act (41USC 241) Section 301 (see Attachment 1).

A.2. Purposes and Use of Information Collection

The Health Message Testing System (HMTS), a generic information collection, enables programs across CDC to collect the information they require in a timely manner to:

• Ensure quality and prevent waste in the dissemination of health information by CDC to the public.

• Refine message concepts and to test draft materials for clarity, salience, appeal, and persuasiveness to target audiences.

• Guide the action of health communication officials who are responding to health emergencies, Congressionally-mandated campaigns with short timeframes, media-generated public concern, time-limited communication opportunities, trends, and the need to refresh materials or dissemination strategies in an ongoing campaign.

Message testing questions will focus on issues such as comprehension, impressions, personal relevance, content and wording, efficacy of response, channels, and spokesperson/sponsor. Such information will enable message developers to enhance the effectiveness of messages for intended audiences.

Data collection methods proposed for HMTS includes intercept interviews, telephone interviews, focus groups, online surveys, and cognitive interviews. In almost all instances, data will be collected by outside organizations under contract with CDC.

Because every testing instrument will be based on specific health issues or topics, it is not possible to develop one instrument for use in all instances. However, the same kinds of questions are asked in most message testing. This package includes generic questions and formats that can used to develop health message testing data collection instruments. (See Attachments 3-10) These include a list of screening questions, comprised of demographic and introductory questions, along with other questions that can be used to create a mix of relevant questions for each proposed message testing data collection method. However, programs may request to use additional questions if needed.

A.3. Use of Improved Information Technology and Burden Reduction

Whenever possible, the HMTS will use advanced technology to collect and process data in order to reduce respondent burden and to make data processing and reporting maximally efficient. Particular emphasis will be placed on compliance with the Government Paperwork Elimination Act (GPEA), Public Law 105-277, title XVII. Telephone surveys will rely on Computer Assisted Telephone Interviewing (CATI) and other computer-assisted strategies, and some surveys will be conducted online. In all message testing, the number of questions will be held to the absolute minimum required for the intended use of the data.

CATI Telephone Interviews

When item response alternatives are “close ended,” CATI will be utilized to help phone interviewers move quickly and accurately through items and skip patterns. The CATI system will not accept entry of illegal response values, and thus avoids lost data and the need to contact the respondent again for clarification. This system also allows for both on-site and remote quality assurance monitoring of interviewers.

Brief screening questionnaires will precede message testing questions to allow quick ascertainment of an individual’s membership in a targeted audience segment; calls to ineligibles can be terminated immediately. In some cases, ESRI GSI and mapping software will be used to determine where individuals from audience segments of interest are most likely to live and work. Random digit dialing (RDD) in these geographic areas will limit calls to ineligible individuals.

Online Research (Individual Surveys and Focus Groups)

Online questionnaires and focus groups ease burden because they can be completed in the respondent’s home or workplace, at the respondent’s convenience. They are comparable to mailed questionnaires in these respects, and in that they do not require the presence of an interviewer. They are less burdensome than mailed questionnaires in that they eliminate the need to handle and return paper copies.⁷

Online surveys of members of organizations will be used when appropriate. For example, if it were necessary to test messages for pediatricians about immunization, CDC might partner with the American Academy of Pediatrics to recruit some of their members for an online survey.

In other instances, it is necessary to query widely dispersed members of the general public. For example, this might occur when there is a need to know whether specific messages reach intended audiences through specific channels during a time-limited, ongoing campaign. To fill such a knowledge gap, the preferred online methodology would be the Web-enabled panel approach described below.

The Web-enabled panel approach is an advanced survey technique that uses online technology to collect data from households that participate in an ongoing panel. The panels are very large, allowing selection from the overall pool, the construction of large and diverse samples, and the rapid identification of several potential respondents from extremely small subgroups of the population.

Although smaller samples generally suffice for the purposes of message testing, assembling these groups can be extremely time-consuming. The Web-enabled or online panel approach also allows for the immediate turnaround of transcripts from on-line focus groups and data from on-line surveys.

Relative to less technically advanced methods, this data collection approach has several advantages: speed, cost, access and reduced burden.

Speed: Digital data collection supports quick data turnaround. About half of the completed questionnaires from a Web-enabled panel are received within the first 3-4 days of the assignment, a more rapid response than that of mailed surveys. Email reminders and phone calls have been found to increase online survey response rates. Additionally, the automated data collection allows results to be delivered to CDC quickly – in days or weeks, rather than months.

Cost: Surveys that access an already existing panel save money compared with one-time survey costs.

• Costs of hardware and recruitment can be amortized over the life of the panel. This permits use of the expensive sampling techniques that help to achieve higher response rates without having to pass all the recruitment costs on to the client.

• The expense of collecting profile information on panel members is incurred just once, and then the profile data can be used in conjunction with data collected later. In cross-sectional designs, by contrast, demographic data must be collected with every survey.

• In traditional longitudinal cohort surveys, re-contact costs are much higher. Email is inexpensive and repeated call attempts are not needed because email does not require the respondent to be at home when a re-contact is attempted.

Access: With tens of thousands of members and minority group over-sampling, online panels have the diversity to enable the construction of small “tailor-made” samples, such as one composed of Native American mothers of 9- and 10-year-old girls. Response rates are higher than in mail or phone surveys because respondents have agreed to participate in a series of surveys over the period of their panel membership and they receive a package of incentives, so panel survey data tend to be more representative of individuals who are older and less educated.⁸

Reduced Burden: Like other online surveys, Web-enabled panel surveys are self-administered, allowing respondents to complete the survey at their convenience, in the comfort and privacy of their homes. Furthermore, from the respondent’s point of view, the inclusion of video, audio, and 3-D graphics in the questionnaire make the survey experience much more engaging and less burdensome than conventional telephone interviews.

Online Studies

The panel status of online survey respondents presents a special circumstance of data collection. The frequency of survey receipt, the length of panel membership, and the incentive package received in exchange for member participation varies by commercial survey outlet, but all respondents to these kinds of surveys are frequent participants in interviews. However, steps are taken through management of the panels to prevent overburdening respondents.

Most surveys take about 30-60 minutes to complete. Surveys longer than 15 minutes are often broken into segments and administered incrementally. As mentioned above, surveys are completed at the leisure of respondents and thereby minimizes burden on respondents. Finally, most companies do not permit the selection of a respondent for more than one survey on the same topic in any three-month period.

A.4. Efforts to Identify Duplication and Use of Similar Information

Health messages developed by CDC are unique in their mix of intended audience, health behavior, concept, and execution. Therefore, in the majority of cases, there are no similar data available. CDC reviews existing published literature and unpublished qualitative pretesting reports when they are available, and also consults with outside experts to identify information that could facilitate message development prior to conducting any data collection.

In the past, CDC joined other U.S. government agencies in networks of organizations that sponsor or endorse health communication projects, such as the Children’s Environmental Health Subcommittee on Health Communication, the NIH/ CDC collaboration on diabetes, the NCI/CDC collaboration on DES, and more recently, the HHS Health Literacy Work Group. These affiliations serve as information channels and help prevent redundancy.

A.5. Impact on Small Businesses or Other Small Entities

Physicians, other health care providers, and small businesses or non-profit organizations can be important intermediaries or target audiences for health messages. When testing messages for these audiences is required, CDC works through established medical and professional societies to gain access to the audience and to obtain feedback on our instruments and data collection plans. As a result, no single “convenience sample” of small entities is overburdened and burden will be kept to a minimum.

This section has limited applicability to general population surveys that would be conducted through the online panel. Current panel members are individuals from the general population, and data collection via the online panel involve no burden to small businesses or entities. In fact, the quick data turnaround potential of the online panel should make message testing with a general audience more feasible, thereby reducing burden on physicians by eliminating the need for expert opinions about how general audience members would respond.

A.6. Consequences of Collecting the Information Less Frequently

Health message testing might take place at more than one point in a campaign. If the message is not tested, time and money may be wasted developing materials that cannot achieve the health communication objective. Subsequently, if draft materials and messages are not tested, poor execution can undercut a good concept. Each activity will be treated as a separate, one-time study with different respondents. Rarely will information be collected more than once from any given respondent.

There are no legal obstacles to reduce the burden.

A.7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

There are no special circumstances. The message testing activities fully comply with the regulations and guidelines in 5 CFR 1320.5.

A.8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

A.8.a. Federal Register Notice

A 60-day Federal Register Notice was published in the Federal Register on Wednesday, May 26, 2021 Vol. 86, No. 100, Page(s) 28354 - 28355. One non-substantive comment received.

A.8.b Outside Consultation

To ensure there is no duplication or redundancy of effort across projects and programs, program staff will consult with a variety of sources on the availability of data, frequency of collection, clarity of instructions, and record keeping, disclosure, and reporting format (if any), and on the data elements to be recorded, disclosed, or reported. CDC staff will consult with relevant Federal agencies and national associations that conduct health communication campaigns (e.g., American Cancer Society, American Heart Association, the March of Dimes, etc.).

In addition, the following contractors are available for health communication consultation through the OADC Communication Services Contract Mechanism, if needed:

• AED, AIR, Battelle, Danya, ESI, Hager Sharp, HMA, Ketchum, NOVA, Ogilvy PR, ORAU, ORC Macro, PRR, RTI, SRA International,Weber Shandwick, and Westat.

CDC Center and office health communication specialists throughout the agency will conduct, direct, and use these data collections. The Associate Directors of Communication Science (ADCS) in National Centers at CDC are the senior communication specialists in their respective Centers and will consult with programs on their data collections.

Center/Office	ADCS	E-Mail/Work#2
Center for Global Health (CGH)	Nili Larish	[email protected] 404-639-6080
Center for Preparedness and Response (CPR)	John Anderton	[email protected] 404-498-6145
Center for State, Tribal, Local, and Territorial Support (CSTLTS)	Tracy Ingraham	[email protected] 404-718-8161
Center for Surveillance, Epidemiology and Laboratory Services (CSELS)	Angela Fisher	[email protected] 404-639-1665
Deputy Director for Infectious Diseases/OD (DDID/OD)	Hilary Eiring	[email protected] 770-488-3901
Deputy Director for Non-Infectious Diseases/OD (DDNID/OD)	Ellen Wan	[email protected] 404-639-2231
Deputy Director for Public Health Service and Implementation Science/OD (DDPHSIS/OD)	Rebecca Greco Koné	[email protected] 404-639-3224
Director for Public Health Scientific Services (DPHSS/OD)	Dana Pitts	[email protected] 404-639-1151
National Center on Birth Defects and Developmental Disabilities (NCBDDD)	Betsy Mitchell	[email protected] 404-498-0251
National Center for Chronic Disease Prevention and Health Promotion (NCCDPHP)	Gabbi Promoff	[email protected] 770-488-6491
National Center for Environmental Health (NCEH)/Agency for Toxic Substances and Disease Registry (ATSDR)	Heather Bair - Blake	[email protected] 404-639-3323
National Center for Emerging and Zoonotic Infectious Diseases (NCEZID)	John O’Connor	[email protected] 404639-2769
National Center for HIV/AIDS, Viral Hepatitis, STD, and TB Prevention (NCHHSTP)	Rachel Powell	[email protected] 404-639-8435
National Center for Health Statistics (NCHS)	Dagny Olivares	[email protected] 404-639-3180
National Center for Injury Prevention and Control (NCIPC)	Kelly Holton	[email protected] 404-639-2112
National Center for Immunization and Respiratory Diseases (NCIRD)	Allison Fisher	[email protected] 404-639-8696
National Institute for Occupational Safety and Health (NIOSH)	Christina Spring	[email protected] 202-245-0633
Office of the Associate Director for Communication Science (OADC)	Dogan Eroglu	[email protected] 404-498-6119
Office of the Associate Director for Policy and Strategy (OADPS)	Laurie Ishak	[email protected] 202-245-0748
Office of the Chief Operating Officer	John Patterson	[email protected] 404-639-0639
Office of Laboratory Science and Safety (OLSS)	Elizabeth Greene	[email protected] 404-639-3459
Office of Minority Health and Health Equity (OMHHE)	Laura Ross	[email protected] 770-488-6073
Office of Science (OS)	Rosa Herrera	[email protected] 404-639-3986

A.9. Explanations of Any Payment or Gift to Respondents

To defray the costs of participation (e.g., transportation) and to boost response rates, participants in face-to-face interviews, traditional focus groups, central location interviews, and online surveys may receive, if warranted and justified, $0-$75 or the equivalent, depending on the time required. The range of monetary reward is consistent with current rates for participation in formative research studies. Incentives will take the form of cash or gift certificates. In most cases, there will be no remuneration for participation in phone interviews. As described at the end of this section, online panel surveys use an incentive program

to improve response rates and maintain membership. Study-for-study, incentives are in line with the others that will be offered under the HMTS. However, the default for the government is not to offer incentives for interviews and surveys.

Reviewed literature revealed the payment of incentives can provide significant advantages to the government in terms of direct cost savings and improved data quality. It also should be noted that message testing is a marketing technique, and it is standard practice among commercial market researchers to offer incentives as part of respondent recruitment.

Doryn Chervin, Ph.D., [(404) 321-3211] a spokesperson from Macro International, Inc. (ICF Macro), a firm experienced in health communication message testing and the various methods used in this research, explained, “Given busy schedules and a plethora of commercial marketing efforts that provide such incentives, social and behavioral science studies cannot compete for the respondents’ time unless an incentive is provided.”

Background on the Use of Response Incentives

Payments vs. Non-monetary Incentives

Cash incentives have been shown to be most effective in increasing survey response rates for one-time surveys of panel members. For example, Singer and her colleagues noted that, following a series of experiments on the impacts of incentives on various types of survey data collection, “...gifts in this study were less effective in increasing response rates than cash, even with the value of the incentive controlled.”

This finding replicates previous research on the effectiveness of incentives, including a meta-analysis of 38 experiments and quasi-experiments conducted by Church.⁹ Church reported that non-monetary gifts were significantly less effective than cash in generating survey response, and noted that offering prepaid monetary incentives yielded an average increase of 19.1 percentage points over comparison groups. Moreover, the impacts of monetary incentives seem greater than the impacts of promised charitable donations, lotteries for cash prizes, and other non-monetary rewards.

Level of Incentive Payment

Despite its apparent logic, simply increasing the size of cash incentives to non-respondents does not always result in proportional increases in response rates. In fact, there is some evidence of diminishing returns as incentive levels increase. However, Findlay and Schaible¹⁰ found that increasing the incentive payments from $10 to $20 was successful in increasing overall response rates. This incentive was often supported in the literature. Meta-analyses conducted by Church noted incentives provided with initial mailings (i.e., not conditionally linked to the completion of the survey) were the most effective in encouraging increased response.

Reduced Data Collection Cost

Discussion of incentives as a technique to speed responses and expand response rates is not complete without mentioning the trade-off between the costs of incentives and the costs of reminders and other efforts to foster timely and complete participation^11. The goal is to find the highest response rate at the lowest overall cost to the government. In the National Adult Literacy Survey by Berlin and colleagues,¹² a $20 incentive resulted in not only higher response rates from the sample cohort, but also lower costs per completed case than the comparison group. Importantly, the incentives provided higher response rates from adults with lower-than-average levels of education and basic literacy and numeracy skills.

Reduced Bias

The most important aspect of an incentive plan may be its potential for reducing response bias, underreporting bias, and similar sources of error. Findings from the National Survey of Family Growth (a study in which highly sensitive and personal information is collected from young adults) demonstrated that incentives not only had positive effects on response rates, but they also increased the accuracy of reporting. Incentives are necessary for message testing in order to ensure that those who are willing to participate reflect the target audiences. Failure to provide a basic incentive is likely to bias samples in the direction of well-educated individuals who are generally predisposed to be helpful.

A.10. Assurance of Confidentiality Provided to Respondents

The CDC Privacy Act Officer has determined that the Privacy Act does not apply to data collections conducted according to the procedures described in this application. Although personal information (e.g., gender, age, and race) will be gathered in message testing activities, no personal identifiers (e.g., full name, address or phone number, social security number, etc.) will be collected or maintained. Surveys done through online panels will use already-established records systems.

A.10.1 Privacy Impact Assessment Information

Although personal information (e.g., gender, age, and race) will be gathered in message testing activities, no personal identifiers (e.g., full name, address or phone number, social security number, etc.) will be collected or maintained. Surveys done through online panels will use already-established records systems.

All data provided by respondents will be treated in a secure manner and will not be disclosed, unless otherwise compelled by law. Respondents will be informed prior to participation that their responses will be treated in a secure manner.

Overview of the Data Collection System

An overview of the data collection system and methods of collection may vary with each GenIC. The approved template includes a section to identify the audience and methods for each data collection. (Attachment 3)

Many of the data collections in the HMTS will be conducted through either the Creative Domain or the Evaluation Domain of the OADC Communication Services Contract Mechanism managed by the Strategic & Proactive Communication Branch, Division of Communication Service, Office of the Associate Director for Communication.

Creative contractors include AIR, Danya, ESI, Hager Sharp, HMA, Ketchum, NOVA, Ogilvy PR, ORAU, Macro, PRR, RTI, SRI International, Weber Shandwick, Westat and the Academy for Educational Development (AED). Evaluation contractors include AED, AIR, Battelle, Danya, NOVA, ORC Macro International, Inc., RTI, and Westat. Work may be conducted through other contractors carrying out campaign activities but, in all cases, the Strategic & Proactive Communication Branch, Division of Communication Service will require data management procedures be followed.

Also, this project is exempt from IRB requirements. Formative research on health messages (ranging in topics from terrorism to colorectal cancer) is conducted to identify the clarity, salience, and persuasiveness of the message under review. The information gathered in this process is then utilized to target messages appropriately so in the unfortunate event of an urgent health threat, for example, messages then can be disseminated to the public in a clear and sensitive manner. Time-sensitive data collection activities permitted under HMTS, OMB control number 0920-0572, are not research and, therefore, do not require IRB review.

Central Location Intercept Interviews

Respondents will be advised of the nature of the activity, the length of time it will require, and that participation is purely voluntary. Respondents will be assured that no penalties will occur if they wish not to respond to the information collection as a whole or to any specific questions. These procedures conform to ethical practices for collecting data from human participants.

However, because this activity is considered formative evaluation, the procedures normally do not meet the criteria for CDC IRB review (see Attachment 11 for the judgment from OADC Associate Director for Communication Science).

If data are collected by means of paper questionnaires, the questionnaires will be kept in locked filing cabinets in the offices of project staff employed by CDC contractors. When the data have been coded into electronic files and cleaned, the paper records will be destroyed. Electronic files (whether generated

by touch-screen technology, email, or by coding paper records) will be handled as described in the section

on phone interviews below. In reports, all presentation of data will be in aggregate form, and no links to individuals will be preserved. Reports will not include identifiable information on respondents.

Telephone Interviews

Phone interviews will employ a CATI system; no names or other information that could identify the respondent will be recorded. A code number will be assigned to an individual’s responses. It will not be possible to link these code numbers to respondents’ phone numbers.

All selected participants will be informed at the beginning of the phone interview that their responses will be treated in a secure manner, that all data will be safeguarded closely, and that no individual identifiers will be used in study reports. Interviewers will be extensively trained to impart this information.

All data will be stored in secured electronic files at a contractor’s office and will be accessible only to staff directly involved in the project. All members of the project will be required to sign a statement developed by the contractor pledging their personal commitment to guard the confidentiality of data. Data files will be retained by the contractor for a period of no more than three years and then destroyed.

Online data collections will conform totally to federal regulations [the Hawkins-Stafford Amendments of 1988 (P.L. 100-297) and the Computer Security Act of 1987] and will be required to have comprehensive, written plans to maintain confidentiality. This plan will include having all personnel who will have access to individual identifiers sign confidentiality agreements. They will also be trained in the meaning of confidentiality, particularly as it relates to handling requests for information from respondents, and in providing assurance to respondents about the protection of their responses.

Surveys sent electronically from the Web site will be sent to an email address solely dedicated for the research project. The electronic surveys will be received, a record of the receipt will be made, and the survey will be separated from any identifying information, including the email address of the sender. These surveys will be forwarded to other staff for data analysis.

A.11. Institutional Review Board (IRB) and Justification for Sensitive Questions

IRB Approval

The activities undertaken under HMTS are solely for the purpose of tailoring communication messages in a timely manner, and sometimes in emergency situations, they are non-research. Therefore, they do not require IRB review.

Sensitive Question

The majority of questions asked will not be of a sensitive nature. There will be no request for a respondent’s Social Security Number (SSN).

It will, at times, be necessary to ask questions considered to be of a sensitive nature in order to test messages about health-relevant behavior. Questions about messages concerning lifestyle (e.g., messages about dietary habits, smoking, drug use, and intimate partner issues), and questions about messages related to illnesses such as cancer or HIV could be considered sensitive. To avoid fear of disclosure of sensitive information, respondents will be told that all data provided by respondents will be treated in a secure manner and will not be disclosed, unless otherwise compelled by law.

To avoid negative reactions to these questions, several steps will be taken:

• Respondents will be informed that they need not answer any question that makes them feel uncomfortable or that they simply do not wish to answer.

• Where possible, use of touch-screen methodology or other self-directed techniques will provide privacy; not having to verbalize a response may increase comfort.

• When such numbers are available and appropriate, participants will be provided with specific agency hotline numbers to call in case they have a question or concern about the sensitive issue.

• Interviewers will be trained to ask questions in a sensitive manner and to handle any subsequent discussion skillfully. Where appropriate, interviewers and respondents will be matched for gender and other demographic criteria (e.g., age, preferred language use).

• Questions included in these interviews will be pilot-tested with 9 individuals matching the characteristics of the target audience.

All online panel surveys are self-administered and allow respondents to complete the surveys at their convenience, in the comfort and privacy of home. If a survey asks about sensitive topics that respondents may not want other household members to know about, the respondents can protect their own privacy by answering questions on the digital device when other household members are not at home or are sleeping. Additionally, respondents can toggle between the survey and another web page to further protect their privacy and change the view on the screen if a household member enters the room. Analyses of response timing have shown that respondents take advantage of the ability to respond to surveys at any hour.

A.12. Estimates Annualized Burden Hours and Cost

Response burden and cost for each type of message testing method are summarized in Table A12A and Table A12B.

Table A12A. Estimated Annualized Burden Hours

Data Collection Methods	No. of Respondents per Method	No. of Responses per Respondent	Average Burden Per Response (in hours)	Total Burden Hours
Central Location Intercept Interviews, Telephone Interviews, Individual In-depth Interview (Cognitive Interviews), Focus Group Screenings, Focus Groups, Online Surveys	18, 525	1	8/60	2, 470
Total	18, 525			2,470

The total estimated annualized hourly burden anticipated for all data collection methods would be approximately 2,470 hours. A total of 7,410 burden hours was approved under the existing HMTS package covering a three-year period from August 13, 2018 – August 31, 2021. Based on our projections, we request 7,410 burden hours for this revised HMTS package covering a three-year period.

Each National Center or National Institute (see page 8) specializes in a particular health area (e.g., chronic disease prevention, environmental health). It is assumed that 10 of the 21 would initiate two or three message-testing activities each over the course of a year.

Table A12B. Estimated Annualized Burden Costs

Data Collection Methods	No. of Respondents per Method	No. Responses per Respondent	Average Burden per Response	Total Burden Hours	Hourly Wage Rage	Total Respondents Costs
Central Location Intercept Interviews, Telephone Interviews, Individual In-depth Interview (Cognitive Interviews) Focus Group Screenings, Focus Groups, Online Surveys	18, 525	1	8/60	2, 470	$22.33	$55,155.10
Total	18, 525					$55,155.10

Because the time required for responding to a survey or interview, and to participate in a focus group has a monetary value, this table estimates the total annual cost to the respondents for all activities.

According to the U.S. Department of Labor (DOL) May 2013 National Occupational Employment and Wage Estimates the average hourly wage is $22.33. Because of the scope of this generic clearance and the variety of the types of participants, this average salary was utilized rather than attempting to estimate salaries for groups of audiences. The total annualized burden cost is estimated at $55,155.10 per year.

A.13. Estimates of Annualized Respondent Capital and Maintenance Costs

There are no capital/start-up or ongoing operation/maintenance costs associated with this information collection.

A.14. Estimates of Annualized Cost to the Federal Government

The average estimated annual cost to the Federal government for conducting the 28 message testing activities proposed in Table A12B is $1,550,000. This total cost includes approximately $1,425,000 for contractual costs (e.g., test design, data collection, analysis, and reporting), and $125,000 for personnel costs for Federal employees involved in project oversight activities (20% time for a HMTS project officer and .80 FTE for data collection project staff).

The Office of the Associate Director for Communication manages a multi-vendor contract appropriate for message testing; the $1,425,000 figure reflects typical costs for such activities under the contract. Any vendor covered by the contract might conduct the actual work.

A.15. Explanation for Program Changes or Adjustments

This is a request for approval of a currently approved GENERIC data collection. Each GenIC will be submitted using a template approved by OMB during a conference call on June 8, 2011. This template, “Health Message Testing System Expedited Review Form” (Attachment 3), will contain relevant information about the proposed information collection and will be submitted along with copies of the data collection instruments and other pertinent information.

A.16. Plans for Tabulation and Publication and Project Time Schedule

In most cases, the results of tests of messages and materials will not be published; instead, the information will be used to inform health promotion activities across CDC. Project timelines will vary, depending on the program requirements and the program itself. Message testing ordinarily requires at least one-two weeks to organize, and at least one-two weeks to implement.

A.17. Reason(s) Display of OMB Expiration Date is Inappropriate

All data collection instruments will display the expiration date for OMB approval of the information collection.

A.18. Exceptions to the Certification Statement

No exceptions to the certification statement are being sought.

List of Attachments

Attachment 1: Authorizing Legislation

Attachment 2: Federal Register Notice

Attachment 3: Health Message Testing System Expedited Review Form

Attachment 4: Demographic Questions

Attachment 5: Market Research Experience Questions

Attachment 6: Introductory Questions

Attachment 7: Core Questions

Attachment 8: Activity/Task Questions

Attachment 9: Follow-Up Questions

Attachment 10: Examples:

Example 1: Central Location Intercept Interview

Example 2: Telephone Interview

Example 3: Individual In-depth Interview (Cognitive Interview)

Example 4: Screener Questions for Focus Group Testing

Example 5: Focus Group Moderator’s Guide

Example 6: Online Interview

Attachment 11: IRB Ruling

1 Pechman, C. (2001) A comparison of health communication models: Risk learning versus stereotype priming. Media Psychology, 3(2): 189-210.

2 Delong, D.W., & Fahey, L. (2000) Diagnosing cultural barriers to knowledge management. The Academy of Management Executive, 14(4): 113-127.

3 Andreasen, A. 1995 Marketing Social Change. San Francisco, Jossey-Bass.

4 Black, D.R., Blue, C.L., & Coster, D.C. (2001) Using social marketing to develop and test tailored messages. American Journal of Health Behavior, 25(3): 260-271.

5 Wallendorf, M. (2001) Literally literacy. The Journal of Consumer Research, 27(4): 505-511.

6 Harris-Kojetin, D., McCormack, L.A., Jael, L.A., Sangl, E.F., & Garfinkel, S.A. (2001) Creating more effective health plan quality reports for consumers: Lessons from a synthesis of quality testing. Health Services Research, 36(3): 447-476.

7 Stempel, G.H., Stewart, R.K. (2000) The internet provides both opportunities and challenges for mass communication researchers. Journalism and Mass Communication Quarterly, 77(3):541-548.

8 Bourque, L.B. & Fielder, E.P. (1995) How to Conduct Self-Administered and Mail Surveys. Sage: Thousand Oaks, CA.

9 Church, A.H. (1993). Estimating the Effect of Incentives on Mail Survey Response Rates: A Meta‑Analysis. Public Opinion Quarterly, 57, 62‑79.

10 Findlay, J.S., & Shaible, W. L. (1980). A Study of the Effect of Increased Remuneration on Response in a Health and Nutrition Examination Survey. In the American Statistical Association (ed.), Proceedings of the American Statistical Association Section on Survey Research Method. (pp. 590-594). Washington, D.C.: American Statistical Association.

11 Kulka (1994).

12 Berlin, M., Mohadjer, L., Waksberg, J., Kolstad, A., Kirsch, I., Rock, D., & Yamamoto, K. (1992). An experiment in monetary incentives. In the American Statistical Association (ed.), Proceedings of the American Statistical Association Section on Survey Research Methods (pp. 393-398). Alexandria, VA: American Statistical Association.