Health Message Testing System (HMTS)
OMB No. 0920-0572
Part A
Project Manager: William E. Pollard, Ph.D.
Health Communications Specialist
Office of the Associate Director for Communication
Phone: (404) 639-2881
Fax: (404) 639-0497
Email address: [email protected]
Centers for Disease Control and Prevention
1600 Clifton Road, Mailstop G-21
Atlanta GA 30333
September, 2011
Table of Contents
Section
Page
A. Justification 3
A.1. Circumstances Making the Collection of Information Necessary 3
A.2. Purposes and Use of Information Collection 4
A.3. Use of Improved Information Technology and Burden Reduction 5
A.4. Efforts to Identify Duplication and Use of Similar Information 7
A.5. Impact on Small Businesses or Other Small Entities 7
A.6. Consequences of Collecting the Information Less Frequently 7
A.7. Special Circumstances 7
A.8. Comments in Response to the FR Notice and Consultation 8
A.8.a. Federal Register Notice 8
A.8.b Outside Consultation 8
A.9. Explanation of Any Payment or Gift to Respondents 9
A.10. Assurance of Confidentiality Provided to Respondents 11
A.11. Justification for Sensitive Questions 12
A.12. Estimates of Annualized Burden Hours and Cost 13
A.13. Estimates of Annualized Respondent Capital and
Maintenance Costs 14
A.14. Estimates of Annualized Cost to the Federal Government 14
A.15. Explanation for Program Changes or Adjustments 14
A.16. Plans for Tabulation and Publication and Project Time Schedule 15
A.17. Reason(s) Display of OMB Expiration Date is Inappropriate 15
A.18. Exceptions to the Certification Statement 15
List of Tables
Table A12A: Estimated Annualized Burden Hours 14
Table A12B: Estimated Annualized Burden Costs 15
List of Attachments 16
A. Justification
Background
This is a revision of a currently approved data collection, Health Message Testing System (HMTS), 0920-0572, which expires on November 30, 2011. CDC is resubmitting its generic clearance of the HMTS and is requesting approval for an additional three years. The current ICR has an approved question bank and information collection requests are evaluated at the program level. However, in a conference call on June 8, 2011, OMB staff indicated that after November 30, 2011, and contingent on its approval, all requests under the revised generic will be reviewed by OMB. Upon re-approval, CDC program staff will submit requests to OMB for review and approval. Each submission will consist of an approved template containing relevant information about the planned data collection and copies of the data collection instruments.
Before CDC disseminates a health message to the public, the message always undergoes scientific review. However, reflecting the current state of scientific knowledge accurately provides no guarantee that the public will understand a health message or that the message will move people to take recommended action.1 Communication theorists and researchers agree for health messages to be as clear and influential as possible, target audience members or representatives must be involved in developing the messages,2 and provisional versions of the messages must be tested with members of the target audience.3, 4
Increasingly there are circumstances when CDC must move swiftly to protect life, prevent disease, or calm public anxiety. Health message testing is even more important in these instances, because of the critical nature of the information need.
In the interest of timely health message dissemination, many programs forgo the important step of testing messages on dimensions such as clarity, salience, appeal, and persuasiveness (i.e., the ability to influence behavioral intention). Skipping this step avoids the delay involved in the standard OMB review process, but at a high potential cost. Untested messages can waste communication resources and opportunities because the messages can be perceived as unclear or irrelevant.5 Untested messages can also have unintended consequences, such as jeopardizing the credibility of Federal health officials.6
The CDC is authorized to conduct research with the public under the Public Health Service Act (41USC 241) Section 301 (see Attachment 1).
Privacy Impact Statement
The CDC Privacy Act Officer has determined that the Privacy Act does not apply to data collections conducted according to the procedures described in this application. Although personal information (e.g., gender, age, and race) will be gathered in message testing activities, no personal identifiers (e.g., full name, address or phone number, social security number, etc.) will be collected or maintained. Surveys done through online panels will use already-established records systems.
Overview of the Data Collection System
An overview of the data collection system and methods of collection may vary with each GenIC. The proposed template includes a section to identify the audience and methods for each data collection. (Attachment 3)
Items of Information to be Collected
No individually identifiable information is being collected.
Identification of Websites(s) and Website Content Directed at Children Under 13 Years of Age
The methods of collection may vary with each genIC. The proposed template includes a section to identify the audience and methods for each data collection. (Attachment 3)
A.2. Purposes and Use of Information Collection
The Office of the Associate Director for Communication (OADC) will have the capacity to track submissions under the proposed HMTS. This information will be used to:
Ensure quality and prevent waste in the dissemination of health information by CDC to the public.
Refine message concepts and to test draft materials for clarity, salience, appeal, and persuasiveness to target audiences.
Guide the action of health communication officials who are responding to health emergencies, Congressionally-mandated campaigns with short timeframes, media-generated public concern, time-limited communication opportunities, trends, and the need to refresh materials or dissemination strategies in an ongoing campaign.
It is very important to extend the HMTS package to:
Encourage health communication program directors to test the clarity and effectiveness of urgent health messages.
Maintain the credibility of the nation’s public health communication and bioterrorism preparedness efforts.
The results of message testing intended for a specific audience or context may inform future campaigns designed for similar situations.
Because every testing instrument will be based on specific health issues or topics, it is not possible to develop one instrument for use in all instances. However, the same kinds of questions are asked in most message testing. This package includes generic questions and formats that can used to develop health message testing data collection instruments. (See Attachments 3-10) These include a list of screening questions, comprised of demographic and introductory questions, along with other questions that can be used to create a mix of relevant questions for each proposed message testing data collection method. However, programs may request to use additional questions if needed.
Data collection methods proposed for HMTS includes intercept interviews, telephone interviews, focus groups, online surveys, and cognitive interviews. In almost all instances, data will be collected by outside organizations under contract with CDC.
A.3. Use of Improved Information Technology and Burden Reduction
Whenever possible, the HMTS will use advanced technology to collect and process data in order to reduce respondent burden and to make data processing and reporting maximally efficient. Particular emphasis will be placed on compliance with the Government Paperwork Elimination Act (GPEA), Public Law 105-277, title XVII. Telephone surveys will rely on Computer Assisted Telephone Interviewing (CATI) and other computer-assisted strategies, and some surveys will be conducted online. In all message testing, the number of questions will be held to the absolute minimum required for the intended use of the data.
CATI Telephone Interviews
When item response alternatives are “close ended,” CATI will be utilized to help phone interviewers move quickly and accurately through items and skip patterns. The CATI system will not accept entry of illegal response values, and thus avoids lost data and the need to contact the respondent again for clarification. This system also allows for both on-site and remote quality assurance monitoring of interviewers.
Brief screening questionnaires will precede message testing questions to allow quick ascertainment of an individual’s membership in a targeted audience segment; calls to ineligibles can be terminated immediately. In some cases, ESRI GSI and mapping software will be used to determine where individuals from audience segments of interest are most likely to live and work. Random digit dialing (RDD) in these geographic areas will limit calls to ineligible individuals.
Online questionnaires and focus groups ease burden because they can be completed in the respondent’s home or workplace, at the respondent’s convenience. They are comparable to mailed questionnaires in these respects, and in that they do not require the presence of an interviewer. They are less burdensome than mailed questionnaires in that they eliminate the need to handle and return paper copies.7
Online surveys of members of organizations will be used when appropriate. For example, if it were necessary to test messages for pediatricians about immunization, CDC might partner with the American Academy of Pediatrics to recruit some of their members for an online survey.
In other instances, it is necessary to query widely dispersed members of the general public. For example, this might occur when there is a need to know whether specific messages reach intended audiences through specific channels during a time-limited, ongoing campaign. To fill such a knowledge gap, the preferred online methodology would be the Web-enabled panel approach described below.
The Web-enabled panel approach is an advanced survey technique that uses online technology to collect data from households that participate in an ongoing panel. The panels are very large, allowing selection from the overall pool, the construction of large and diverse samples, and the rapid identification of several potential respondents from extremely small subgroups of the population.
Although smaller samples generally suffice for the purposes of message testing, assembling these groups can be extremely time-consuming. The Web-enabled or online panel approach also allows for the immediate turnaround of transcripts from on-line focus groups and data from on-line surveys.
Relative to less technically advanced methods, this data collection approach has several advantages: speed, cost, access and reduced burden.
Speed: Digital data collection supports quick data turnaround. About half of the completed questionnaires from a Web-enabled panel are received within the first 3-4 days of the assignment, a more rapid response than that of mailed surveys. Email reminders and phone calls have been found to increase online survey response rates. Additionally, the automated data collection allows results to be delivered to CDC quickly – in days or weeks, rather than months.
Cost: Surveys that access an already existing panel save money compared with one-time survey costs.
Costs of hardware and recruitment can be amortized over the life of the panel. This permits use of the expensive sampling techniques that help to achieve higher response rates without having to pass all the recruitment costs on to the client.
The expense of collecting profile information on panel members is incurred just once, and then the profile data can be used in conjunction with data collected later. In cross-sectional designs, by contrast, demographic data must be collected with every survey.
In traditional longitudinal cohort surveys, re-contact costs are much higher. Email is inexpensive and repeated call attempts are not needed because email does not require the respondent to be at home when a re-contact is attempted.
Access: With tens of thousands of members and minority group over-sampling, online panels have the diversity to enable the construction of small “tailor-made” samples, such as one composed of Native American mothers of 9- and 10-year-old girls. Response rates are higher than in mail or phone surveys because respondents have agreed to participate in a series of surveys over the period of their panel membership and they receive a package of incentives, so panel survey data tend to be more representative of individuals who are older and less educated.8
Reduced
Burden: Like
other online surveys, Web-enabled panel surveys are
self-administered, allowing respondents to complete the survey at
their convenience, in the comfort and privacy of their homes.
Furthermore, from the respondent’s point of view, the inclusion
of video, audio, and 3-D graphics in the questionnaire make the
survey experience much more engaging and less burdensome than
conventional telephone interviews.
Online Studies
The panel status of online survey respondents presents a special circumstance of data collection. The frequency of survey receipt, the length of panel membership, and the incentive package received in exchange for member participation varies by commercial survey outlet, but all respondents to these kinds of surveys are frequent participants in interviews. However, steps are taken through management of the panels to prevent overburdening respondents.
Most surveys take about 30-60 minutes to complete. Surveys longer than 15 minutes are often broken into segments and administered incrementally. As mentioned above, surveys are completed at the leisure of respondents and thereby minimizes burden on respondents. Finally, most companies do not permit the selection of a respondent for more than one survey on the same topic in any three-month period.
A.4. Efforts to Identify Duplication and Use of Similar Information
Health messages developed by CDC are unique in their mix of intended audience, health behavior, concept, and execution. Therefore, in the majority of cases, there are no similar data available. CDC reviews existing published literature and unpublished qualitative pretesting reports when they are available, and also consults with outside experts to identify information that could facilitate message development prior to conducting any data collection.
In the past, CDC joined other U.S. government agencies in networks of organizations that sponsor or endorse health communication projects, such as the Children’s Environmental Health Subcommittee on Health Communication, the NIH/ CDC collaboration on diabetes, the NCI/CDC collaboration on DES, and more recently, the HHS Health Literacy Work Group. These affiliations serve as information channels and help prevent redundancy.
A.5. Impact on Small Businesses or Other Small Entities
Physicians, other health care providers, and small businesses or non-profit organizations can be important intermediaries or target audiences for health messages. When testing messages for these audiences is required, CDC works through established medical and professional societies to gain access to the audience and to obtain feedback on our instruments and data collection plans. As a result, no single “convenience sample” of small entities is overburdened and burden will be kept to a minimum.
This section has limited applicability to general population surveys that would be conducted through the online panel. Current panel members are individuals from the general population, and data collection via the online panel involve no burden to small businesses or entities. In fact, the quick data turnaround potential of the online panel should make message testing with a general audience more feasible, thereby reducing burden on physicians by eliminating the need for expert opinions about how general audience members would respond.
A.6. Consequences
of Collecting the Information Less Frequently
Health message testing might take place at more than one point in a campaign. If the message is not tested, time and money may be wasted developing materials that cannot achieve the health communication objective. Subsequently, if draft materials and messages are not tested, poor execution can undercut a good concept. Each activity will be treated as a separate, one-time study with different respondents. Rarely will information be collected more than once from any given respondent.
There are no legal obstacles to reduce the burden.
A.7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5
There are no special circumstances. The message testing activities fully comply with the regulations and guidelines in 5 CFR 1320.5.
A.8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency
A.8.a. Federal Register Notice
A 60-day Federal Register Notice was published in the Federal Register on Tuesday, July 26, 2011, Volume 76, No. 143, Page(s) 44589-44590. (See Attachment 2) One comment was received from the public. (Attachment 12)
A.8.b Outside Consultation
To ensure there is no duplication or redundancy of effort across projects and programs, program staff will consult with a variety of sources on the availability of data, frequency of collection, clarity of instructions, and record keeping, disclosure, and reporting format (if any), and on the data elements to be recorded, disclosed, or reported. CDC staff will consult with relevant Federal agencies and national associations that conduct health communication campaigns (e.g., American Cancer Society, American Heart Association, the March of Dimes, etc.).
In addition, the following contractors are available for health communication consultation through the OADC Communication Services Contract Mechanism, if needed:
AED, AIR, Battelle, Danya, ESI, Hager Sharp, HMA, Ketchum, NOVA, Ogilvy PR, ORAU, ORC Macro, PRR, RTI, SRA International,Weber Shandwick, and Westat.
CDC Center and Office health communication specialists throughout the agency will conduct, direct, and use these data collections. The Associate Directors of Communication Science (ADCS) in National Centers at CDC are the senior communication specialists in their respective Centers and will consult with programs on their data collections.
Center/Office |
ADCS |
E-Mail/Work#2 |
Office of the Associate Director for Communications (OADC) |
Dogan Eroglu |
404-4986119 |
Office of Surveillance, Epidemiology, and Laboratory Services (OSELS) |
Hilary Polk |
404-498-6340 |
National Center for Health Statistics (NCHS) |
Jeff Lancashire |
301-458-4334 |
Laboratory Science, Policy and Practice Program Office |
Linda Carnes |
404-498-6645 |
Public Health Informatics and Technology Program Office |
Rick Jones |
404-498-6874 |
Public Health Surveillance Program Office |
Oona Powell |
404-498-6578 |
Epidemiology and Analysis Program Office |
John Anderton |
404-498-6145 |
Scientific Education and Professional Development Program Office |
Pat Drehobl |
404-498-6319 |
Office for State, Tribal, Local, and Territorial Support (OSTLTS) |
Karen Resha |
404-498-0296 |
Center for Global Health (CGH) |
Marsha Vanderford |
404-639-7531 |
National Institute for Occupational Safety and Health (NIOSH) |
Fred Blosser (acting) |
202-245-0645 |
Office of Noncommunicable Diseases, Injury, and Environmental Health (ONDIEH) |
|
|
National Center on Birth Defects and Developmental Disabilities (NCBDDD) |
Kate Galatas |
404-498-3484 |
National Center for Chronic Disease Prevention and Health Promotion (NCCDPHP) |
Jeff McKenna |
770-488-8238 |
National Center for Environmental Health (NCEH)/Agency for Toxic Substances and Disease Registry (ATSDR) (Jay Dempsey-Social Media Lead, [email protected], 7/488-3979) |
Jana Telfer |
770-488-0578 |
National Center for Injury Prevention and Control (NCIPC) |
Wendy Holmes |
770-488-4265 |
Office of Infectious Diseases (OID) |
|
|
National Center for Immunization and Respiratory Diseases (NCIRD) |
Kris Sheedy |
404-639-6482 |
National Center for Emerging and Zoonotic Infectious Diseases (NCEZID) |
John O’Connor |
404639-2769 |
National Center for HIV/AIDS, Viral Hepatitis, STD, and TB Prevention (NCHHSTP) |
Susan Robinson |
404-639-8025 |
Office of Public Health Preparedness and Response (OPHPR) |
Dave Daigle |
404-639-1143 |
A.9. Explanations of Any Payment or Gift to Respondents
To defray the costs of participation (e.g., transportation) and to boost response rates, participants in face-to-face interviews, traditional focus groups, central location interviews, and online surveys will receive $5-$75 or the equivalent, depending on the time required. The range of monetary reward is consistent with current rates for participation in formative research studies. Incentives will take the form of cash or gift certificates. In most cases, there will be no remuneration for participation in phone interviews. As described at the end of this section, online panel surveys use an incentive program to improve response rates and maintain membership. Study-for-study, incentives are in line with the others that will be offered under the HMTS.
Reviewed literature revealed the payment of incentives can provide significant advantages to the government in terms of direct cost savings and improved data quality. It also should be noted that message testing is a marketing technique, and it is standard practice among commercial market researchers to offer incentives as part of respondent recruitment.
Doryn Chervin, Ph.D., [(404) 321-3211] a spokesperson from Macro International, Inc. (ICF Macro), a firm experienced in health communication message testing and the various methods used in this research, explained, “Given busy schedules and a plethora of commercial marketing efforts that provide such incentives, social and behavioral science studies cannot compete for the respondents’ time unless an incentive is provided.”
Payments vs. Non-monetary Incentives
Cash incentives have been shown to be most effective in increasing survey response rates for one-time surveys of panel members. For example, Singer and her colleagues noted that, following a series of experiments on the impacts of incentives on various types of survey data collection, “...gifts in this study were less effective in increasing response rates than cash, even with the value of the incentive controlled.”
This finding replicates previous research on the effectiveness of incentives, including a meta-analysis of 38 experiments and quasi-experiments conducted by Church.9 Church reported that non-monetary gifts were significantly less effective than cash in generating survey response, and noted that offering prepaid monetary incentives yielded an average increase of 19.1 percentage points over comparison groups. Moreover, the impacts of monetary incentives seem greater than the impacts of promised charitable donations, lotteries for cash prizes, and other non-monetary rewards.
Level of Incentive Payment
Despite its apparent logic, simply increasing the size of cash incentives to non-respondents does not always result in proportional increases in response rates. In fact, there is some evidence of diminishing returns as incentive levels increase. However, Findlay and Schaible10 found that increasing the incentive payments from $10 to $20 was successful in increasing overall response rates. This incentive was often supported in the literature. Meta-analyses conducted by Church noted incentives provided with initial mailings (i.e., not conditionally linked to the completion of the survey) were the most effective in encouraging increased response.
Reduced Data Collection Cost
Discussion of incentives as a technique to speed responses and expand response rates is not complete without mentioning the trade-off between the costs of incentives and the costs of reminders and other efforts to foster timely and complete participation11. The goal is to find the highest response rate at the lowest overall cost to the government. In the National Adult Literacy Survey by Berlin and colleagues,12 a $20 incentive resulted in not only higher response rates from the sample cohort, but also lower costs per completed case than the comparison group. Importantly, the incentives provided higher response rates from adults with lower-than-average levels of education and basic literacy and numeracy skills.
Reduced Bias
The most important aspect of an incentive plan may be its potential for reducing response bias, underreporting bias, and similar sources of error. Findings from the National Survey of Family Growth (a study in which highly sensitive and personal information is collected from young adults) demonstrated that incentives not only had positive effects on response rates, but they also increased the accuracy of reporting. Incentives are necessary for message testing in order to ensure that those who are willing to participate reflect the target audiences. Failure to provide a basic incentive is likely to bias samples in the direction of well-educated individuals who are generally predisposed to be helpful.
A.10. Assurance of Confidentiality Provided to Respondents
The CDC Privacy Act Officer has determined that the Privacy Act does not apply to data collections conducted according to the procedures described in this application. Although personal information (e.g., gender, age, and race) will be gathered in message testing activities, no personal identifiers (e.g., full name, address or phone number, social security number, etc.) will be collected or maintained. Surveys done through online panels will use already-established records systems.
Much of the data collections in the HMTS will be conducted through either the Creative Domain or the Evaluation Domain of the OADC Communication Services Contract Mechanism managed by the Strategic & Proactive Communication Branch, Division of Communication Service, Office of the Associate Director for Communication.
Creative contractors include AIR, Danya, ESI, Hager Sharp, HMA, Ketchum, NOVA, Ogilvy PR, ORAU, Macro, PRR, RTI, SRI International, Weber Shandwick, Westat and the Academy for Educational Development (AED). Evaluation contractors include AED, AIR, Battelle, Danya, NOVA, ORC Macro International, Inc., RTI, and Westat. Work may be conducted through other contractors carrying out campaign activities but, in all cases, the Strategic & Proactive Communication Branch, Division of Communication Service will require data management procedures be followed.
All data provided by respondents will be treated in a secured manner and will not be disclosed, unless otherwise compelled by law. Respondents will be informed prior to participation that their responses will be treated in a confidential manner.
Also, this project is exempt from IRB requirements. Formative research on health messages (ranging in topics from terrorism to colorectal cancer) is conducted to identify the clarity, salience, and persuasiveness of the message under review. The information gathered in this process is then utilized to target messages appropriately so in the unfortunate event of an urgent health threat, for example, messages then can be disseminated to the public in a clear and sensitive manner. Time-sensitive data collection activities permitted under HMTS, OMB control number 0920-0572, are not research and, therefore, do not require IRB review.
Central Location Intercept Interviews
Respondents will be advised of the nature of the activity, the length of time it will require, and that participation is purely voluntary. Respondents will be assured that no penalties will occur if they wish not to respond to the information collection as a whole or to any specific questions. These procedures conform to ethical practices for collecting data from human participants.
However, because this activity is considered formative evaluation, the procedures normally do not meet the criteria for CDC IRB review (see Attachment 11 for the judgment from CDC’s Acting Deputy Associate Director for Science).
If data are collected by means of paper questionnaires, the questionnaires will be kept in locked filing cabinets in the offices of project staff employed by CDC contractors. When the data have been coded into electronic files and cleaned, the paper records will be destroyed. Electronic files (whether generated
by touch-screen technology, email, or by coding paper records) will be handled as described in the section
on phone interviews below. In reports, all presentation of data will be in aggregate form, and no links to individuals will be preserved. Reports will not include identifiable information on respondents.
Telephone Interviews
Phone interviews will employ a CATI system; no names or other information that could identify the respondent will be recorded. A code number will be assigned to an individual’s responses. It will not be possible to link these code numbers to respondents’ phone numbers.
All selected participants will be informed at the beginning of the phone interview that their responses will be treated in a confidential manner, that all data will be safeguarded closely, and that no individual identifiers will be used in study reports. Interviewers will be extensively trained to impart this information.
All data will be stored in secured electronic files at a contractor’s office and will be accessible only to staff directly involved in the project. All members of the project will be required to sign a statement developed by the contractor pledging their personal commitment to guard the confidentiality of data. Data files will be retained by the contractor for a period of no more than three years and then destroyed.
Online data collections will conform totally to federal regulations [the Hawkins-Stafford Amendments of 1988 (P.L. 100-297) and the Computer Security Act of 1987] and will be required to have comprehensive, written plans to maintain confidentiality. This plan will include having all personnel who will have access to individual identifiers sign confidentiality agreements. They will also be trained in the meaning of confidentiality, particularly as it relates to handling requests for information from respondents, and in providing assurance to respondents about the protection of their responses.
Surveys sent electronically from the Web site will be sent to an email address solely dedicated for the research project. The electronic surveys will be received, a record of the receipt will be made, and the survey will be separated from any identifying information, including the email address of the sender. These surveys will be forwarded to other staff for data analysis.
A.11. Justification for Sensitive Questions
The majority of questions asked will not be of a sensitive nature. There will be no request for a respondent’s Social Security Number (SSN).
It will, at times, be necessary to ask questions considered to be of a sensitive nature in order to test messages about health-relevant behavior. Questions about messages concerning lifestyle (e.g., messages about dietary habits, smoking, drug use, and intimate partner issues), and questions about messages related to illnesses such as cancer or HIV could be considered sensitive. To avoid fear of disclosure of sensitive information, respondents will be told that all data provided by respondents will be treated in a secure manner and will not be disclosed, unless otherwise compelled by law.
To avoid negative reactions to these questions, several steps will be taken:
Respondents will be informed that they need not answer any question that makes them feel uncomfortable or that they simply do not wish to answer.
Where possible, use of touch-screen methodology or other self-directed techniques will provide privacy; not having to verbalize a response may increase comfort.
When such numbers are available and appropriate, participants will be provided with specific agency hotline numbers to call in case they have a question or concern about the sensitive issue.
Interviewers will be trained to ask questions in a sensitive manner and to handle any subsequent discussion skillfully. Where appropriate, interviewers and respondents will be matched for gender and other demographic criteria (e.g., age, preferred language use).
Questions included in these interviews will be pilot-tested with 9 individuals matching the characteristics of the target audience.
All online panel surveys are self-administered and allow respondents to complete the surveys at their convenience, in the comfort and privacy of home. If a survey asks about sensitive topics that respondents may not want other household members to know about, the respondents can protect their own privacy by answering questions on the digital device when other household members are not at home or are sleeping. Additionally, respondents can toggle between the survey and another web page to further protect their confidentiality and change the view on the screen if a household member enters the room. Analyses of response timing have shown that respondents take advantage of the ability to respond to surveys at any hour.
A.12. Estimated Annualized Burden Hours and Cost
Response burden and cost for each type of message testing method are summarized in Table A12A and Table A12B.
Table A12A. Estimated Annualized Burden Hours
Data Collection Methods |
No. of Respondents per Method |
No. of Responses per Respondent |
Average Burden Per Response (in hours) |
Total Burden Hours |
Central Location Intercept Interviews, Telephone Interviews, Individual In-depth Interview (Cognitive Interviews), Focus Group Screenings, Focus Groups, Online Surveys |
18, 525 |
1 |
8/60 |
2, 470 |
Total |
18, 525 |
|
|
2,470 |
The total estimated annualized hourly burden anticipated for all data collection methods would be approximately 2,470 hours. A total of 7,410 burden hours were approved under the existing HMTS package covering a three-year period from November 30, 2008 – November 30, 2011. Based on our projections, we request 7,410 burden hours for this revised HMTS package covering a three-year period.
There are 21 major units of CDC; they are referred to as Centers, Institutes, and Offices. Each National Center or National Institute (see page 8) specializes in a particular health area (e.g., chronic disease prevention, environmental health). It is assumed that 10 of the 21 would initiate two or three message-testing activities each over the course of a year.
Table A12B. Estimated Annualized Burden Costs
Data Collection Methods |
No. of Respondents per Method |
No. Responses per Respondent |
Average Burden per Response |
Total Burden Hours |
Hourly Wage Rage |
Total Respondents Costs |
Central Location Intercept Interviews, Telephone Interviews, Individual In-depth Interview (Cognitive Interviews) Focus Group Screenings, Focus Groups, Online Surveys |
18, 525 |
1 |
8/60 |
2, 470 |
$21.35
|
$52,734.50 |
Total |
18, 525 |
|
|
|
|
$52, 734.50 |
Because the time required for responding to a survey or interview, and to participate in a focus group has a monetary value, this table estimates the total annual cost to the respondents for all activities.
According to the U.S. Department of Labor (DOL) May 2010 National Occupational Employment and Wage Estimates the average hourly wage is $21.35. Because of the scope of this generic clearance and the variety of the types of participants, this average salary was utilized rather than attempting to estimate salaries for groups of audiences. The total annualized burden cost is estimated at $52,734.50 per year.
A.13. Estimates of Annualized Respondent Capital and Maintenance Costs
There is no estimate of respondent capital and maintenance costs.
A.14. Estimates of Annualized Cost to the Federal Government
The average estimated annual cost to the Federal government for conducting the 28 message testing activities proposed in Table A12B is $1,550,000. This total cost includes approximately $1,425,000 for contractual costs (e.g., test design, data collection, analysis, and reporting), and $125,000 for personnel costs for Federal employees involved in project oversight activities (20% time for a HMTS project officer and .80 FTE for data collection project staff).
The Office of the Associate Director for Communication manages a multi-vendor contract appropriate for message testing; the $1,425,000 figure reflects typical costs for such activities under the contract. Any vendor covered by the contract might conduct the actual work.
A.15. Explanation for Program Changes or Adjustments
This is a request for approval of a currently approved GENERIC data collection, which host an approved question bank, managed by the program. In a conference call on June 8, 2011, OMB staff indicated that question banks will no longer be approved and that each GEN IC would be reviewed on an individual basis for approval. Due to this, Program revised their procedures on how GEN ICs would be submitted and created a template for review, “Health Message Testing System Expedited Review Form” (Attachment 3), which request relevant information about the proposed information collection and copies of the data collection instruments.
A.16. Plans for Tabulation and Publication and Project Time Schedule
In most cases, the results of tests of messages and materials will not be published; instead, the information will be used to inform health promotion activities across CDC. Project timelines will vary, depending on the program requirements and the program itself. Message testing ordinarily requires at least one-two weeks to organize, and at least one-two weeks to implement.
A.17. Reason(s) Display of OMB Expiration Date is Inappropriate
All data collection instruments will display the expiration date for OMB approval of the information collection.
A.18. Exceptions to the Certification Statement
No exceptions to the certification statement are being sought.
List of Attachments
Attachment 1: Authorizing Legislation
Attachment 2: Federal Register Notice
Attachment 3: Health Message Testing System Expedited Review Form
Attachment 4: Demographic Questions
Attachment 5: Market Research Experience Questions
Attachment 6: Introductory Questions
Attachment 7: Core Questions
Attachment 8: Activity/Task Questions
Attachment 9: Follow-Up Questions
Attachment 10: Examples:
Example 1: Central Location Intercept Interview
Example 2: Telephone Interview
Example 3: Individual In-depth Interview (Cognitive Interview)
Example 4: Screener Questions for Focus Group Testing
Example 5: Focus Group Moderator’s Guide
Example 6: Online Interview
Attachment 11: IRB Ruling
1 Pechman, C. (2001) A comparison of health communication models: Risk learning versus stereotype priming. Media Psychology, 3(2): 189-210.
2 Delong, D.W., & Fahey, L. (2000) Diagnosing cultural barriers to knowledge management. The Academy of Management Executive, 14(4): 113-127.
3 Andreasen, A. 1995 Marketing Social Change. San Francisco, Jossey-Bass.
4 Black, D.R., Blue, C.L., & Coster, D.C. (2001) Using social marketing to develop and test tailored messages. American Journal of Health Behavior, 25(3): 260-271.
5 Wallendorf, M. (2001) Literally literacy. The Journal of Consumer Research, 27(4): 505-511.
6 Harris-Kojetin, D., McCormack, L.A., Jael, L.A., Sangl, E.F., & Garfinkel, S.A. (2001) Creating more effective health plan quality reports for consumers: Lessons from a synthesis of quality testing. Health Services Research, 36(3): 447-476.
7 Stempel, G.H., Stewart, R.K. (2000) The internet provides both opportunities and challenges for mass communication researchers. Journalism and Mass Communication Quarterly, 77(3):541-548.
8 Bourque, L.B. & Fielder, E.P. (1995) How to Conduct Self-Administered and Mail Surveys. Sage: Thousand Oaks, CA.
9 Church, A.H. (1993). Estimating the Effect of Incentives on Mail Survey Response Rates: A Meta‑Analysis. Public Opinion Quarterly, 57, 62‑79.
10 Findlay, J.S., & Shaible, W. L. (1980). A Study of the Effect of Increased Remuneration on Response in a Health and Nutrition Examination Survey. In the American Statistical Association (ed.), Proceedings of the American Statistical Association Section on Survey Research Method. (pp. 590-594). Washington, D.C.: American Statistical Association.
11 Kulka (1994).
12 Berlin, M., Mohadjer, L., Waksberg, J., Kolstad, A., Kirsch, I., Rock, D., & Yamamoto, K. (1992). An experiment in monetary incentives. In the American Statistical Association (ed.), Proceedings of the American Statistical Association Section on Survey Research Methods (pp. 393-398). Alexandria, VA: American Statistical Association.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | Center for Disease Control and Prevention and |
| Author | Angela Ryan |
| File Modified | 0000-00-00 |
| File Created | 2021-01-31 |