SUPPORTING STATEMENT A For:
A Generic Submission for Formative Research, Pretesting, and Customer Satisfaction of
National Cancer Institute’s (NCI) Communication and Education Resources (NCI)
OMB No. 0925-0046, Expiration Date 7/31/2019
Date: July 10, 2019
Check off which applies:
New
X Revision
Reinstatement without Change
Extension
Emergency
Existing
Federal Government Employee Information :
Office of Communications and Public Liaison (OCPL)
Address: 9609 Medical Center Dr., Rockville, MD 20850
Phone: (240) 276-7787
Fax: N/A
Email: [email protected]
Table of Contents
A. Justification 4
A.1 Circumstances Making the Collection of Information Necessary 4
A.2. Purpose and Use of the Information Collection 7
A.3 Use of Information Technology and Burden Reduction 9
A.4 Efforts to Identify Duplication and Use of Similar Information 12
A.5 Impact on Small Businesses or Other Small Entities 13
A.6 Consequences of Collecting the Information Less Frequently 13
A.7 Special Circumstances Relating to the Guidelines of 5 CFR 1320.5 13
A.8 Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency 14
A.9 Explanation of Any Payment or Gift to Respondents 15
A.10 Assurance of Confidentiality Provided to Respondents 17
A.11 Justification for Sensitive Questions 18
A.12 Estimates of Hour Burden Including Annualized Hourly Costs 19
A.13 Estimates of Other Total Annual Cost Burden to Respondents or
Record Keepers 21
A.14 Annualized Cost to the Federal Government 21
A.15 Explanation for Program Changes or Adjustments 22
A.16 Plans for Tabulation and Publication and Project Time Schedule 22
A.17 Reason(s) Display of OMB Expiration Date is Inappropriate 23
A.18 Exceptions to Certification for Paperwork Reduction Act Submissions 23
Attachments
Attachment 1: Approved Generic Sub-studies in 0925-0046 from 2016 to current
ATTACHMENT 2: Privacy Act Memo
JUSTIFICATION
This is a request for OMB to approve a revision for this generic submission for three years. As part of NCI’s mandate from Congress to disseminate information on cancer research, detection, prevention, and treatment, the Institute develops a wide variety of messages and materials. Testing these messages and materials assesses their potential effectiveness in reaching and communicating with their intended audience while they are still in the developmental stage and can be revised. The formative research and pretesting process thus contributes to maximizing NCI’s limited dollar resources for information dissemination and education. NCI also must ensure the relevance, utility, and appropriateness of the many educational programs and products that the Institute produces. Customer satisfaction studies help NCI identify modifications necessary to meet the needs of NCI’s various target audiences. Since the previous submission, there have been 11 approved (and 1 pending) sub-studies with an approved request of 2,682 burden hours over 2.5 years. Approval is requested for the conduct of multiple studies annually using such methods as interviews, focus groups, and various types of surveys. The content, timing, and number of respondents to be included in each sub-study will vary, depending on the nature of the message/material/program being assessed, the methodology selected, and the target audiences.
A.1. Circumstances Making the Collection of Information Necessary
The National Cancer Institute’s (NCI) Office of Communications and Public Liaison (OCPL) conducts various types of research to identify and learn about target audiences.
Specifically, OCPL conducts market and user-centered research and evaluation in order to:
Identify perceived cancer-related needs of diverse audiences
Inform the design and development of NCI resources and ensure that they are appropriate and effective, and reach the intended audiences
Monitor audience trends
Build on and advance the Institute's knowledge base and positioning in the field of user-centered informatics research
Assess the impact of resources and activities
The research conducted in OCPL helps ensure that NCI communication and education resources are appropriate, useful, and effective. This NCI office is requesting OMB to review this package to request a revision on OCPL’s package entitled, “Pretesting, Formative Research, and Customer Satisfaction of NCI’s Communication and Education Resources,” OMB#0925-0046, Expiration: 7/31/2019. The planned proposed changes are:1) an increase in cost due to inflation, and 2) a request for additional burden hours to accommodate the change in response time and anticipated requests over the next three years. Purpose, use, methodology, and design remain the same as the previously approved submission.
The National Cancer Institute (NCI), established under the National Cancer Act of 1937, is the Federal Government's principal agency for research on cancer cause, prevention, detection, diagnosis, treatment, and rehabilitation, and for the dissemination of information for the control of cancer. Current authorization for NCI's education and information dissemination activities is contained in Section 410 of the Public Health Service Act (42 USC § 285).
NCI’s OCPL is the main office within the NCI that provides communications expertise within the Institute and between NCI and a variety of organizations and audiences, including Congress, other executive agencies, state and local governments, scientific and medical communities and institutions, voluntary groups, the press, the general public, and cancer patients. The OCPL, which supports NCI’s priorities through activities that span NCI programs, specializes in the design, implementation, and evaluation of education programs over the entire cancer continuum, including prevention, screening, diagnosis, treatment, survivorship, and palliative care. The office also manages NCI initiatives that address specific challenges in cancer research and treatment. Health care providers, professional societies, patient groups, federal agencies, and the public are audiences for OCPL’s educational programs and products/materials.
The OCPL ensures that the National Cancer Institute’s (NCI) communication and education resources are appropriate, useful, and effective. OCPL uses scientific methods to: identify the cancer-related needs of diverse audiences; inform the design and development of NCI resources; monitor audience trends; and, assess the impact of resources and activities.
Information programs and campaigns within NCI create and use a variety of media including print (e.g., brochures, posters, fact sheets, information kits), broadcast (e.g., public service announcements, video news releases), and electronic formats (e.g., Internet, listservs), as well as direct response (Cancer Information Service) to inform and educate the public and health professionals about cancer. Production of these materials is the major way that the Institute relays messages to the audiences it is mandated to reach.
It is OCPL’s responsibility to ensure that the materials produced by NCI can be understood and are well-received by intended audiences (Formative Research/Pretesting), and meet the satisfaction of NCI’s audiences (Customer Satisfaction). Formative research/pretesting helps ensure that messages have the potential to be received, understood, and accepted by those for whom they are intended; while the customer satisfaction research helps NCI ensure the relevance, utility, and appropriateness of the many educational programs and products that the Office produces. The types of research that are typically employed under this clearance include:
Formative Research and Pre-testing1
To ensure that health messages have the potential to be received, understood, and accepted by those for whom they are intended, Office of Communications and Public Liaison (OCPL) employs formative research and pretesting. Specifically, these types of research involve: 1) assessing audience knowledge, attitudes, behaviors and other characteristics for the planning/development of health messages, education products, communication strategies, and public information programs; and 2) pretesting these health messages, products, strategies, and program components while they are in developmental form to assess audience comprehension, reactions, and perceptions. The information obtained from audience research and pretesting can lead to improvements in materials and strategies while revisions are still affordable and possible. By maximizing the effectiveness of these messages and strategies for reaching targeted audiences, the frequency with which publications, products, and programs need to be modified is reduced.
Because OCPL is NCI's chief resource for cancer communication and education, it has the responsibility to both the Institute and its many constituencies to provide credible cancer information messages and services. The complexity of these messages will only continue to increase as scientific progress in cancer continues its exponential growth. Unless NCI is able to empirically show that its messages and materials have the potential to be understood and useful to their target audiences, it will be unable to carry out its mandate effectively.
Formative research and pretesting are activities that OCPL and other communication offices perform on almost all of NCI’s print pieces, broadcast products, and informational services and messages in order to maximize their usefulness. The studies are conducted on a small scale and focus on potential effectiveness with specific target audiences.
Customer Satisfaction
The OCPL is also responsible for ensuring the relevance, utility, and appropriateness of the many educational programs and products that the Office produces through implementation of customer satisfaction research. By obtaining information from intended audiences on the extent to which materials satisfy their needs, OCPL will be able to systematically establish and follow a feedback loop that provides immediately useful information to revise and enhance educational programs and products so that they attain maximum relevance, utility, appropriateness, and impact. Attachment 1 contains a listing of studies approved along with information collection methods used OMB 0925-0046 expiration date 07/31/2019 from 2016 to current.
OCPL is requesting approval of this generic clearance to continue conducting formative research, pretesting, and customer satisfaction activities through a revision to OMB of No. 0925-0046.
A.2. Purpose and Use of the Information
Formative research, pretesting, and the assessment of customer satisfaction are integral parts of NCI’s overall plan for its communications strategies. Pretesting of print, broadcast, and web products and information services assesses the potential effectiveness of these products while they are still in the developmental stage. Given the large number and wide spectrum of cancer education programs and products OCPL and NCI develop and disseminate, NCI requires accurate, timely, and useful information about the relevance, usefulness, and appropriateness of these products to its customers.
The formative research process is used to determine whether or not a draft message or message concept is effective in reaching and communicating with its audience. Pretesting involves presentation of draft messages designed to convey specific information to a sample of the audience for whom the materials are intended. These respondents are asked to give their reaction to the messages through either individual or group interviews. Information collected to determine the level of customer satisfaction with products helps NCI identify strategies for improving the accessibility of materials/programs, their user-friendliness, and their relevance to the needs of cancer patients and their families, health educators and interventionists, cancer advocates, cancer information specialists, and health care professionals. Research explores various domains shown in the table below.
Factors assessed |
Research questions |
Attention |
Do the messages attract and/or hold the audience's attention? |
Comprehension |
Are the messages and language clearly understood, and does the main theme of the message resonate with the audience? |
Personal relevance and Self-efficacy |
Do members of the target audience perceive the message as personally relevant, and do they see themselves as capable of acting on the message provided? |
Believability |
Is the message and/or its source perceived as credible? |
Acceptability |
Is there anything in the message that may be offensive or unacceptable to the target audience, and is the piece culturally sensitive? |
Accessibility |
Will members of the target audience be able to find the information or message with relative ease, considering factors such as the availability of products in multiple formats (e.g., print materials, videos, online documents, CDs, audio tapes) and in a variety of settings (e.g., clinics, hospitals, doctors’ offices, community organizations, libraries)? |
Usability |
How likely is the respondent to use the information in the format provided, and is the content provided in a logical, organized and user-friendly way? And, do the topics addressed by NCI and OCPL products meet the needs, concerns and interests of its customers? |
Behavioral intent |
Do respondents think they will take action as a result of seeing/hearing the message? |
Readability and clarity |
Can the reader understand publications and educational materials, both in terms of the needs of low-literacy audiences and with respect to plain language principles and design? |
Availability |
Are NCI and OCPL products present for low-English fluency audiences? |
Cultural appropriateness |
Are foreign-language translations or adaptations of products accurate and available for customers? |
Respondents' input and reactions to each of these areas provide insight into how the audiences for these messages may react, how the messages should be formulated or revised to communicate most effectively, and the usefulness, relevance and appropriateness of these products to its customers.
Other information that may be gathered on respondents regarding gender, age, socioeconomic level, race/ethnicity, and family medical history provides a basis for evaluating whether the messages may be perceived differently by different segments of the audience. For example, selected age groups may find a particular brochure or message on cancer prevention more relevant than other age groups.
Systematic formative research and pretesting has been widely adopted by health education program planners as an integral step in the development and targeted dissemination of messages and materials. Through this research, NCI is able to:
Understand characteristics of the target audience—its attitudes, beliefs, and behaviors—and use these in the development of effective communications tools;
Design messages and select formats that have the greatest potential to influence the target audience’s attitudes and behavior in a favorable way;
Determine the best promotion and distribution channels to reach the target audience with appropriate messages;
Expend limited program resource dollars wisely and effectively; and
Results of past pretesting and customer satisfaction efforts have been instrumental in helping NCI carry out its legislative mandate. This generic has hosted a wide variety of sub-studies from formative research, pretesting messages/materials/strategies/formats, as well as customer satisfaction efforts (Attachment 1). In the past three years, information has been collected from various audiences including cancer researchers, the public who reaches out to NCI with questions, health professionals, and various audiences who access cancer.gov to name a few. Information was collected to gain insights on topics such as clinical trials, tobacco education, cancer training, and use of NCI programs such as the NCI Cohort Consortium, Center for Cancer Training, and Communications Fellowship Program. The information collected in the sub-studies helps NCI to best understand the needs of the audiences it is trying to reach to ensure informed and responsibly program planning.
A.3. Use of Information Technology and Burden Reduction
Information collection may be conducted using a variety of methodologies and technologies, such as one-on-one interviews, group interviews, or self-administered questionnaires, depending upon the target audience being questioned and the subject matter being addressed.
As computer technology has continued to improve and become more widespread, opportunities to pretest messages on the Internet using either Web site questionnaires or on-line focus groups with Internet users have increased. Improved technology in the collection and processing of data has the potential to reduce the time burden for respondents and data collectors. For example, respondents can access and respond to data collection requests at a time and place that is convenient to them, eliminating the need to travel for in-person or group interviews. Wherever possible, NCI will make use of Web- or computer-based data collection methods. Transmission of data collection instruments and responses by electronic mail or facsimile will be utilized as appropriate (for example, with intermediary audiences such as NCI’s Cancer Information Service or Public Affairs Network). NCI anticipates that of the majority of data will be collected electronically.
Possible information technologies to assess customer satisfaction include
computer-assisted telephone interviewing (CATI), computer-assisted personal interviewing (CAPI), audio and computer-assisted self-interviewing (ACASI), on-line surveys, and focus groups. Each technology/method is briefly described below:
CATI’s technological capabilities include automated dialing, scheduling unanswered calls or interrupted interviews for efficient callbacks, random selection of respondents, automated skip patterns, instantaneous out-of-range checks, insertion of information from one question to guide a subsequent question, and the automated generation of databases for subsequent analysis. CATI will be utilized when geographic diversity is important and participants come from hard-to-recruit populations, such as physicians or Native Americans. CAPI technology allows interviewers to ask questions of a respondent using a computer to enter data. ACASI software technology offers many advantages of CAPI technology, but removes the need to have a person administer an interview. Instead, survey questions are pre-recorded and played back through the sound system of a computer, which the respondent can listen to privately by using headphones. Respondents select an answer by pressing a key that corresponds to one choice shown on the screen, after which answers are fed directly into a computer database.
ACASI surveys can also be administered over a telephone by entering the response on the telephone keypad. ACASI technology is particularly useful in administering surveys to low-literacy populations or when addressing sensitive topics that respondents may not feel comfortable discussing in the presence of someone else.
On-line surveys represent an especially convenient option for eliciting feedback from consumers of Web-based products. Respondents complete online surveys regarding a product and then submit the data electronically over the Internet. With online surveys, respondents can easily submit feedback during or immediately after using a Web-based product. They also allow participation from international audiences with virtually no additional costs.
Technology now enables the conduct of focus groups, which traditionally take place in-person with a moderator facilitating a discussion regarding a product, issue, or program with a small group of individuals in a designated physical location. Depending on factors such as geographic distribution and schedules of NCI customers and on the nature of the products and services under investigation, focus groups, when appropriate, may be implemented using a variety of technology-based formats:
Videoconferencing uses video and satellite technology to allow a group of participants located in multiple geographic locations to interact with one another both visually and aurally. A facilitator and a technical team located in a hub site maintain the video and audio connections among participating sites.
Internet or online conferencing is especially useful for discussions with specific individuals or international participants. This format functions as a sort of “chat room” in which a moderator intercepts and distributes e-mail transmissions from participants who have logged onto a specially designated Web site.
Teleconferencing uses telephone technology to facilitate an exchange among participants located in multiple geographic locations. Participants dial into a specially designated phone number or “bridge line” that is moderated by a focus group facilitator.
Advanced Strategies Lab (ASL) is a qualitative research process that uses an online interactive discussion. In the 90-minute, online ASL sessions, participants receive instructions and are asked questions from the moderator over the phone on a conference call. Participants’ answers are typed into their designated dialogue box (which hides their identity from the other participants to ensure privacy). The facilitator then guides the group through online brainstorming, discussion, and assessment exercises. The ASL’s software automatically formats and tabulates data as the session proceeds. Summaries of survey results are available seconds after respondents complete a question, and full verbatim reports are available within hours after a session.
The NCI Privacy Act Coordinator was consulted, and it was determined that a Privacy Impact Assessment (PIA) is not needed. In rare instances, when PII is collected to gather and retain respondent names and contact information (such as addresses, telephone numbers, or email addresses), this information would be used to measure customer satisfaction information regarding an NCI/OCPL product at one time and then at some point later (for example, to learn whether customers have gone back to the product for additional review or information). Data will not be retrieved by personal identifiers unless the respondent voluntarily agrees to provide the information, so he/she can be contacted for a follow-up.
A.4. Efforts to Identify Duplication and Use of Similar Information
The areas in which information needs to be gathered (as described in A.2. above - attention, comprehension, etc.) to pretest effective cancer messages (brochures, PSAs, media campaigns, website, etc.) are generally similar from pretest to pretest. However, since the cancer field is so diverse and complex, and each message is essentially different, new data collection instruments generally must be prepared for each pretest. As each new message, strategy, or product is developed, NCI reviews existing literature and data bases, including pretesting reports on existing messages and materials, and consults with outside experts to evaluate available information on similar messages with comparable audiences.
NCI will continue to assess other NIH active generic ICs upon submission of applications to conduct each sub-study. Currently, the existing NIH active generic ICs are very site- or method-specific. Though other Institutes at NIH have a Generic submission, none would be able to accommodate the population and content that the National Cancer Institute’s Generic IC anticipate needing in the areas of customer satisfaction, pretesting NCI communication messages, and formative research. Additionally, NCI has an internal review process for surveys that will be used by this generic to assess the quality of each survey prior to its use. The NCI will provide direct oversight for any and all surveys conducted under this generic clearance to avoid duplication of effort and information collected.
A.5. Impact on Small Businesses or Other Small Entities
Small businesses that are non-profits and independently-owned may be participants in this generic submission. The small businesses we may include are physicians, health care providers, and highly specialized individuals may sometimes be the target audience for NCI’s communication information and customer satisfaction materials. When small businesses are asked to complete an information collection, all efforts will be made to reduce their burden by using a short questionnaire/survey and interviewing fewer small businesses than larger ones.
A.6. Consequence of Collecting the Information Less Frequently
For the most part, formative research, pretesting, and customer satisfaction information will be collected only one time for each print, broadcast, or electronic message, product/material, or strategy tested. However, there may be occasion where a pre- and post-test to assess differences in communication and/or satisfaction messages may be useful for a particular sub-study. Additionally, previous respondents may be contacted to participate in follow-up studies if they have originally granted consent for such and if the subsequent study uses that population.
A.7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5
NCI recognizes the need to collect information in a manner that places minimal burden on each respondent. Therefore, NCI anticipates that information collections under this clearance package will comply with 5 CFR 1320.5(d)(2) requirements with only two anticipated exceptions.
When NCI requires responses to a self-administered written questionnaire in less than 30 days, receipt of the questionnaire is generally preceded by advance notification to respondents explaining the purpose of the questionnaire, the approximate length of time that the questionnaire will take, and the voluntary nature of participation. All efforts are made to keep such questionnaires short and focused.
Because NCI’s pretesting activities are often qualitative in nature, the results are not generalizable to the population at large or to the particular target audience under study. However, the nature of pretesting is such that generalizability is not a critical feature; the emphasis is on obtaining timely, useful information that can be fed back into the development of new messages or materials or the revision of existing ones.
There are no other special circumstances.
A.8.1 Comments in Response to the Federal Register Notice
The 60-Day Federal Register notice soliciting comments on this study prior to initial submission to OMB was published on April 5, 2019 (Volume 84, No. 66, Page 13670). No public comments were received.
A.8.2 Efforts to Consult Outside Agency
NCI, along with other Public Health Service agencies, has been a leader in the development of methods for developing, testing, and disseminating health information. The work of many advisers over many years has brought us to where we are today. A number of outside health communications experts were consulted to review the plans contained herein for program development research and evaluation of NCI communications programs and their comments and suggestions have been incorporated into these data collection plans.
Although all consumer materials, as well as materials produced for the media, must go through DHHS approval procedures, NCI is not required to coordinate with other Federal agencies in its education, information dissemination, and evaluation activities. Nevertheless, NCI has consulted with other agencies in the past and will continue to do so in the future, as appropriate, to help ensure accuracy and consistency, and to avoid duplication of effort. It should also be noted that many of the pretesting and formative research efforts conducted by NCI have been requested and used by outside agencies to inform their own communications activities. OCPL also consults with several internal experts across NCI, most frequently in NCI’s Division of Cancer Control and Population Sciences (DCCPS) on the conduct of study design, methods, and data analysis and reporting. Any collaborations and consultations that occur for specific sub-studies will be mentioned as part of the request for approval in the sub-study.
A.9. Explanation of Any Payment or Gift to Respondents
It is possible that some information collection activities will entail a small payment or gifts to respondents. Small amounts of money, a free meal or snack scheduled around the time of the pretest, and/or incentive for parking and/or transportation are most often used, particularly when recruiting hard-to-reach and minority respondents. Incentives may also be necessary in some sub-studies where highly specialized individuals are invited to participate in a survey and will cover the cost of transportation and other types of local expenses.
Research has shown the advantages of providing a small incentive for improving response rates and decreasing item nonresponse, especially in mail and telephone surveys.2 Studies of participants in the original National Health and Nutrition Examination Surveys (NHANES) found that response rates for those told they would receive an incentive versus not were 82% and 70%, respectively.3 The National Survey of Family Growth conducted an experiment with incentive of respondents and found that incentives increased response rates, reduced interviewer labor (broken appointments and callbacks), and improved data quality.4 In the 1999 Observing Protein and Energy Nutrition study, incentive was credited as contributing to high response and retention rates.5
Studies involving physicians in healthcare surveys suggest giving a high-value incentive since physicians are hard-to-reach and often busy.,.6 For the individuals (in-person focus groups), the increase is similarly warranted in that a higher amount is closer to industry standards and will make our recruitment go faster. Based on this information, program would like to increase the incentive rates for Physicians from $100.00-$150.00 to $150.00-$200.00, a $50.00 increase, and for individuals who participate in-person focus groups from $50.00-$75.00 to $50.00-$100.00, a $25.00 increase. This increase will encourage those hard to reach, busy physicians and individuals to respond quickly. A quicker response to the request for information by the physicians and individuals will reduce the overall time that the contractor has to spend reaching out to the physicians and individuals, thereby reducing the overall government contractor cost.
Instances for offering an incentive will be determined on a case-by-case basis (depending on the particular information collection design). The following are the kinds of situations for which respondents may be provided incentive or offered a gift:
Physicians who participate in a survey on their opinions about the relevance of materials for their patients may receive an incentive (for example, $150-200/hour based on current hourly salary rates).7
Individuals who participate in in-person focus groups may receive an honorarium ($50-100/hour depending on segmentation strategy, methodology, time and effort required, average hourly salary, and recruitment challenges) to cover their time, transportation costs, and childcare expenses.8
Health educators who support the acquisition of data related to customer satisfaction with specific educational programs and products may be able to request and receive certain quantities of materials that exceed the limits usually established for those materials.
Circumstances, however, do not always require that incentive be given; many audiences including the public, patients, survivors, and other health professionals often participate gratis because of their interest or involvement in the topic, or as a professional courtesy. For example, in situations when the general public is completing an online survey, no incentive will be involved unless influenced by other factors.
A.10. Assurance of Confidentiality Provided to Respondents
Information provided by respondents will be kept private, to the extent provided by law. This will be communicated to respondents by means of introductory letters, explanatory texts on the cover pages of questionnaires, scripts read prior to focus groups or telephone interviews, and consent forms. Respondents will also be advised of the following: the nature of the activity; the purpose and use of the data collected; NCI sponsorship; and the fact that participation is voluntary at all times. Because responses are voluntary, respondents will be assured that there will be no penalties if they decide not to respond, either to the information collection as a whole or to any particular questions.
As a further guarantee of confidentiality, all presentation of data in reports will be in aggregate form, with no links to individuals preserved. Reports will be used only for research purposes and for the development of communication messages and educational materials. Only NCI staff and contractor personnel conducting the information collection will have access to individual-level survey, interview, or focus group data. All project/contractor staff conducting the information collection will sign a confidentiality agreement, and all electronic and hard-copy data will be maintained securely throughout the information collection and data processing phases. While under review, electronic data will be stored in locked files on secured computers; hard-copy data will be maintained in secure building facilities in locked filing cabinets. Before any data are released for public use data sets, any identifying information will be stripped from each respondent’s record and the identifying information will be destroyed.
The NIH Privacy Act Officer has reviewed the work scope of this proposal to determine whether the Privacy Act is applicable to this data collection and will be asked to review the protocol of each collection under this blanket clearance to ensure that NCI adheres to privacy requirements (Attachment 2).
Although some personally identifiable information will be collected, data will not be retrieved by personal identifiers unless the respondent voluntarily agrees to provide the information, so he/she can be contacted for follow-up (in rare situations). Instances could arise for activities that, for example, gather and retain respondent names and contact information (such as addresses, telephone numbers, or email addresses). This information would be used to measure customer satisfaction information regarding an NCI/OCPL product at one time and then at some point later (for example, to learn whether customers have gone back to the product for additional review or information.)
The Privacy Act is applicable as determined by the NIH Privacy Officer in the Privacy Act Memo (Attachment 2). The applicable SORN is NIH Privacy Act Systems of Record 09-25-0156; “Records of Participants in Programs and Respondents in Surveys Used to Evaluate Programs of the Public Health Service, HHS/PHS/NIH/OD.”
Pretesting efforts described in this proposal are considered exempt from the “Regulations for the Protection of Human Subjects,” in accordance with paragraph (b)(3) of 45 CFR Sec. 46.101 (http://citfm.cit.nih.gov/ohsr/nih/guidelines/GrayBooklet82404.pdf).9
A.11. Justification for Sensitive Questions
As mentioned in sections A.2. and A.10. above, some studies require the inclusion of people who match selected characteristics of the target audience that NCI is trying to reach. Therefore, personally identifiable information (PII) such as race/ethnicity, income, education and/or medical/health status, are required to be asked on the initial screening questionnaire used for recruiting. Potential participants are informed that this is being done to make sure that NCI speaks with the kinds of people for whom its messages are intended. Again, respondents are assured that the information is voluntary and will be treated as private. All information on race/ethnicity will comply fully with the standards of OMB Statistical Policy Directive No. 15, October 1997.
Since NCI communications are concerned with the detection, diagnosis, treatment, and prevention of cancer, some projects may involve asking questions about (or discussing) one’s experience with their cancer journey or how one perceives his/her own personal risk for serious illness. Fears of cancer and experiences with cancer may also be covered. This information is needed to gain a better understanding of the target audience so that the messages, strategies, and materials designed will be appropriate and sensitive. Questions of this nature, while not as personal as those on sexual behavior or religious beliefs, still require some sensitivity in how they are worded and approached. In face-to-face data collections, questions of this kind are generally asked later in the interview or group discussion when respondents are more comfortable with the interview situation and are more at ease with the interviewer/moderator. As noted in section A.10., participants are informed in advance about the nature of the activity and the voluntary nature of their participation. The interviewer/moderator makes it clear that they do not have to respond to any question that makes them uncomfortable, and that no questions of a sensitive nature will be asked in the course of collecting information from respondents.
Raw data from data collections that include sensitive information (for example, screening questionnaires and audio tapes) are not retained once the data have been extracted and aggregated; nor does the information become part of a system of record containing permanent personally identifiable information that can be used for retrieval.
A.12. Estimates of Hour Burden Including Annualized Hourly Costs
It is estimated that on an annual basis, the total number of respondents is 18,000 which is a change from the previously submitted number of 36,000 as the new totals more closely reflect the true numbers, and the annual estimated burden was increased to 13,500 hours versus 7,200 hours due to the increase in average response time from 12 minutes to 45 minutes to accommodate the response time to complete the surveys/interviews . The types of respondents may include but are not limited to the general public (patients, survivors, family members), physicians, other health professionals, and researchers. The Category of Respondents will primarily be Individuals and Households, but at times may include Local, State, or Federal Government and the Private Sector. The number of respondents to be included in each sub-study will vary, depending on the target audience and the nature of the message/materials/program being assessed. Typically, the frequency of responses per respondents would be once. However on occasion, respondents may give permission for a follow-up survey or interview and since this happens fairly infrequently it is not being accounted for in Table A.12-1. Sub-study submissions will indicate when NCI anticipates follow-up surveys/interviews with a given number of respondents. The average time, or burden per respondent, can range from five minutes to 90 minutes, with the average being around 45 minutes. It is also difficult to pinpoint the actual number of studies required during the next three years; we can only anticipate need based on past studies.
Table A.12-1 Estimated Annualized Burden Hours
Form Name |
Type of Respondents |
Number of Respondents |
Number of Responses per Respondent |
Average Time Per Response (in hours) |
Total Annual Burden Hours |
Focus Groups, Individual In-Depth Interviews, Brief Interviews, Surveys, Website Usability Testing |
Individuals (General Public) |
9,000 |
1 |
45/60 |
6,750 |
Focus Groups, Individual In-Depth Interviews, Brief Interviews, Surveys, Website Usability Testing |
Individuals (Health Care Professionals) |
9,000 |
1 |
45/60 |
6,750 |
Total |
|
|
18,000 |
|
13,500 |
A.12-2 Annualized Cost to Respondents
The annual cost to respondents is $426,397.50. Table A.12-2 presents the annualized approximate cost to respondents.
A12.2. Annualized Cost to the Respondents
Type of Respondents |
Total Annual Burden Hours |
Hourly Wage Rate* |
Respondent Cost |
Individuals (General Public) |
6,750 |
$24.34 |
$164,295.00 |
Individuals (Health Care Professionals) |
6,750 |
$38.83 |
$262,102.50 |
Total |
13,350 |
|
$426,397.50 |
*The source of the Mean Hourly Wage Rate is for Health Care Professionals (Physician and Scientist, 29-1069) and General Public (All occupations, 00-0000) from the Bureau of Labor Statistics (http://www.bls.gov/oes/current/oes290000.htm)
A.13. Estimate of Other Total Annual Cost Burden to Respondents or Recordkeepers
There are no capital or start-up costs to the data collection efforts requested; nor are there any costs associated with operation, maintenance or purchase of services.
A.14. Annualized Cost to the Federal Government
The estimated annualized cost to the federal government is approximately $300,095.90. Table A.14-1 shows the breakdown of estimated costs, with figures that include the costs of study design, facility rental (e.g., for focus groups), data collection, analysis, and report/publication writing. This estimate also includes monitoring by the Senior Researcher 1 and Senior Researcher 2 (GS 14/8) and Project Director (GS-14/10), projected to be about 10%, 5%, and 10% respectively effort a year for each position. The federal personnel will be planning, executing, and reporting the research and information collection. Contractor cost is estimated to be $264,000.00 to plan for, conduct, and analyze information collection activities.
Table A.14-1 Annualized Costs to the Federal Government
Cost Descriptions |
Grade/Step |
Salary** |
Percent Effort |
Fringe (if applicable) |
Total Cost to Gov’t |
Federal Oversight |
|
|
|
|
|
Senior Researcher 1 |
14/8 |
$141,328 |
10% |
|
$14,132.80 |
Senior Researcher 2 |
14/8 |
$141,328 |
5% |
|
$7,066.40 |
Project Director |
14/10 |
$148,967 |
10% |
|
$14,896.70 |
Contractor Cost |
|
|
|
|
$264,000.00 |
Travel |
|
|
|
|
$0 |
Other Cost |
|
|
|
|
$0 |
Total |
|
|
|
|
$300,095.90 |
** the salary table above is cited from https://www.opm.gov/policy-data-oversight/pay-leave/salaries-wages/salary-tables/19Tables/html/DCB.aspx
A.15. Explanation for Program Changes or Adjustments
This information collection request is a revision, due to an increase in the incentives to the respondents, the proposed incentive amount for Physicians has been increased by $50.00, and for Individuals (General Public) by $25.00, a reduction in the number of respondents from 18,000 per year to 9,000 per year, and the Average Time Per Response (in hours) from 12 minutes to 45 minutes, which will increase our Burden Hours from 7,200 to 13,500. As discussed in A.9, the proposed incentive amount increases for Physicians and for Individuals (General Public) will ensure that NCI can attract hard-to-recruit populations to improve response rates, data quality, and motivate respondents to participate.
A.16. Plans for Tabulation and Publication and Project Time Schedule
Evaluation/research staff will review the material to be tested; discuss the objectives with the individuals responsible for development; determine the analytic questions to be addressed; and then prepare the research procedures, instruments, and data analysis plan. The analysis conducted for each study will be determined by the objectives of the research, the messages being tested, and the audience for the messages. Specifics of each analysis cannot be determined until the messages to be tested are prepared. Research techniques may include qualitative analysis, customer satisfaction analysis, descriptive statistics, statistical analysis and parametric statistical tests. Analysis may be conducted using cross-tabulation procedures, with categorical variables; or between-group procedures, with continuous variables.
While the primary purpose of all studies is to provide information to the developers of the messages/materials/services for the purpose of improving them, NCI shares information internally and also makes results available to a variety of health program planners at Government agencies, voluntary organizations, health professional organizations, and medical institutions. Information collected will be compiled and presented in reports and briefings for staff from OCPL and other divisions within NCI and NIH. Reports will include information regarding respondent demographics, basic descriptive data with OCPL products and services, comparisons across demographic and customer subgroups, recommendations for improving programs and products, and analyses of longitudinal changes. In addition, NCI may also publish results in journals and present the findings of its research at meetings of professional associations, for example, the American Public Health Association and the Society for Public Health Education. Formative research conducted by OCPL is also sometimes summarized in news-related publications such as the NIH Record.
The specific messages and materials/service that will be evaluated and the timing of these studies are not known at this time. While the research period varies somewhat depending on the complexity of the testing and number of respondents required, the typical study will require approximately 12 weeks from initial design to preparation of the report of pretest findings. A schedule for a typical pretest is shown below:
A.16-1 Project and Publication Timeline
A.16-1 Project Time Schedule |
|
Activity |
Time Schedule |
Collection of data |
1 – 8 weeks after OMB approval |
Analysis of data |
9-10 weeks after OMB approval |
Write summary of findings |
11-12 weeks after OMB approval |
A.17. Reason(s) Display of OMB Expiration Date is Inappropriate
NCI will continue displaying the OMB control number and expiration date in the upper right-hand corner of all data collection instruments.
A.18. Exceptions to Certification for Paperwork Reduction Act Submissions
NCI is in full compliance with the provisions contained within the Certification for Paperwork Reduction Act.
1 The following information was in the approved 2016 Revision as an attachment and is now being included in the SSA.
2 E. Singer, J. Van Hoewyk, and M. P. Maher, “Experiments with Incentives in Telephone Surveys,” Public Opinion Quarterly, Vol. 64, No. 2, Summer 2000, pp. 171-188; A. H. Church, “Estimating the Effect of Incentives on Mail Survey Response Rates: A Meta-Analysis,” Public Opinion Quarterly, Vol. 57, No. 1, Spring 1993, pp. 62-79.
U.S. Department of Health, Education, and Welfare, “A Study of the Effect of Remuneration Upon Response in the Health and Nutrition Examination Survey, United States,” Vital and Health Statistics, Series 2-No.6, 1975.
4 W. D. Mosher, W. F. Pratt, and A. P. Duffer, “CAPI, Event Histories and Incentives in the NSFG Cycle 5 Pretest,” American Statistical Association, 1994 Proceedings of the Section on Survey Research Methods, Vol. 1, 1995, pp. 59-63.
A. F. Subar, V. Kipnis, R. P. Troiano, et al., “Using Intake Biomarkers to Evaluate the Extent of Dietary Misreporting in a Large Sample of Adults: the OPEN Study.” American Journal of Epidemiology, Vol. 158, 2003, pp. 1–13.
9 Reference pages 8 and 21 for details about exemptions.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | McCarthyK |
| File Modified | 0000-00-00 |
| File Created | 2021-01-15 |