Evaluation of the TravAlert Electronic Messaging System
Data Collection for Evaluation of Education, Communication, and Training (ECT) Activities for the Division of Global Migration and Quarantine
Generic Information Collection Request
OMB No. 0920-0932
Submitted on: January 28, 2013
Statement B
Program Official/Project Officer
Amy McMillen
OMB Specialist
Office of the Director
National Center for Emerging and Zoonotic Infectious Diseases
1600 Clifton Road, NE, MS C12
Atlanta, Georgia 30333
Phone: 404-639-1045
Fax Number: 404-639-7090
Email: [email protected]
Statement B. Collections of Information Employing Statistical Methods 3
1. Respondent Universe and Sampling Methods 3
2. Procedures for the Collection of Information 3
3. Methods to Maximize Response Rates and Deal with Nonresponse 4
4. Test of Procedures or Methods to be Undertaken 5
5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data 5
No statistical methods will be employed in this qualitative study. Statistical methods will not be used to select respondents. Respondents will be selected by interviewers using a type of convenience sample (a turn and select approach). This “turn and select” approach consists of the interviewer physically turning then selecting and approaching the next person that they see. This approach will greatly reduce the effects of interviewer bias in the participant selection process.
Participants will be international travelers arriving at US. airports. The respondent universe consists of English speaking international travelers who are 18 years of age or older and do not have immediate connections to other international destinations. The total universe of potential respondents is 600.
International travelers are a hard to reach population. Exact numbers of English-speaking travelers over the age of 18 and those connecting to other international flights arriving into the United States are unknown. In 2011, there were approximately 62.7 total international arrivals into the US. [1].
Purposive non-probability sampling, with a focus on homogeneous sampling, will be used in order to recruit the 300 English-speaking international travelers needed for interviews for this evaluation. Respondents will be selected by interviewers using a type of convenience sample (a turn and select approach). This “turn and select” approach consists of the interviewer physically turning then selecting and approaching the next person that they see.
This method may provide a slight sampling bias since it is not random and involves specific recruitment efforts. However, the materials being evaluated are designed for this very specific, hard-to-reach audience, and purposive non-probability sampling is needed for this type of population. This approach will greatly reduce the effects of interviewer bias in the participant selection process. The number of respondents to participate in the interviews was chosen based on resources available for the evaluation.
The contractor will work with the respective quarantine station staff and Customs and Border Protection (CBP) officers to determine the best location within the Federal Inspection Services area to recruit for and conduct the interviews. CBP officers and quarantine station staff regularly interact and serve members of the target audience, are familiar with the location of the TravAlert systems containing the measles messages (Attachment 7), and know the best locations for recruiting participants after they have passed by a TravAlert system, and CBP and quarantine station staff will advise the evaluation team on the appropriate locations to identify potential participants and conduct interviews. Once the contractor has identified potential participants, contract staff will use a screening tool and tally sheet to carry out the recruitment (Attachment 3 and 4). In the event participants have additional questions concerning the evaluation, the contractor will distribute a copy of the participant information sheet (Attachment 5) in advance of the interview.
Interviews are an appropriate method for gathering qualitative feedback on educational materials. Information will be collected from 300 English-speaking respondents at four international airports. Each interview will last approximately 5 minutes.
Structured interviews will be conducted by a trained contractor from the contracted agency using an interview guide (Attachment 6). The interviews will be conducted in English. Responses to interview questions will be written down on the interview guide by the contractor.
Analysis will begin after the data have been entered into a data base, following the final interview. A final summary report will be provided to DGMQ and will include aggregate data showing common themes in the data and recommendations for how to improve the evaluated materials, therefore in-depth qualitative analysis is not necessary.
The following procedures have proven effective in previous evaluations and will be used when possible to obtain at least an 80% response rate for this evaluation:
Informing respondents of what the project is asking, why it is being asked, who will see the results, and how the results will be used, as well as discussing how respondents will benefit from the results and how the findings will be put into action (Attachment 5).
A token of appreciation for a respondent’s time and interest will be given to research participants. Tokens will include a plastic bag containing tissues, an adult and childhood immunization schedule, QBHSB informational cards and DGMQ health magnets, lip balm, hand sanitizer, and a small first aid kit.
Addressing data security with respondents, since respondents who know their answers will not be linked to them in any way will be more likely to respond and more likely to provide truthful responses.
Minimizing the time needed for participation in the project.
Informing respondents how much time the project will take so that they know what to expect.
Potential respondents will be informed about the importance of this project.
Interviews
The interview guide is organized by topic (Attachments 6). The topics that will be covered are demographics, environment, message testing, and behavior related to measles. All questions are structured, most are open-ended and a few are closed-ended.
The interview guide’s length of 15 questions was compared with the length of a similar evaluation titled “Evaluation of the Travelers’ Health Port of Entry Influenza Posters” in which intercept interviews were conducted with land border travelers. The Traveler’s Health interviews consisted of 28 questions that focused on influenza and influenza prevention posters that were displayed at the land border crossing. On average, it took participants of the Traveler’s Health evaluation 5-7 minutes to complete the interview. Since the current interview guide contains fewer questions (15 rather than 28), we conservatively estimate that it will take participants no more than 5 minutes to complete.
The current interview guide used the same wording in questions 1, 2, 3a, 3d, 5, and 15 that have been used in numerous other questionnaires. Questions 5, 6, and 10 were modeled after similar questions used in the land border crossing survey. Questions 8,11, 12, and 14b were modeled after questions in the Health Message Testing System Question Bank (OMB No. 0920-0572). The contractor informally pilot tested several of the questions on the interview guide that were created specifically for this evaluation in order to validate the questions (3b, 3c, 4). The pilot testing of these questions was conducted with several individuals in the target audience. Communication and science subject matter experts in the Quarantine and Border Health Services Branch and communication subject matter experts at the OD level of the Division of Global Migration and Quarantine reviewed the intercept interview instruments to ensure that the questions were understandable and that the data being collected were needed for the evaluation project.
No statistical methods will be employed in this qualitative study.
The protocol, screeners, and discussion guides were developed in collaboration with the Oak Ridge Institute for Science and Education and were reviewed by staff of the Division of Global Migration and Quarantine (DGMQ). Staff that participated in the compilation and review of this information included:
Blanche C. Collins, Health Education Specialist, Quarantine and Border Health Services Branch, DGMQ
Gabrielle Benenson, Associate Director for Communications, DGMQ
Amanda McWhorter, former Deputy Team Lead, Communications Preparedness and Response Team, Quarantine and Border Health Services Branch. DGMQ
Nicole Cohen, Associate Chief for Science, Quarantine and Border Health Services Branch. DGMQ
Clive Brown, Associate Chief for Science, DGMQ
Chris de la Motte Hurst, Policy Analyst, DGMQ
Kate Pearson, Policy Lead, Quarantine and Border Health Services Branch, DGMQ
[1] |
U.S. Travel Association, "World Tourism," 13 August 2013. [Online]. Available: http://www.ustravel.org/sites/default/files/page/2009/09/World_US_Datasheet_2011.pdf. |
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Kelly, Kristin |
| File Modified | 0000-00-00 |
| File Created | 2021-01-31 |