ECT Supporting Statement B_QR codes 06112013

Information Collection for Evaluation of Education, Communication, and Training (ECT) Activities for the Division of Global Migration and Quarantine

Evaluating the Effectiveness of Quick Response Codes in Educating Panel Physicians

Generic Information Collection Request

OMB No. 0920-0932

June 11, 2013

Statement B

Program Official/Project Officer

Amy McMillen

OMB Specialist

Office of the Director

National Center for Emerging and Zoonotic Infectious Diseases

1600 Clifton Road, NE, MS C12

Atlanta, Georgia 30333

Phone: 404-639-1045

Fax Number: 404-639-7090

Email: [email protected]

TABLE OF CONTENTS

PART B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS 2

1. Respondent Universe and Sampling Methods 2

Table 1: Potential Respondent Universe 3

2. Procedures for the Collection of Information 3-5

3. Methods to Maximize Response Rates and Deal with Non-response 5

4. Test of Procedures or Methods to be Undertaken 5

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data 6

REFERENCES 7

ATTACHMENTS 7

Section B – Collections of Information Employing Statistical Methods

1. Respondent Universe and Sampling Methods

The evaluation research participants include international panel physicians 18 years of age and older who are certified physicians in their respective countries and who have been appointed by the consular section to serve as an international panel physician. Randomized (probabilistic), purposive, and theoretical sampling methods will be employed.

Sample size:

The Centers for Disease Control and Prevention’s (CDC), National Center for Emerging and Zoonotic Infectious Diseases (NCEZID), Division of Global Migration and Quarantine (DGMQ), Immigrant Refugee, and Migrant Health branch (IRMH) works with approximately 700 international panel physicians from 151 jurisdictions (countries). The number of panel physicians per country varies depending on country size, population, and the number of immigrants and refugees migrating from that country to the United States.

In 2007, CDC updated the tuberculosis Technical Instructions to use newer, more precise tests to improve TB detection rates before refugees and immigrants come to the United States and to improve the health of refugees and immigrants through earlier diagnosis and updated treatment methods. The 2007 Technical Instructions updates make testing more precise by requiring cultures for refugee and immigrant applicants thought to have TB. In addition to the 1991 screening requirements which include a complete medical history, physical examination, chest x-ray and sputum samples, applicants with signs and symptoms of TB have now have to provide cultures to confirm presence of TB bacteria. Drug susceptibility testing (DST) is also a new addition and is required on all positive TB cultures to see which drugs are most effective to kill the TB bacteria found in the culture. The third update is the addition of directly observed therapy (DOT) for applicants diagnosed with TB. To date, 56 jurisdictions have fully implemented the updated 2007 Culture and Directly Observed Therapy Tuberculosis Technical Instructions (CDOT TB TIs). The 95 jurisdictions that have yet to implement the CDOT TB Technical Instructions have a deadline of October 1, 2013. Given that the October 1, 2013 deadline is less than a year away, we decided to select the jurisdictions that have yet to implement the CDOT TB Technical Instructions as our study sample.

The 95 panel site jurisdictions will be randomized into two groups: exposed and non-exposed. We will use www.random.org to generate a randomized list of panel physicians in the 95 jurisdictions. Proceeding from the top to bottom of the list, panel physicians will be assigned one of two numbers; 0 or 1 (0 for non-exposed and 1 for exposed). In cases where there is more than one panel physician in the jurisdiction, we will simply select the first name on the randomized list to be the participant from that jurisdiction. The total universe of respondents is N=95, where N represents panel physicians. Panel physicians in the exposed group will receive new laminated reference guide materials and fact sheets loaded with QR codes (materials will cover a range of topics including CDOT TB implementation guidelines, vaccination reference guides, mental health fact sheets, and sexually transmitted infections screening guidelines etc.). Those in the non-exposed group will receive the same materials, but without QR codes. Panel physicians at the selected sites will be contacted by a member of the research team inviting them to participate in the evaluation study. If a physician cannot be reached or declines to participate, another panel physician from that jurisdiction will be contacted.

Given that all the panel physicians are required to be fluent in English, communication between the research team and panel physicians will be conducted in English.

Table 1: Potential Respondent Universe

Entity	Potential Respondent	N
Panel sites jurisdictions yet to implement 2007 CDOT TB TIs	International Panel Physician(s) Randomize Exposed (s=48) Non-Exposed (s=47)	95
Total Universe of Potential Respondents		95

2. Procedures for the Collection of Information

Panel physicians will receive an email (first screen, Attachment A1) inviting them to participate in the evaluation. At the bottom of the email, there will be a link to a pre-aptitude knowledge survey (see methods section for details). The first page that appears after the participants click on the link will be a statement of consent. In order to proceed to the survey, participants will need to click OK, indicating their consent to participate. In addition, those who agree to participate in the evaluation will receive a statement of consent via email for their records. The statement of consent will include details about the risks and benefits of participating in the evaluation, confidentiality procedures, and participation procedures and requirements.

The research team will assign the participating panel physicians a unique numeric identification code. From this point forward, the numeric code will be used on all documents instead of the panel physician’s name. The research team will have a master list in Microsoft Excel, which indicates the name and numeric code of each participant. All contact information will be stored separately from notes and recordings. Only members of the research team will have access to contact information, field notes, and interview information.

After the participants have provided consent to participate in the evaluation and completed the pre-aptitude knowledge survey (which will be used to collect base demographics such as name, contact information, location, and type of mobile device) they will receive an email with further details indicating the dates when the new materials will be released and when the semi-structured questionnaires will be administered.

Prior to data collection, the panel physicians will be provided with new copies of laminated reference sheets, fact sheets, and other resources. The updated materials will focus on implementing the CDOT TB Technical Instructions as well as vaccination technical instructions, mental health technical instructions, and sexually transmitted infections technical instructions. Panel physicians in the exposed group will receive materials loaded with QR codes. Materials will be mailed or provided during FY 2013 site visits and trainings.

Pre-Knowledge Aptitude Survey: A pre-knowledge aptitude web-based survey will be conducted with all 95 study participants. The survey will consist of closed and open-ended questions. The purpose of the survey is to collect demographic information and to gauge familiarity with QR codes. The survey will be administered before the new materials with QR codes are distributed. The survey will be administered using MS Interview software and will take no longer than 10 minutes to complete. The research team will be notified when participants complete the survey. (See Attachment A(1)—Pre-Knowledge Aptitude Survey).

Semi-Structured Questionnaire: Forty-eight semi-structured questionnaires will be administered to panel physicians in the exposed group. Forty-seven semi-structured questionnaires will be administered to panel physicians in the non-exposed group. The questionnaires--which will be a compilation of open-ended, closed-ended, and Likert scale questions--will be administered online using MS Interview software. Participants will receive an email with a link, which will take them to the questionnaire. The questionnaire will take 15-20 minutes to complete. After completing the questionnaire, the participants will click ‘submit.’ Upon submission, the research team will be notified that the questionnaire has been completed. Participants will have two-weeks to complete the questionnaire. The research team will send a reminder email one-week before the questionnaire opens and before it closes. (See Attachment A(2)—Semi-Structured Questionnaire.

In-Depth Interviews: Interviews will be conducted with 20 key informants. Key informants will include panel physicians and panel site staff from panel site jurisdictions in the exposed group. Because the in-depth interviews will be conducted in-person during FY 2013 quality assessment site visits, the jurisdictions that will be able to participate in the in-depth interviews are already pre-determined. Interviews will be recorded (upon consent) using an audio voice recorder and will last no longer than 60 minutes. The purpose of the in-depth interview is 1) to understand panel physicians’ experience with QR codes for exposed participants, 2) to identify and address specific challenges associated with QR code use for exposed participants, 3) to identify the advantages of using QR codes to access Technical Instructions and the Panel Physician Portal for exposed participants, 4) to determine what types of medical content and information is most useful to panel physicians, and 5) to determine which tool is most effective for linking panel physicians and their staff to Technical Instructions and medical screening requirements. Panel physicians will be informed about the option to participate in the interview before the site visit occurs. A written consent letter will be provided onsite before the interview takes place. (See Attachment A(3)—In-Depth Interview Guide).

With assistance from the web informatics department, we will monitor how frequently the online technical instructions and other reference materials are accessed by scanning a QR code. This will allow us to identify if the QR codes are being used. Unfortunately, due to regulations, the web team is not able to identify the person who is accessing the information, so we cannot make comparative conclusions between the exposed and non-exposed group. Data from the semi-structured questionnaires will be analyzed using the statistical software SPSS. Members of the research team will transcribe and code the in-depth interviews using the qualitative management software Atlas.ti.

3. Methods to Maximize Response Rates and Deal with No Response

The following methodology and best practices will be employed when possible to obtain an 80% response rate:

(1) Informing respondents of what the study is asking, why it is being asked, who will see the results, and how the results will be used, as well as how respondents will benefit from the results and how the findings will be put into action

(2) Using bilingual and bicultural interviewers and culturally and linguistically appropriate data collection instruments

(3) Providing easy access to research instruments. Research instruments will be designed to be easily accessed by electronic means, from a link in an e-mail or on a website

(4) Addressing confidentiality and anonymity with respondents: since respondents who know their answers will not be linked to them in any way will be more likely to respond and more likely to provide truthful responses

(5) Minimizing study length while maximizing the richness of data that can be obtained. Respondents will be told how much time the study will take to complete so they know what to expect.

4. Test of Procedures or Methods to be Undertaken

1. Semi-Structured Questionnaires and Pre-Knowledge Aptitude Survey

2. In-Depth Interviews

The in-depth interview guide, semi-structured questionnaire and survey were pilot-tested by DGMQ health communication and evaluation experts. The tests proposed for survey instruments include several procedures as outlined by Michael W. Eysenck (1) in A Handbook of Cognitive Psychology.

Procedures used included:

• Developing protocols, scenarios, and question probes--follow-up questions used to gain more information about respondents' strategies for answering questions.

• Concurrent think-aloud interview--respondents think aloud while answering questions and responses are probed extensively.

• Retrospective think-aloud interview--respondents answer all questions first, then are asked how they arrived at their answers.

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

The protocol and data collection instruments were developed and reviewed in collaboration with staff of the Division of Global Migration and Quarantine (DGMQ) and the
Immigrant, Refugee, and Migrant Health branch (IRMH) and the Office of the Director (OD).

The following individuals, including contractors, provided advice about the protocol design, sampling methods, and data collection tools:

Statistics

Individual	Role	Organization	Contact Information
Emad Yanni, Phd Senior Service Fellow	Consulted on sampling methods and study design	DGMQ, IRMH	[email protected]
Clive Brown, MBBS Associate Director of Science	Consulted on sampling methods and study design	DGMQ, OD	[email protected]

Data collection, and analysis

Individuals/Titles	Role	Organization	Contact Information
Erica Felker-Kantor, MA, MSPH Research and Evaluation Fellow	Principal Investigator	Chenega Government Consulting, DGMQ, IRMH	[email protected]
Erika Willacy, MPH Lead Health Communication Specialist	Co-Principal Investigator	DGMQ, IRMH	[email protected]
Stefanie Erskine, MPH Communication Specialist	Consulted on design of data collection tools	Chenega Government Consulting, DGMQ, Traveler’s Health	[email protected]
Gaby Benenson, MPH Senior Health Communication Specialist	Consulted entire protocol and the design of the data collection tools	DGMQ, OD	[email protected]

REFERENCES
1. Eysenck MW. A handbook of cognitive psychology. London: Lawrence Erlbaum Associates, Ltd; 1984.

ATTACHMENTS

Note: Attachments are included as separate files.

Attachment A-1—Pre-Knowledge Aptitude Survey

Attachment A-2—Semi-Structured Questionnaire

Attachment A-3—In-Depth Interview Guide