Supporting Statement B for
Assessing the Long-term impacts of research and training programs supported by the John E. Fogarty International Center (FIC)
Date: August 18, 2010
Name: Rachel Sturke, PhD
Address: Fogarty International Center, 31 Center Drive - MSC 2220, Bethesda, MD 20892-2220
Telephone: 301.480.6025
Fax:
Email: [email protected]
B.1 Respondent Universe and Sampling Methods 3
B. 2. Procedures for the Collection of Information 3
B.2.2 Data Analysis Procedures 3
B.3. Methods to Maximize Response Rates 3
B.4. Test of Procedures or Methods to be Undertaken 3
B.5. Individual Consulted on Statistical Aspects and Individual Collecting and/or Analyzing Data 3
List of attachments
Attachment 1: Consent Form for Participants
Attachment 2: Estimate of hourly rate for foreign researchers
Attachment 3: Interview Guide for US-Based Principal Investigators
Attachment 4: Case Study Questionnaire for FIC Trainees
Attachment 5: Focus Group Discussion Guide for Trainees
Attachment 6: Interview Discussion Guide for Trainees (University-Based)
Attachment 7: Interview Discussion Guide for Trainees (Non University-Based)
Attachment 8: Interview Discussion Guide for University Leaders
Attachment 9: Interview Discussion Guide for Policy-Makers and Scientific Leaders
Attachment 10: Conceptual Framework
Attachment 11: Research Questions
For each case study institution, the universe of potential informants consists of all individuals who have participated in or have knowledge of FIC research training programs. Specifically, the case study design includes five categories of informants: 1) US-based principal investigators on research training awards partnering with case study institutions; 2) current and former trainees; 3) leaders and administrators at case-study institutions; 4) direct FIC grantees at case study institutions; and 5) policy-makers and/or scientific leaders in case study institution countries. Estimates of the respondent universe for each category and anticipated number of participants are summarized in Table B.1.1.
Table B.1.1. Estimated Respondent Universe and Use of Sampling Methods by Category
Category of Participant |
Estimated Size of Respondent Universe per Case Study |
Method of Estimation |
Expected number of Participants per Case Study |
Sampling Methods Used to Reduce Burden? |
1) US-based principal investigators on research training awards partnering with case study institutions |
20 |
Based on NIH records of number of awards to US PIs at first two case study institutions |
20 |
No |
2) former trainees |
300-600 |
Estimate based on FIC records of trainees associated with first two case study institutions |
40 (focus groups) 13 (interviews) |
Yes |
3) leaders and administrators at case-study institutions |
unknown |
|
4 |
Yes |
4) direct FIC grantees at case study institutions |
20 |
Based on NIH records of number of awards to foreign PIs at first two case study institutions |
20 |
No |
5) policy-makers and/or scientific leaders in case study institution countries |
unknown |
|
8 |
Yes |
For categories 1 and 4 (US-based PIs and foreign PIs), sampling methods have not been used to reduce burden because each PI is expected to have a unique experiences to share that are specific to his or her particular awards. An understanding of the role played by each award will be essential to the case studies.
For categories 3 and 5 (university leaders and policy-makers or scientific leaders), the universe of respondents with potentially relevant knowledge is difficult to estimate. Snowball sampling methods will be used to identify individuals with knowledge of particular impacts specific to each case.
For category 2 (former trainees), the universe of potential respondents is quite large, and sampling methods will be used to reduce burden. Specifically, trainees listed in the FIC database of trainees (CareerTrac) will be stratified by the type of training program in which they participated. Trainees from a representative sample of training programs relevant to the case study institution will then be randomly selected to participate in focus groups (4 per case consisting of 10 individuals each) and interviews (13 per case).
B.2.1 Data Collection Procedures
Semi-structured interviews will be conducted with Principal Investigators on FIC grants with links to the case study institutions via telephone prior to the site visit. A discussion guide for this data collection activity is included as Attachment 3.
During the in-country site visit, the study team will use in-depth semi-structured interviews with the various categories of respondents described above, plus a short structured survey of and focus groups with FIC trainees. The specific data collection instruments to be used include the following:
A short, structured survey (Attachment 4) to be be administered to those FIC trainees who participate in focus groups, before the start of the focus group. This survey was designed based on previously administered research instruments found in the literature and is targeted around several specific areas, such as: FIC trainee’s job satisfaction, motivation to conduct research, and social networks. The survey will require approximately 30 minutes of a participant’s time.
Semi-structured focus groups with former FIC trainees in order to gain an understanding of shared experiences and organizational norms as related to research and training experiences. Focus groups will be facilitated by experienced study leaders who are knowledgeable about best practices for this data collection method. It is anticipated that these discussions will also be used as a foundation to inform and adapt the in-depth interviews. Focus groups will require approximately 90 minutes of each participant’ time. The focus group discussion guide is included as Attachment 5.
Semi-structured interviews with various groups of informants that will be used to gain a thorough understanding of participant perceptions and individual experiences as related to the FIC program. Interviews will be conducted in a location to be arranged by a collaborator at the case study institution and will require approximately 60 minutes of a participant’s time. Interview guides have been developed for each of the following groups:
Former FIC trainees currently employed in university settings (Attachment 6);
Former FIC trainees not currently employed in university settings (Attachment 7);
Leaders and administrators at case study institutions (Attachment 8); and
Policy-makers/scientific leaders in the countries of interest (Attachment 9).
The data will be analyzed upon return to the US, although preliminary impressions from the data collection will be shared with university leadership prior to departure from the country. Draft findings will be shared with FIC colleagues and the expert panel. The study team will maintain confidentiality of study participants and will not use personal identifiers for any of the information shared or opinions stated, instead respondents will be referred to according to which category they fall into (eg. trainee, university leader, policy maker).
Due to budget and time constraints transcripts will most probably not be made of all qualitative data (although we are looking into the costs of making full transcripts). Rather detailed notes (both handwritten and computer based) will be taken during the interview, and the interviews will be recorded. From these sources a final detailed account of the interview using selected quotations where appropriate will be made. Qualitative data will be analyzed using a software package such as Atlas TI or NVivo.
The data analysis will be centered around the constructs identified in the conceptual framework (Attachment 10) and related research questions (Attachment 11). The study team will use direct interpretation to reorganize the data gathered around the constructs in the conceptual framework. Along this process, the study team will establish patterns between two or more conceptual constructs and experiences from those interviewed.
Quantitative data from the trainee questionnaire will be entered into a database and analyzed using a software package such as Stata, SAS or SPSS. The analysis will include both a descriptive analysis of the career paths of FIC trainees (encompassing for example, their success in attracting additional research grants, and in publishing, and the self-perceived impact of FIC training) but will also seek to analyze how environmental factors, such as how the availability of resources to support their work, and adequacy of remuneration have intervened to affect their productivity and satisfaction.
Data regarding professional collaborations and contacts (section 5 of the case study questionnaire) will be analyzed using social network analysis in order to explore the strength of linkages between those involved in FIC programs, and between FIC trainees and researchers trained through other mechanisms. This analysis will employ software such as NodeXL and will seek to document the extent to which FIC support has led to the development of identifiable professional networks.
Internal validity checks will be carried out through collecting input from the in-country research collaborator, debriefing with University leadership and key informant interviews. Discussions with US based PIs for the respective countries and review of FIC evaluations of study country programs will also be used to validate the findings for each of the case studies. The full internal validity will be limited as the case studies have not been selected through randomization.
External validity checks will be carried out through discussions with the FIC-convened expert panel.
As the proposed data collection is a series of case studies rather than a true statistical survey, response rates will not be calculated. However, the site visits themselves are viewed as a method intended to maximize participation; FIC has learned from experience that, while individuals who have benefited from its international training programs are generally willing and often eager to discuss their experiences, it can be difficult to collect information from individuals in foreign countries via telephone interviews or online surveys. To further maximize participation, the study will employ an in-country collaborator at each institution who will work closely with the study team on logistics, coordination, and recruiting of participants. It is believed that a local collaborator who is familiar with the local language, customs, and institutional culture will have the best chance of maximizing participation.
As the respondent group for each data collection instrument is small, a formal pretest of the instruments will not be undertaken before they are fielded. However, the first case study will essentially serve as a pilot for the rest of the study. Methods will be reviewed carefully after the first site visit and adjusted as necessary.
The design for this study was developed primarily by the contractor, Dr. Sara Bennett, an Associate Professor at the Johns Hopkins Bloomberg School of Public Health. Several individuals at FIC also participated in the development of the study plan. Dr. Bennett and her graduate assistants will have primary responsibility for data collection and analysis. Dr. Bennett’s contact information is:
Sara Bennett
Associate Professor, Department of International Health
Johns Hopkins Bloomberg School of Public Health
Telephone: +1 410 614 5137
E-mail: [email protected]
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | NDPA Proc Eval 30 day Fed Reg Supporting Statement |
| Author | STPI |
| File Modified | 0000-00-00 |
| File Created | 2021-02-02 |