Supporting Statement for the
Paperwork Reduction Act Submission
Part B: Justification
National Institutes of Health
A Process Evaluation of the Global Health Research Initiative Program for New Foreign Investigators (GRIP)
This request seeks approval of a data collection related to the Fogarty International Center’s (FIC) Global Health Research Initiative Program for New Foreign Investigators (GRIP) Process Evaluation. The data collection consists of one online survey to be administered to all awardees and a sample of applicants of the GRIP program. The information gathered from this survey will document the GRIP program operations and outputs and will be used as a guide to the program officers in their future strategic and management decisions.
Part B. Collection of Information Employing Statistical Methods
A census will be conducted of all 56 GRIP award recipients for the initial FY 2002 Request for Applications (RFA) and subsequent Program Announcements (PAs) from FY 2003 to FY 2006. An additional 45 high-scoring applicants who were not awarded GRIP funding from FY2002-FY2007 will also be included in the survey. A census is requested rather than a sample because of the program’s relatively small number of award recipients. Sampling would result in the number of respondents being too small to permit comparative analyses, as well as preclude analyses of factors that influence various program outputs.
B.2.1 Collection Procedures
At the outset of the survey process, an email from the GRIP Program Officer or other FIC officers will contact all participants to describe the purpose of the evaluation, the procedures being used, a request that they participate in the survey, as well as a request to confirm general demographic information. They will also introduce the Science & Technology Policy Institute (STPI), the Federally-Funded Research and Development Center serving as the contractor for this evaluation. To review drafts of this and other anticipated formal correspondence with the survey participants, please consult Attachments 4 and 5 to this document.
Immediately following this initial communication, researchers from the Science & Technology Policy Institute will contact all participants with information on how to access and complete the online survey. Respondents will be allowed 4 weeks to complete the survey. In order to facilitate completion, individuals will be allowed to either fill out the survey in one sitting or pause and return later to finalize the survey. The survey will be organized by topic (e.g., Background, Mentoring, Research) and participants will be allowed to move from section to section via a central navigation interface.
Once the survey is underway, STPI will monitor the status of the submissions (e.g. Never Logged In, Not Started, In Progress, and Complete). After one week, a reminder will be sent to individuals who have never logged in. Individuals who have initiated but have not yet completed the survey will receive a note encouraging them to complete their survey. Both groups will be offered an electronic or hard-copy of the survey if it is inconvenient to complete it online. All persons who have completed the survey will be sent a thank you note. A second reminder email will be sent one week after the first (and about two weeks after the start of the survey process). All new respondents will be thanked at that time, and the persons who had started the survey will be encouraged to complete it. The survey will be closed at the end of week four. The collection timeline for the survey is outlined in Table B.2.1
The survey process is expected to take about 5 weeks (Table B.2.1). Those who do not wish to participate will be classified as non-respondents. When the survey is complete, a non-respondent analysis will be performed by demographics and other characteristics.
Table B.2.1 Anticipated Collection Procedures and Timeline
Week |
Applicant Survey |
1 |
Email introduction by FIC; survey begins (deadline end of week 3) |
2 |
First reminder to non-respondents Encouragement to complete the survey; mailing of hard-copy survey; thank you note to respondents |
3 |
Second reminder to non-respondents Encouragement to complete the survey; mailing of hard-copy survey; thank you note to respondents |
4 |
Survey ends; thank you note to respondents |
This study is being conducted with the aim of understanding applicant and awardee characteristics, activities, and outputs of interest in the GRIP program Evaluation. Analysis of quantitative or semi-quantitative survey data will be primarily descriptive, involving measures or estimates of frequency, central tendency (mean, median, or mode), dispersion (variance or standard deviation), and range. Examples include calculating the mean age of applicants or determining what fraction of awardees are women. To facilitate analysis, some data will be stratified by sociodemographic characteristics, such as age, gender, affiliation, etc., in order to gain a better understanding of the distribution of the data, and to see if any trends exist across strata. Where there are several factors of interest that need to be considered simultaneously, multivariate statistical techniques may be used. Selection of multivariate analysis models will be guided by the GRIP study questions and final models will be selected based on the results of the initial descriptive analyses.
Qualitative data from the open-ended questions will be coded and analyzed using standard qualitative methods. The contractor will decide whether use of specialized software packages for qualitative data analysis appears to be justified by the volume of data.
STPI staff members will conduct all analyses. Once the analyses are complete, STPI will prepare a thorough documentation of the analyses, findings and conclusions, and suggest specific recommendations regarding program activities, accomplishments, and management.
STPI will employ methods found to be generally effective in maximizing response rates; these methods include but are not limited to:
Sending an invitation letter from the FIC program officer to the survey population. Especially for the awardee sub-population, receiving the initial communication is expected to encourage response.
Monitoring web-page responses, and contact non-respondents by email to both solicit their cooperation. These procedures are described in detail in section B.2.
In cases where participants do not have reliable access to the Internet, providing copies of the survey via email or hard copy.
Allowing respondents to navigate from section to section and return to partially completed surveys at a later time.
Conducting web searches to update contact information for any targeted participants for whom a valid email address does not exist in NIH records.
Pre-loading any information that is already available through NIH records so that respondents may correct the data rather than re-entering them from scratch.
Where possible, automatically omitting questions that do not pertain to known subgroups.
Designating a few key questions in each section, asking those questions in earlier in the survey section and strongly encouraging response to at least those questions, even if respondents cannot complete the full survey. It is anticipated that this prioritization and flexibility will contribute to higher response rates.
The target response rate for the survey has been set at 80% of both awardees and unsuccessful applicants. After initiation of the survey, non-respondents will be contacted by mail or by phone in order to solicit or confirm their participation. For the unsuccessful applicants, while the target response rate is 80%, extraordinary efforts beyond what is detailed above will not be undertaken to ensure participation. One goals of the process evaluation is to assess the feasibility of including comparison groups in a future outcome evaluation; non-response by the unsuccessful applicants would suggest that this group would not be a feasible comparison group.
B.4. Test of Procedures or Methods to be Undertaken
As the respondent group for this survey is small, a formal pretest of the instrument will not be undertaken, but FIC staff members and others will test the online survey instrument before it is fielded. Previous studies of similar foreign scientist respondent groups by the contractor have employed methods outlined in B.3 successfully, resulting in high returns and minimum burden on participants.
B.5. Individual Consulted on Statistical Aspects and Individual Collecting and/or Analyzing Data
Several individuals from STPI and FIC participated in the development of the study plan. The team included persons with knowledge of statistical methods, experience in evaluation of science programs, and expertise in scientific research. Dr. Brian Zuckerman is Project Director of the GRIP process evaluation and the point of contact for this project; he has extensive evaluation experience with NIH and other Federal agencies. Dr. Zuckerman and qualified team members at STPI will conduct all data collection and follow-up. He can be contacted at:
Brian Zuckerman
Science & Technology Policy Institute
1899 Pennsylvania Avenue NW, Suite 520
Washington DC 20006
Telephone: (202) 419-5485
Facsimile: (202) 223-1695
E-mail: [email protected]
| File Type | application/msword |
| File Title | NDPA Proc Eval 30 day Fed Reg Supporting Statement |
| Author | STPI |
| Last Modified By | Allison Hodges |
| File Modified | 2008-03-03 |
| File Created | 2008-03-03 |