3145-NEW ADVANCE Information Collection Request
II. Section B
B.1. Respondent Universe and Sampling Methods
Telephone interviews will be conducted with the ADVANCE Project Directors at all IT grantee institutions (N=19) in the cohorts of interest and hence no sampling methods are needed.
Case study site selection will involve purposeful sampling. Two examples of each of the major program models identified from the telephone interview data and document review will be chosen as the subjects for case study. It is anticipated that up to three models may be identified. The six case study sites will be chosen based on pre-established criteria and, within each model type, the researchers will attempt to select institutions that vary in terms of size, governance, Carnegie classification, and other characteristics.
B.2. Information Collection Procedures/Limitations of the Study
This request for clearance includes two types of data collection—telephone interviews and case studies (site visit interviews and focus groups).
The collection will be based on protocols developed for this evaluation. The development of these protocols—for telephone and in-person interviews and focus groups—will be informed by a review of related research, relevant theoretical literature, project documents and, in the case of site visit protocols, results from the telephone interviews.
Telephone Interviews. After a letter of introduction informing grantees of the interviews and requesting cooperation, interviewees will be contacted to schedule interviews. A team of four interviewers will carry out the telephone interviews, which will take approximately one hour to one hour and fifteen minutes each. The team of expert interviewers will undergo training designed to familiarize them with the goals of the evaluation, the purpose of the interviews, the instruments, the specific goals of each question, the method for data entry of interview data, and security measures to follow. Interviews will be confidential and will be tape recorded if permission to record is granted by the informant.
Site Visits. Each site selected for case study will be contacted to seek participation. After obtaining agreement from the sites, evaluation staff will work with a representative at each of the case study sites to identify prospective participants and to coordinate interviews and focus groups that make up the data collection activities of the site visit. Teams of three experienced site visitors will conduct two-day visits to each case study site. Site visitors will undergo training to brief them on the purpose of the site visits and to familiarize them with the data collection instruments. Site visitors will be provided with background information on each of their sites, including the telephone interview data, proposals and other relevant reports, and institutional background data.
B.3. Methods for Maximizing the Response Rate and Addressing Issues of Nonresponse
This study does not include survey questionnaires, but does include telephone interviews with the project directors at each of the nineteen ADVANCE projects funded by NSF in 2001 and 2002, as well as case studies at six sites. Experience with prior evaluations suggests that we should not encounter a participation problem.
Nonetheless, we will engage in activities that, in the past, have helped increase participation:
● Encourage study participation by making a presentation at the annual JAM meeting so that project staff understand the importance of the evaluation and have an opportunity to ask questions and raise concerns;
(this activity was conducted in June 2009)
● Procure from NSF ADVANCE program directors a letter of introduction informing grantees of the interviews and/or case study site visits, and requesting cooperation prior to contacting interviewees for a telephone interview or to schedule site visits;
● Allow sufficient time for data collection. For example, telephone interviews will be carried out over the course of two months to ensure that busy schedules of respondents can be accommodated;
● Design protocols to target the specific information that each respondent group is likely to possess.
B.4. Tests of Procedures or Methods
We field tested the interview protocol with directors/principal investigators at two different ADVANCE sites that, according to documents reviewed, were employing strategies commonly found at ADVANCE projects (such as recruitment and policy revisions). As expected, both interviews lasted about one hour (one slightly less). Because two questions did not have to be administered as neither site had PAID or a Leadership awards, we estimate that the few interviews conducted at sites that have these awards could last slightly more than one hour. We took this into account in our burden estimates.
After completing the interview, we asked respondents to comment on the clarity and wording of the questions. Evaluation project staff reviewed and discussed the information gathered during the field tests and subsequently used results to revise the attached interview protocol. Revisions included the addition of a question, clarification of instructions to interviewers, and reordering of two questions.
Once site visit instruments are developed they will be similarly field-tested and sent to OMB to be appended to this application.
B.5. Names and Telephone Numbers of Individuals Consulted
Agency Unit:
Suzanne Plimpton, Reports Clearance Officer Information Dissemination Branch, National Science Foundation, 703.292.7556.
Jessie DeAro, Program Director ADVANCE, National Science Foundation, 703.292.5350.
Kelley Mack, Program Director ADVANCE, National Science Foundation, 703.292.8575.
Elmima Johnson, Division of Research, Evaluation, and Communication, COTR National Science Foundation, 703.292.5137.
Bernice Anderson, Office of the Assistant Director, National Science Foundation, EHR, 703.292.5151.
Contractor or Grantee:
Clemencia Cosentino de Cohen, Co-PI, ADVANCE Evaluation, Evaluation Studies and Equity Research Program (PEER), The Urban Institute, 202.261.5409.
Beatriz Chu Clewell, Co-PI, ADVANCE Evaluation, Evaluation Studies and Equity Research Program (PEER), The Urban Institute, 202.261.5617.
The Urban Institute will be responsible for the evaluation design, data collection and analyses under the direction of the Co-PIs of this evaluation, Beatriz Chu Clewell (202.261.5617) and Clemencia Cosentino (202.261.5409).
ATTACHMENTS
Telephone Interview Protocol
Draft Correspondence and Introductions
| File Type | application/msword |
| File Title | 3145-NEW ADVANCE Information Collection Request |
| Author | nsfuser |
| Last Modified By | nsfuser |
| File Modified | 2009-07-21 |
| File Created | 2009-07-21 |