Supporting Statement Part B

Supporting Statement B

for

Office of Inspector General Review of Awardee Implementation of NSF’s Requirement for a Responsible Conduct of Research Program

Name: Dr. Aaron Manka

Address: Office of Inspector General, National Science Foundation, Suite II-705, 4201 Wilson Blvd, Arlington, VA, 22230

Email: [email protected]

Telephone: 703-292-5002

Fax: 703-282-9159

Contents

B.1 Respondent Universe and Sampling Methods 4

B. 2. Procedures for the Collection of Information 5

B.3. Methods to Maximize Response Rates 5

B.4. Test of Procedures or Methods to be Undertaken 6

B.5. Individuals Collecting and/or Analyzing Data 6

List of attachments

Attachment 1: Sample introductory letter

Attachment 2: RCR Survey for grantee administrators

Attachment 3: RCR Survey for RCR program administrators

Attachment 4: RCR Survey for trainees

B.1 Respondent Universe and Sampling Methods

There are three categories of individuals from whom information will be collected: 1) upper-level administrators (e.g., Vice Presidents), 2) program administrators (e.g., Research Integrity Officers), and 3) trainees who have participated in the RCR program (undergraduate students, graduate students, and postdoctoral researchers). Estimates of the respondent universe for each category and anticipated number of participants are summarized in Table B.1.

Table B.1. Estimated Respondent Universe and Use of Sampling Methods by Category (estimate for data collection over two years)

Category of participant	Type of data collection proposed	Estimated size of respondent universe	Expected response rate
Pre-interview information respondee	Mail/email	100 over 2 years	93%0
Senior-level administrator	interview	100 over 2 years	93%
RCR administrator	interview	100 over 2 years	93%
Students and postdocs	interview	Minimum of 300 (>300) over 2 years	93%

Two groups of individuals playing critical roles in the administration of NSF’s requirement for a Responsible Conduct of Research (“RCR”) Program: a senior-level administrator and an RCR administrator have been identified as key participants. Speaking with each of them will be essential to understanding the structure and goals of an awardee’s implementation of, and commitment to, the awardee’s RCR Program. Both should be able to provide information on the awardee’s satisfaction with NSF’s RCR requirement and approach. Specifically, we expect awardees to relate their opinions about the strength and weaknesses of NSF’s approach to RCR.

The same is true of each awardee’s affected students or postdocs—each should be able to describe the benefits and drawbacks of the RCR course. Information from students and postdocs is critical to understanding the value, strengths, and weaknesses of the RCR program as a whole.

A survey will be conducted of 100 NSF awardee institutions selected for the survey to ensure a variety of representation, e.g., public and private; and small, medium, and large, as determined by student population. Sampling is feasible to reduce burden because it is not practical to survey every NSF awardee (over 3,000). A high response rate is anticipated; we will ask a total of 100 participating grantees, but need a minimum of 93 (see fn. 1), allowing for some grantees opting not to participate (up to 7%). Follow up interviews (for clarification or to elaborate on particular points) may be conducted, as needed. Interviewees will be selected purposefully as the survey results are tallied in order to provide complementary information and answer any questions that arise.

B. 2. Procedures for the Collection of Information

B.2.1 Data Collection Procedures

We will initially contact the grantee via telephone to describe our RCR project and request its participation. Within one week, we will follow-up with a letter requesting the grantee’s RCR plan (if it is not available online), the number of participants in its RCR program, and information about the program (a sample letter is Attachment 1). We will begin scheduling interviews via email. Structured interviews will be conducted with respondents via telephone and/or video conferencing. Informed consent will be obtained orally at the beginning of each interview. Interviews will be conducted by OIG staff with experience interviewing administrators and students. Discussion templates for these interviews are included as Attachments 2-4 (senior administrators, RCR administrators, and trainees, respectively). The interview survey consists of approximately 17 free-response questions (see Attachments 2-4). We estimate it will require approximately 0.5 hours to complete the interview with the senior administrator, 1.5 hours for the RCR program administrator(s), and 1 hour collectively for the trainees (undergraduate students, graduate students, and post-docs).

If necessary, we will conduct follow-up interviews with particular survey respondents via telephone if it is determined that clarification or additional information is needed. Procedures will be the same as those described above for interviews.

B.2.2 Data Analysis Procedures

Interview data will be analyzed using simple assessment, tabulation, and counting. For privacy considerations, the interviews will not be recorded so we will rely on detailed notes, rather than transcripts. Interview notes will be coded by RCR themes, e.g., support, structure, format, and compliance, and analyzed by staff members with experience in qualitative analysis.

B.3. Methods to Maximize Response Rates

NSF OIG will employ methods found to be generally effective in maximizing response rates. These include:

1. Calling the grantee to inform them of the nature and scope of our project.

2. Sending an introductory letter from the NSF OIG staff to the survey population requesting documentation of the grantee’s plan and cooperation with our survey (see Attachment 1).

3. Monitoring responses continuously and contacting non-respondents periodically with individualized reminders.

B.4. Test of Procedures or Methods to be Undertaken

As the respondent group for each data collection instrument is small, a formal pretest of the instruments will not be undertaken before they are fielded. However, all instruments have been reviewed by NSF OIG staff, including Investigative Scientists with backgrounds similar to the target respondents and experts in evaluation methods.

B.5. Individuals Collecting and/or Analyzing Data

This study was designed in consultation with OIG management and external consultants who are recognized as RCR experts. NSF OIG Investigative Scientists will have primary responsibility for data collection and analysis. The point of contact is Dr. Manka:

Aaron Manka, Ph.D.

Senior Investigative Scientist

Office of Inspector General,

National Science Foundation, Suite II-705

4201 Wilson Blvd.

Arlington, VA 22230

Phone: 703-292-5002

Email: [email protected]

0 We calculated a sample size of 93 surveys would provide a 95% confidence level with a 10% confidence interval (margin of error).

3