0990PartB _ EIVRI study

OMB Section B Statistical Methodology for EIVRI Study

B. Collection of Information Employing Statistical Methods If statistical methods will not be used to select respondents and item 17 on Form 83-I is checked “No” use this section to describe data collection procedures.

1. Respondent Universe and Sampling Methods

The universe from which the study participants will be drawn will be the email list of NIH grant recipients held by the Office of Research Integrity. This list comprises the universe of study subjects, i.e. all approximately 40,000 individuals in any field of study who have received NIH research funds in the past 10 years. This list consists of Research Integrity Officers (RIOs), instructors, lecturers, research administrators, principal investigators (PIs) and others who haverequested funding from NIH. For the purposed of this evaluation study, this group is the respondent universe.

We plan to send an email request to participate to all individuals on this list. There may be some errors. Some individuals will have moved or retired, some will have invalid email addresses. Moreover, we do not anticipate the response rate to be particularly high. However, by inviting all 40,000 individuals, we anticipate receiving sufficient subgroups, (e.g. by gender, race/ethnicity and age), so that we will be able to conduct subgroup analyses. No other similar studies have been conducted to investigate the effectiveness of the EIVRI to the ORI’s knowledge.

2. Procedures for the Collection of Information

A questionnaire has been developed which will collect data on two areas: 1. Demographics of the respondent and 2. Their satisfaction and/or intent to use the DVD in future training sessions. This questionnaire will be developed using on Survey Monkey and sent to all invitees, as described in section 1 above. Any non-responses will be reminded three times to respond to the survey. This should ensure the maximum possible response rate.

Using Survey Monkey as a tool for data collection has several advantages over other types of data collection methods:

1. It enables the survey designer to create complex surveys, which include skip patterns, required answers, multiple or single check-offs and open-ended questions in text boxes;

2. It allows the designer to enter email addresses and keeps track of who responds and who doesn’t; this ensures that only non-respondents are re-contacted for follow-up;

3. It allows for data to be downloaded to statistical packages, such as SPSS, for data cleaning and analysis.

We anticipate that many of the respondents will have already viewed at least one portion of the DVD. After approximately 1,000 surveys have been completed, we will determine whether the viewing of each module is approximately equal. If the viewing is heavily weighted to one module, then a random assignment to one of the remaining three modules will be added, to ensure equal representation and evaluation of each module.

The study will use an alpha of 5% for the test accuracy and a regression formula to compare the results.

3. Methods to Maximize Response Rates and Deal with Non-response

Our goal for this web based survey evaluation is to discover whether: 1. Respondents are satisfied with the content of the DVD and 2. They will use the DVD in their research ethics or research integrity classes.

After approximately two weeks, respondents who do not reply to the first email invitation will be sent a second email reminder. After approximately one month, non-respondents to the first two requests will be sent a thirdreminder, along with a final date at which no data will be accepted.

Since we anticipate sending the invitation to participate to approximately 40,000 individuals, even a very low response rate (e.g. 15%) will yield enough surveys to conduct overall and subanalyses.

4. Tests of Procedures or Methods to be Undertaken

Pilot test: The outcome measures for this study will be pilot tested before being used in data collection. We have requested nine interested parties to pre-test the evaluation questions (for the pilot study). From their responses, we will obtain feedback on the overall design of the questionnaire, as well as potential questions that should be included in the questionnaire. Please see a copy of the questionnaire (see Appendix A).The data used for this study will be collected from electronic questionnaires. The independent variables are the subject demographics. The dependent variables will include the satisfaction and use questions. We will use both qualitative and quantitative analyses for this study. The proposed quantitative analysis will consist of univariate descriptive statistics of the demographic variables and the individual items for each of the satisfaction and use questions. Some stratified analysis will be completed with key demographic variables, such as race, sex and instructor level. Multivariate analyses will include analyses on satisfaction scales, using demographic variables as control items. The satisfaction and use questions will be taken from validated scales, but their use in this type of survey has not been validated. Thus, factor analysis will be done on the satisfaction items to identify whether these questions may be asking about the same general concepts or whether there are additional concepts that were not anticipated. The open-ended questions will allow some qualitative analysis. The analyses of this data will also be included in the final report.

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or

Analyzing Data

Statistician:

Lynn D. Disney, Ph.D., J.D., M.P.H., Senior Research Associate, CSR, Incorporated.

2