Supporting Statement B NINR_SGI_v4

Supporting Statement B for:

National Institute of Nursing Research (NINR)
Summer Genetics Institute Alumni Survey

Amanda Greene, PhD, MPH, RN

National Institute of Nursing Research (NINR)

6701 Democracy Blvd, Suite 700/703E

One Democracy Plaza

Bethesda, MD 20892

Telephone: 301-496-9601

Fax: 301-480-8260

Email: [email protected]

Table of Contents

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS 1

B.1 Respondent Universe and Sampling Methods 1

B.2 Procedures for the Collection of Information 1

B.3 Methods to Maximize Response Rates and Deal with Nonresponse 2

B.4 Test of Procedures or Methods to be Undertaken 3

B.5 Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data 3

List of Attachments

Attachment 3: Introductory, Invitation, Reminder, and Thank you emails

Attachment 4: Informed Consent

Attachment 7: Summer Genetics Institute pilot survey analysis

B. Statistical Methods

This supporting statement addresses data collection procedures related to the evaluation of the National Institute of Nursing Research (NINR) Summer Genetics Institute (SGI) Alumni Survey. This section will describe in detail the various aspects of the questionnaire data collection methods.

B.1 Respondent Universe and Sampling Methods

Data will be collected annually from all participants (hereafter referred to as alumni) of the annual NINR Summer Genetics Institute which started in 2000. As of July 1, 2011, 230 individuals have participated in the Summer Genetics Institute. The potential survey respondents will be identified from the Summer Genetics Institute participant database roster.

This survey is voluntary. A response rate of 50 percent is expected because over half of the alumni have applied for research and training grants at the National Institutes of Health since attending Summer Genetics Institute. In addition, most Summer Genetics Institute alumni have self-identified as researchers in the application to participate in the Summer Genetics Institute.

B.2 Procedures for the Collection of Information

a. Rationale for Surveying All Respondents of a Given Type

As of July 1, 2011, 230 individuals have participated in the Summer Genetics Institute. The number of participants in future Institutes is limited to less than 30 per year. The number of expected respondents – 115 alumni – is based on a target response of 50 percent. This response rate is expected because over half of the alumni have applied for research and training grants at the National Institutes of Health since attending Summer Genetics Institute. In addition, most Summer Genetics Institute alumni have self-identified as researchers in the application to participate in the Summer Genetics Institute. Given the small number, surveying all alumni is indicated in order to detect any trends.

a. Questionnaire Procedures

An online survey will be used to collect information from Summer Genetics Institute alumni.

Personal identity of potential respondents will be protected by assigning an ID number to each response so that the respondent’s name and any other personal information will not be directly linked with their response. The identity of respondents will not be released. Identifying information of respondents will be stored separately from the questionnaire responses. All collected survey information will be analyzed and reported in aggregate.

Communication with Alumni. Communication with Summer Genetics Institute alumni about the survey will be conducted via email as described below.

1. An email notification letter (see Attachment 3) will be sent to Summer Genetics Institute alumni. The notification letter will inform the Summer Genetics Institute alumni about the upcoming survey. The letter will indicate that the survey is sponsored by the National Institute of Nursing Research of the National Institutes of Health (NIH).

2. An email invitation letter (see Attachment 3) will be sent to Summer Genetics Institute alumni one (1) week after the email notification letter is sent. This letter will inform the Summer Genetics Institute alumni about the survey and how to participant in the survey. The letter will indicate that the survey is sponsored by the National Institute of Nursing Research of the National Institutes of Health (NIH). Specifically, the letter will:

• Briefly describe the Summer Genetics Institute Alumni Survey, the purpose of the Survey, and the importance of respondents’ participation in the survey.

• Inform participants that completing the Summer Genetics Institute Alumni Survey online will on average take about 30 minutes.

• Specify that the identity of respondents will not be released. Identifying information will be stored separately from the questionnaire responses. All collected information will be analyzed in aggregate.

• Include a URL address that will take participants directly to a secure Website with the questionnaire.

• Provide contact information in case respondents have any questions about the study.

• Be sent from an NIH project email address at [email protected]... Email replies and non-deliverable emails will be sent to this NIH email address.

• Dr. Amanda Greene, NINR science evaluation officer, will sign the email invitation letter.

3. One (1) week after the initial email announcement, a reminder email (Attachment 3) will be sent to non-respondents. Potential participants will experience no burden if they choose not to respond.

Informed Consent. When potential participants access the secure, questionnaire Web site, they will be routed to the informed consent form. The consent form may be found in Attachment 3. In addition to being informed about the Summer Genetics Institute Alumni Survey and the purpose of this survey, potential participants will be provided with more details regarding their rights as participants in the study. Specifically, the consent form will:

• Describe the process to ensure privacy of the information provided by participants.

• Explain how participants’ identity and anonymity will be protected.

• Explain that participation is voluntary and that respondents can stop participation at any time.

• Describe how the data collected will be protected and safeguarded.

• Provide contact information in case respondents have any questions about the study.

Potential participants will be prompted to accept or decline participation by selecting the appropriate button at the bottom of the electronic form. Only if participants click on the “Yes, I accept” button, they will be routed to the first question in the questionnaire.

Survey Data Collection. Respondents will be directed to a secure, web-based questionnaire. Respondents will be assigned an ID number. Personal identifying information will be stored separately from responses. Responses to questions will be collected in an electronic database that can be analyzed using statistical software such as Excel, SPSS, STATA, or SAS. Data files will be kept in a secure environment and no one outside of this study will have access to them.

B.3 Methods to Maximize Response Rates and Deal with Nonresponse

Several procedures will be implemented to maximize the response rate for the Summer Genetics Institute Alumni Survey. The survey has been designed for maximum ease of administration, and the data collection protocol has been tailored to acknowledge respondents’ contributions. The presentation of the survey has also been taken into consideration so that respondents can differentiate it from other mail and research requests.

A notification email (see attachment 3) will announce that a web-based survey of Summer Genetics Institute alumni will be conducted. One week later, an email introductory letter (see attachment 3) with the link to the survey will indicate that it is sponsored by NINR. The email will succinctly inform readers of the importance of the questionnaire, as well as procedures for maintaining the privacy of the respondents. The email will be sent from an NIH address (i.e., [email protected]).

Online administration of the questionnaire is expected to greatly increase the ease of data collection for these alumni. The strategy for email follow-up for those who have not responded to the email request has been carefully designed. By following up consistently and persistently, NINR will demonstrate that it is committing time and energy to obtain the most valid data possible by information from as many alumni as possible.

Consistent with the response rate calculations approved by the American Association for Public Opinion Research (AAPOR), response rates for this study will be calculated as follows:

Number of Completed Questionnaires

Number of Completed Questionnaires + Number of Nonrespondents

Although the target respondent group for year one of the survey consists of 230 participants, the number of expected respondents is 115. The number of expected respondents – 115 participants – is based on a target response of 50 percent. This response rate is expected because over half of the alumni have applied for research and training grants at the National Institutes of Health since attending Summer Genetics Institute. In addition, most Summer Genetics Institute alumni have self-identified as researchers in the application to participate in the Summer Genetics Institute.

B.4 Test of Procedures or Methods to be Undertaken

The Summer Genetics Institute Alumni Survey was programmed for Web-based administration and tested by Discovery Logic (a Thomson Reuters business) staff before pilot-testing. A pilot-test of the Summer Genetics Institute Alumni Survey by five (5) alumni using a preliminary instrument that contained draft items for the final survey was conducted to ensure that useful and relevant information was gathered. Main data collection aspects and evaluation procedures were also tested with the pilot-study. Information gathered from the pilot-test was summarized and used to improve data collection using the Summer Genetics Institute Alumni Survey.

The Pilot-test was conducted with 5 individuals who had attended the Summer Genetics Institute during different years, i.e., 2000, 2003, 2004, 2007, and 2009. A cognitive testing interview was conducted with the respondents after completion of the pilot survey to inquire about their experiences completing the Summer Genetics Institute Alumni survey pilot and related procedures. According to respondents, the survey was very easy to navigate and complete, the questions and response options were clear and easy to understand, and that no important questions were missing from the survey. Based on examination of responses, several changes were made to the survey. See Attachment 7 for analysis of pilot data, cognitive interviews, recommendations, and change-decisions made to the survey.

Discovery Logic worked together with the NINR Science Evaluation Officer and other designated NINR staff to implement needed refinements in the survey and procedures.

B.5 Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

The following individuals were consulted on statistical aspects of the design:

• Rebecca Hawes, PhD (office telephone: 301-594-0791; email: [email protected])

• Unni Jensen, PhD, PMP (office telephone: 301-545-4289; email: [email protected])

• Margaret Metzger (office telephone: 301-545-4227; email: [email protected]).

• Jodi Simco, PhD (office telephone 703-841-3111; email: [email protected])

Responsibility for collecting and analyzing information obtained through the methodologies described above will rest with NINR with support from the contractor Discovery Logic. All data collection and analysis will be performed in compliance with Office of Management and Budget, Privacy Act, and Protection of Human Subjects requirements. The NINR project officer for the Summer Genetics Institute Alumni Survey is Dr. Amanda Greene (office telephone: 301-496-9601; email: [email protected]). The Summer Genetics Institute Alumni Survey project manager from Discovery Logic is Ms. Margaret Metzger (office telephone: 301-545-4227; email: [email protected]).

0