Supporting Statement A For:
Surveys to Support an Evaluation of the National Human Genome Research Institute (NHGRI) Summer Workshop in Genomics (Short Course)
Carla L. Easter
Education and Community Involvement Branch
National Human Genome Research Institute
Building 31, Room B1B55
Bethesda, MD 20892
Telephone: 301-594-1364
Fax: 301-480-5008
E-mail: [email protected]
Table of Contents
A.3 Use of Improved Information Technology and Burden Reduction 5
A.4 Efforts to Identify Duplication and Use of Similar Information 6
A.5 Impact on Small Businesses or Other Small Entities 6
A.6 Consequences of Collecting the Information Less Frequently 6
A.7 Special Circumstances Relating to the Guidelines of 5 CFR 1320.5 6
A.8 Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency 6
A.9 Explanation of Any Payment or Gift to Respondents 6
A.11 Justification for Sensitive Questions 8
A.12 Estimates of Annualized Burden Hours and Costs 8
A.13 Estimates of Other Total Annual Cost Burden to Respondents and Record Keepers 9
A.14 Annualized Cost to the Federal Government 9
A.15 Explanation for Program Changes or Adjustments 10
A.16 Plans for Tabulation and Publication and Project Time Schedule 10
A.17 Reason(s) Display of OMB Expiration Date is Inappropriate 11
A.18 Exceptions to Certification for Paperwork Reduction Act Submissions 11
List of Attachments
Attachment 1: NHGRI Short Course Survey
Attachment 2: Background and Rationale for Short Course
Attachment 3: IRB Exemption Notice
Attachment 4: Invitation Letter to Short Course Survey for Students and Faculty
Attachment 5: Applicability of the Privacy Act
ABSTRACT
The National Human Genome Research Institute’s (NHGRI) Education and Community Involvement Branch (ECIB) has been administering the NHGRI Summer Workshop in Genomics (referred to as “Short Course”) since 2003, as a way to accelerate the dissemination of genetic and genomic information to science faculty, especially those at minority serving institutions. The Short Course is an intensive six-day educational workshop designed to update biology instructors, as well as other instructors and researchers in related disciplines, on genomic science.
NHGRI has collected course evaluations from active participants immediately upon course completion, and used that data to inform presenters for the next year. However, there is an absence of information about if, and how the new knowledge and skills gained by participants has been disseminated over the years, and whether or not the course is meeting program goals. Therefore, NHGRI will electronically survey past program participants (from 2004-2012) to assess the methods and degree to which faculty participants updated their curriculum, to identify resources and barriers associated with curriculum integration, and to explore the program influence on student participant’s career paths. The evaluation will be used to inform extramural budget allocations and future program design.
A.1 Circumstances Making the Collection of Information Necessary
The National Human Genome Research Institute (NHGRI), part of the National Institutes of Health (NIH), supports the development of resources and technology that will accelerate genome research and its application to human health. Within NHGRI, the Education and Community Involvement Branch (ECIB) has been administering the NHGRI Summer Workshop in Genomics (aka, the Short Course) since 2003, as a way to accelerate the dissemination of genetic and genomic information to science faculty, especially those at minority serving institutions.
The Short Course is an intensive six-day educational workshop designed to update biology instructors, as well as other instructors and researchers in related disciplines, on genomic science. The course focuses on the continuing effort to find the genetic basis of various diseases and disorders, and current topics on the ethical, legal and social implications of genomics. The course targets college and university faculty seeking to update their curriculum or develop new courses related to genetics. Preference is given to applicants from racial and ethnic groups underrepresented in health related sciences; institutions that predominantly train students with disabilities; or disadvantaged backgrounds including certain rural and inner-city environments. Students from those same circumstances have been included in the program since 2004, but student selection has evolved. Initially, faculty attendees selected students for attendance, and then in 2012, NHGRI began actively recruiting, screening and accepting graduate students independent of faculty attendees, as a way of preparing and attracting future scientists and health care professionals to a genomics workforce. Additional background is provided in Attachment 2.
The Short Course is a highly visible program within NHGRI and is considered by NHGRI leadership as an important investment by the Institute. The Short Course annual budget is approximately $80,000 and represents a total investment of almost one million dollars since its inception. Each year, more than 15 NIH faculty are recruited (from NHGRI and other ICs) to serve as instructors and presenters for the Short Course; 1 to 2 NHGRI staff are responsible for coordinating the logistics and planning throughout the year. As genomic discoveries continue to accelerate, and the need for an informed and educated workforce and public becomes more relevant to health care and decision-making, NHGRI wants evidence of Short Course performance results to inform future course offerings.
NHGRI has assessed the course each year by gathering feedback from participants immediately upon course completion, and using that data to inform the next year, but a full-scale and long-term outcome evaluation has not been conducted. There is an absence of information about if, and how, the new knowledge and skills gained by participants have been disseminated over the years, and whether or not the course is meeting program goals. The collection of information activities set forth herein will be conducted under the authorities granted in the Public Health Service Act, Title 42 USC 285s.
A.2 Purpose and Use of the Information Collection
The purpose of collecting data is to evaluate the long-term outcomes of the Short Course and determine if the program is meeting the following program goals:
Prepare the next generation of genomics professionals for an era of genomic medicine
Train and diversify the pipeline of genome professionals in alignment with the NIH and US Department of Health and Human Services diversity and inclusion efforts
Expand NIH and NHGRI’s professional network to reach out to diverse communities, and to create new partnership opportunities
Specifically, the evaluation will focus on assessing the methods and degree to which faculty participants updated their curriculum (and the timeline required to make changes), identify resources and barriers associated with curriculum integration, and explore the program influence on student participant’s career paths. NHGRI completed a feasibility study in 2013, which has provided the framework for the current outcome evaluation design.
Each program year, administrative staff gathered feedback from participants directly following their participation in the Short Course. The evaluation forms typically offered a three-point rating scale and space for open-ended comments in response to questions, which focused on the quality of: course content; presenters; workshop activities; and supports. Feedback was used to evolve the program in successive years, and although annual process evaluations, applicant data and anecdotal evidence all suggest the Short Course is a valuable training opportunity, popular among educators and highly recommended by past-participants, formal evidence is unavailable. This evaluation will use the proposed survey for both faculty and students to provide detailed information and insight about medium and long-term outcomes of the Short Course. The survey protocol focuses on collecting information on integration of Short Course knowledge into teaching materials; dissemination of genomics beyond the classroom; pursuit of careers related to genomics; and continued participation in NIH-related activities. The web-based survey protocol for students and faculty (Attachment 1) obtains different perspectives on the Short Course from students and faculty by using skip logic.
NHGRI will use findings from this evaluation to make funding decisions and appropriate changes to the program design. For example, understanding the degree of curriculum integration and the methods past-participants are using to update their curriculum will give program staff and presenters better practical instruction about how to pass information along to participants in a meaningful way. Further, knowing about the limitations that exist for past-participants trying to integrate curriculum without success could shed light on the needs of either instructors or their institutions, and lead to important program insights that could have funding ramifications for NHGRI.
If evaluation results indicate the program is meeting its intended goals, findings could translate into program expansion; however, if evidence shows the program is not meeting its goals, the current NHGRI Director would like to make programmatic changes to meet the institute’s mission and strategic plan.
Senior leadership across NIH, and within NHGRI, is particularly interested in viewing the evidence of an outcome evaluation of the Short Course. During his time as NHGRI Director, Dr. Francis Collins was interested in the format of this course at both the IC and trans-NIH levels with regard to its potential to enhance genomics education within the undergraduate curriculum and increase the diversity of trainees within NIH funded training programs. Since one of the program goals is to enhance diversity of the workforce, the results of this evaluation would provide input into current trans-NIH discussions on this subject. NHGRI leadership will share the information with the NHGRI Director and other key decision-makers to help guide the scope of the program, and branch level-chief will use the results to inform the type and scope of program activity. If effective, the Short Course may stand as a model program for other ICs to follow as an effective means for disseminating cutting-edge information and for increasing diversity in the scientific workforce.
The use of web-based surveys will reduce the burden to respondents. A database will be developed that contains the names of the respondents, contact information, and the dates of all email and telephone contacts. All quantitative data will be standardized and compiled into a searchable database. If deemed necessary and worthwhile by the evaluation lead, more sophisticated data management and quantitative analysis software tools (e.g. SPSS software) may be employed.
In addition, a Privacy Impact Assessment (PIA) is being conducted.
This is the first full-scale outcome evaluation conducted since the Short Course’s inception in 2003. The Short Course is unique to NIH.
No small businesses will be involved in this study.
This is a one-time collection.
This study complies fully with the guidelines of 5 CFR 1320.5. No exceptions to the guidelines are required.
A 60-day Federal Register Notice (Volume 80, page 13845) was published on March 17, 2015 and allowed 60 days for public comment and review. There were no comments.
No payment or gift will be made to the respondents.
A.10 Assurance of Confidentiality Provided to Respondents
Respondents will be provided with information about the voluntary survey and relevant evaluation contact information on the Start Page prior to giving consent and continuing with the Web-based survey. Individuals completing this survey will be at least 18 years of age. The survey platform, Survey Analytics, meets all FIPS security and Section 508 Accessibility requirements. Respondents will have the option to skip any question they would prefer not to answer and to quit the survey at any time. Unless express permission is provided by the respondent, all data will be de-identified and reported in the aggregate.
To protect the security of respondents’ information, all project files will be password protected and access to the files will be limited to authorized project staff. All contractors are required to sign an agreement developed by eRA that says that they agree to protect grant data. In addition, project members will be notified in writing prior to the survey data collection process their responsibilities for protecting the information collected. Survey information will be stored on a secure server protected with a Secure Sockets Layer (SSL) certificate and 128-bit encryption, the strongest online data encryption protection available. The tracking database with individual contact information will be stored separately from the data. Contractors need access to e-mail and phone information for grantees to faciliate data collection and follow-up. This includes distributing the survey via e-mail and following up via phone and e-mail to ensure a sufficient response rate. The database will contain IDs only.
The tracking database that links IDs to individual information will be destroyed at the end of the project. This includes removing variables that may identify respondents if viewed in conjunction with other variables. Project reports will not identify individuals who completed the survey. No names or personal identifying information will be used in any published reports of this study unless given express permission from the respondent. Survey reports will present all findings in aggregate so individual responses cannot be identified.
The NIH Privacy Act Officer has reviewed this application and has determined that the Privacy Act is applicable (Attachment 5). Additionally, the Office of Human Subjects Research Protection (OHSRP) has reviewed this project and deemed it does not apply (Attachment 3).
The questions being asked do not require responses that will include privacy information.
Web-based surveys will be issued to approximately 310 respondents. These surveys should take approximately 30 minutes for each respondent. It is estimated that the total and annualized burden will be 155 hours over a year information collection request (Table A.12-1).
Table A.12-1 Estimate of Annual Burden Hours
Form Name |
Type of Respondents |
Number of Respondents |
Number of Responses per Respondent |
Average Burden per Response (in hours) |
Total Annual Burden (in hours) |
Short Course Survey |
Students and Faculty |
310 |
1 |
30/60 |
155 |
Totals |
|
310 |
|
|
155 |
Faculty respondents are comprised primarily of scientists and health professionals; therefore, a mean hourly wage was calculated taking the average of three occupations: $42.98 Medical Scientists (occupation code 19-1040); $37.32 Biological Scientists (occupation code 19-1020); and $44.87 Health Diagnosing and Treating Practitioners (occupation code 29-1000). The May 2013 National Occupational Employment and Wage Estimates - United States (http://www.bls.gov/oes/current/oes_nat.htm#19-0000) was used for the above general categories, which were averaged and resulted in $41.72 per hour. Student respondents are at the graduate student level. An hourly wage was calculated taking the average monthly stipend levels for graduate students at the Office of Intramural Research at NIH and dividing this value by 160 (20 8-hour days a month). This resulted in $17.20 per hour (http://oma1.od.nih.gov/Manualchapters/person/2300-320-7/Appendices/Student14.PDF). The total and annualized cost to respondents is estimated to be $5,118 annually (Table A.12-2).
Table A.12-2 Annualized Cost to Respondents
Type of Respondents |
Number of Respondents |
Total Annual Burden Hours |
Hourly Wage Rate* |
Respondent Cost |
Short Course Students |
110 |
55 |
$17.20 |
$946.00 |
Short Course Faculty |
200 |
100 |
$41.72 |
$4,172.00 |
Total |
$5,118.00 |
|||
There are no direct costs to respondents other than their time to participate in the study.
The total and annualized cost to the Federal Government is approximately $124,442.00 for this information collection. Non-federal personnel costs include the salary and benefits of a project director (contractor), research associates, and a survey statistician for approximately 4 to 5 months. Federal personnel costs include several hours of participation from 2 program officers and an NIH privacy liaison over 4 to 5 months. Materials costs include hosting charges for Survey Analytics. The total costs are in Table A.14-1.
A.14-1 Estimate of Total Cost-Government
Personnel Costs |
Grade/Step |
Annual Rate |
% of time over 12 Months |
Total Cost |
Paperwork Reduction Act Liaison |
13/8 |
$112,000 |
0.5 % |
$560.00 |
Program Officer 1 |
Title 42 |
$200,000 |
0.5 % |
$1,000.00 |
Program Officer 2 |
15/3 |
$134,662 |
5.0 % |
$6,733.00 |
Sub-Total Federal Personnel |
|
|
|
$8,293.00 |
Total Contractor Costs (Ripple Effect Communications Inc.) |
$116,149.00 |
|||
TOTAL |
$124,442.00 |
|||
This is a new information collection.
Data Analysis
Data quality control and quality assurance procedures will be developed and implemented by senior evaluation professionals and applied to all collected data. This will include procedures to ensure accuracy and consistency in data entry, data manipulation, and calculation. Internal validity will be checked as necessary for analysis. Descriptive and summary statistics will be calculated. If warranted, data from multiple sources may be cross-tabulated to address the study questions. Qualitative data from the survey will be coded and analyzed using standard qualitative methods.
Plans for Publication
Publication is not a main goal of this survey, but the team may investigate publishing the findings within the context of the larger evaluation, in a peer reviewed journal.
Project Time Schedule
It is estimated that it will take an experienced evaluation team 12 months to complete the evaluation, with the information collection component taking approximately 3 months. Provided below is an anticipated timeline of major tasks.
Table A.16-1 Project Time Schedule
Major Tasks |
Months after OMB approval |
|||
Month 1-3 |
Month 4-6 |
Months 7-9 |
Months 10-12 |
|
Identify Respondent Groups |
|
|
|
|
Contact Respondents |
|
|
|
|
Administer surveys |
|
|
|
|
Summarize results and deliver final report |
|
|
|
|
No exemption from the display of the OMB Expiration Date is being requested.
There are no exceptions to certification being requested.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | PLCO SCREENING TRIAL |
| Author | Linda Suit |
| File Modified | 0000-00-00 |
| File Created | 2021-01-24 |