Supporting Statement B for
Effectiveness of the NIH Curriculum Supplements Programs and Career Resources Study (OD/OSE)
December 5, 2008
Name: Dave Vannier, Ph.D. (OD/OSE)
Address: 6100 Executive Blvd, Suite 3E01
Telephone: (301) 496-8741
Fax: (301) 402-3034
Email: [email protected]
16,000 randomly-selected NIH curriculum supplement customers will be directed, via e-mail, to a 10-minute, Web-based survey that will measure implementation and satisfaction. This instrument will be very similar the survey used for the same purpose in the evaluation of the first three supplements (OMB No. 3145-0192). The survey will be sent once OMB approval is obtained.
Since 2004, 7,600 individuals have requested the Women Are Scientists videos. OSE will randomly survey 1,500 of these requestors, 20% of the respondent universe.
Since FY 2002, over 6,000 science teachers have attended a workshop on using the NIH curriculum supplements. 2,000 of these teachers will be sent a short follow-up survey after he/she attends a workshop. This e-mailed questionnaire will capture 1) how effective the workshop was in preparing the teacher to implement the lessons and 2) suggestions for further support in using the lessons.
In years two and three, two groups of 100 workshop attendees will be selected to answer a more in-depth survey. The selection criteria will reflect populations of interest to OSE, such as middle school teachers, pre-service teachers, urban teachers, and less-experienced teachers. Results from the short survey will likely identify sub-groups that are not being effectively trained during the workshops. These are of obvious interest to OSE and would likely be sent the longer survey. We anticipate an 80% response rate and will offer a $25 honorarium for completing the survey.
The data collected from the teacher workshop surveys will be subjected to very simple statistical analysis, such as determining means and standard deviations. The evaluation seeks to validate the overall effectiveness of the teacher workshops. It is unlikely that a more statistically rigorous approach will lead to greater insight.
B.2. Procedures for the Collection of Information
Data collection will be conducted electronically. Respondents will be contacted by e-mail and directed to a Web site to complete their respective survey. Data entered on line will go directly into a specific database for tabulation. Data from surveys returned by e-mail will be entered into the specific database by OSE staff.
These surveys seek to measure the effectiveness of OSE programs. A complicated statistical approach should not be necessary for us to meet our evaluation goals.
Teacher Workshop Evaluations: 2,000 teachers will be surveyed for their assessment of the workshop they attended. Since these teachers have met the OSE staff, we anticipate that they will be likely to respond. The initial survey will be extremely concise to minimize the time burden on the participant. During years 2 and 3, 200 respondents from the initial survey will be asked complete a longer survey. We are optimistic that over 80% will respond, and we will offer a $25 honorarium for completing the longer survey.
Prior to release, all surveys will be tested among federal employees for ease of completion. Once the surveys have been optimized in house, nine teachers will be randomly selected to test the each instruments as well. The Teacher Workshop Surveys are ongoing evaluations. Early results will influence the procedures and questions for future surveys. Each revised survey will be sent to OMB for approval prior to use.
B.5 Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data
Evaluations will rely exclusively on staff in the NIH Office of Science Education. Dave Vannier, Ph.D., will coordinate both projects and may be reached at (301) 496-8741 and [email protected].
-
| File Type | application/msword |
| File Title | Date |
| Author | IsomC |
| Last Modified By | curriem |
| File Modified | 2008-12-04 |
| File Created | 2008-12-04 |