3145-0217 Part a - December Revision

Cambridge, MA

Bethesda, MD

Durham, NC

Atlanta, GA

Abt Associates Inc.

55 Wheeler Street

Cambridge, MA 02138-1168

Evaluation of the Robert Noyce Teacher Scholarship Program

Request for Approval of Master Teaching Fellow Survey

Addendum to OMB Package Control: No. 3145-0217

September 1, 2011

Prepared for

Connie Della-Piana

National Science Foundation

Prepared by

Abt Associates Inc.

Contents

Section A 1

Justification 1

Introduction 1

A.1. Circumstances Requiring the Collection of Data 2

A.2. Purposes and Uses of the Data 2

A.3. Use of Information Technology to Reduce Burden 2

A.4. Efforts to Identify Duplication 2

A.5. Small Business 2

A.6. Consequences of Not Collecting the Information 2

A.7. Special Circumstances Justifying Inconsistencies with Guidelines in 5 CFR 1320.6 2

A.8. Selection of Public Comments and Consultation with People Outside the Agency 3

A.9. Payments or Gifts to Respondents 3

A.10. Assurance of Confidentiality 3

A.11. Questions of a Sensitive Nature 4

A.12. Estimates of Response Burden 4

A.13. Estimate of Total Capital and Startup Costs/Operation and Maintenance Costs to Respondents or Record Keepers 5

A.14. Estimates of Costs to the Federal Government 5

A.15. Changes in Burden 5

A.16. Plans for Publication, Analysis, and Schedule 5

A.17. Approval to Not Display Expiration Date 6

A.18. Exceptions to Items 19 of OMB Form 83-I 6

Section B 1

Collection of Information Employing Statistical Methods 1

B.1. Respondent Universe and Sampling Methods 1

Survey and Interview Samples 1

Noyce Recipient Sample 1

Noyce Principal Investigators 2

IHE Faculty 2

K-12 Principals 3

B.2. Procedures for the Collection of Information 3

B.3. Methods to Maximize Response Rates and Deal with Nonresponse 6

B.4. Test Procedures or Methods 6

B.5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data 6

Appendix A: Noyce Master Teaching Fellow Survey

Section A

Justification

Introduction

The National Science Foundation (NSF) received clearance for the evaluation of the Robert Noyce Teacher Scholarship Program on June 13, 2011 through OMB Control Number: 3145-0217. This included collecting primary data via surveys and interviews with Principal Investigators, Faculty, Noyce Recipients, and K-12 Principals.

The Noyce program operates within NSF’s Division of Undergraduate Education, and bridges the higher education and the K-12 system. The Noyce Program encourages talented science, technology, engineering, and mathematics (STEM) majors and professionals to become K-12 mathematics and science teachers. The program provides funds to institutions of higher education (IHEs) to support scholarships, stipends, and academic programs for undergraduate STEM majors and post-baccalaureate STEM students who commit to teaching in high-need K-12 school districts as a condition of receiving financial support. Additionally, the program provides support to undergraduate freshmen and sophomores who serve as summer interns with no commitment to teacher certification or to teaching.

The program supports two other strands for individuals who have already become STEM and/or teaching professionals. The Noyce program supports STEM professionals who enroll as NSF Teaching Fellows (TFs) in master's degree programs leading to teacher certification by providing academic courses, professional development, and salary supplements as the Teaching Fellows fulfill a four-year teaching commitment in a high need school district. This programmatic strand also supports exemplary math and science teachers, who have master’s degrees, to become Master Teaching Fellows (MTFs) in high need school districts by providing professional development and salary supplements.

Since TFs are supported by the Noyce program in preparing for teacher certification and their early years of teaching, they will take same survey that will be given to other recipients that was already approved by OMB.

This additional package is to request a new survey as part of the overall evaluation for the MTFs. The MTF survey will be similar to the other recipient surveys for recipients who are teaching. However, it will focus more on the leadership activities expected of these more experienced teachers. Since MTFs were not supported by the Noyce Program in preparing for certification or their early teaching years, there are no questions in this survey about their teacher preparation program or support during early teaching.

A.1. Circumstances Requiring the Collection of Data

To learn more about the activities of Master Teaching Fellows (MTFs), the study team designed a new survey for this group. A separate survey for the MTFs was not included in the original August 2010 submission. In that submission, data collection was expected to begin in fall 2010, and data for the MTFs was not yet available at that time.

A.2. Purposes and Uses of the Data

The primary purpose for collection is program evaluation. The data collected will allow NSF to describe the supports used by MTFs, and what leadership activities they engage in.

A.3. Use of Information Technology to Reduce Burden

As described in the original OMB clearance package, to reduce burden, online surveys will be used to collect information from participants. MTFs will be asked to complete a web-based survey to learn more about the supports they receive and the leadership activities they engage in. This approach has become more commonly used in recent years. NSF tends to favor web-based systems because they can facilitate respondents’ data entry across computer platforms.

Email will be used to send respondents their invitations to complete the survey and follow-up with the non-respondents to ensure their participation.

A.4. Efforts to Identify Duplication

This evaluation does not duplicate other NSF efforts. There are no other studies at this point that examine activities of MTFs.

Project monitoring data for Noyce will be available to the contractor and the NSF staff working on this research. Surveys will be constructed to ask about elements of program participation that are not captured in the Noyce Monitoring System.

A.5. Small Business

No information for this research will be collected from small businesses.

A.6. Consequences of Not Collecting the Information

If this information is not collected, NSF will be unable to describe the supports and activities of their Master Teaching Fellows.

A.7. Special Circumstances Justifying Inconsistencies with Guidelines in 5 CFR 1320.6

The data collections will comply with 5 CFR 1320.6.

A.8. Selection of Public Comments and Consultation with People Outside the Agency

8A. Federal Register Announcement

Comments on the overall data collection effort were solicited in the Federal Register on June 8 in Volume 75, Number 109 and on August 11 in Volume 75, Number 154. No substantial comments were received.

8B. Consultations Outside the Agency

Consultation was conducted by the research firm, Abt Associates Inc., contracted by NSF to evaluate the Noyce Program, its data collection subsidiary, Abt-SRBI, Principal Investigator from the previous evaluation of the Noyce Program. Senior technical staff from these organizations who are conducting the study are listed below.

Abt Associates, Inc. Ellen Bobronnikov 617-349-2718

Beth Gamse 617-3492808

Amanda Parsad 301-634-1791 Radha Roy 301-347-5722

Abt-SRBI Paul Schroeder 301-608-3883

Andy Weiss 301- 608-3883

In addition, the MTF survey was pilot tested with seven respondents drawn from the target populations. Respondents were asked to comment on the clarity and content of the questions. The duration of the data collections was recorded to help with an accurate estimation of time burden.

A.9. Payments or Gifts to Respondents

No payment or gift will be provided to respondents.

A.10. Assurance of Confidentiality

Respondents will be advised that any information on specific individuals will be maintained in accordance with the Privacy Act of 1974. Data collected will be available to the evaluation contractor, the contractor hired to manage data collection, and at the aggregate level to NSF officials and staff. Data are processed in accordance to Federal and State privacy statutes. Detailed procedures for making information available to various categories of users are specified in the Education and Training System of Records (63 Fed, Reg. 264, 272 January 5, 1998). The system limits access to personally identifiable information to authorized users. Data submitted will be used in accordance with criteria established by NSF for monitoring research and education grants, and in response to Public Law 99-383 and 42 USC 1885c. The information requested may be disclosed to qualified researchers and contractors in order to coordinate programs and to a Federal agency, court or party in court, or Federal administrative proceeding, if the government is a party.

Individuals surveyed will be assured that the information they provide will not be released in any form that identifies them as individuals and their responses will be kept confidential. The contractor will be expected to maintain the confidentiality, security, and integrity of survey data and interview data. The web-based survey data, notes, and transcripts of the interviews will be maintained on a secure server with appropriate levels of password and other types of protection.

All assurances of confidentiality will be reviewed by the contractor’s Institutional Review Board prior to data collection.

A.11. Questions of a Sensitive Nature

The proposed surveys ask for demographic information (gender, race, ethnicity, and marital status) from Noyce recipients on a voluntary basis, thus respondents may choose not to provide information that they believe is sensitive in nature. This information is being collected so that NSF can answer questions about how effective the Noyce program has been in the recruitment and preparation of a diverse population of recipients. All survey questions have been reviewed by the contractor’s Institutional Review Board.

A.12. Estimates of Response Burden

A.12.1. Number of Respondents, Frequency of Response, and Annual Hour Burden

The target population for the additional MTF survey includes 104 individuals who received Master Teaching Fellowships from 10 awards in the 2010-2011 academic year. The response burden for these surveys is estimated to be 52 hours for the 104 respondents.

Exhibit 2 indicates the number of average respondents per year to be surveyed for each respondent type and the time demand these surveys will place on each individual respondent.

Exhibit 2: Estimated Annual/Total Burden Hours for MTF Surveys
Average Number of Respondents	Time Per Response (Hours)	Number of Responses	Total Time Burden (Hours)
104	0.5	104	52

A.12.2. Hours Burden Estimates by Each Form and Aggregate Hours Burdens

The total response burden for surveys is estimated to be the same as the annual burden specified in the previous section, since there is expected to be only be one wave of data collection.

A.12.3. Estimates of Annualized Cost to Respondents for the Hour Burdens

The overall total cost to the MTFs for participating in surveys is $1,196, as indicated in Exhibit 3.

Exhibit 3: Estimated Cost to Respondents for MTF Survey
Hourly Salary Estimate	Burden Time Per Respondent		Estimated Cost to Respondent	Number of Respondents in Category	Estimated Total Cost for MTFs
$23.00	0.5	$11.50		104	$1,196.00
Salary estimates for Master Teaching Fellows are based on public school teacher salary estimates as reported in NCES Projection of Education Statistics to 2015. NCES September, 2006

A.13. Estimate of Total Capital and Startup Costs/Operation and Maintenance Costs to Respondents or Record Keepers

There is no overall annual cost burden to respondents that results from this study other than the time spent responding to the survey attached as an appendix to this request.

A.14. Estimates of Costs to the Federal Government

The estimated cost to the Federal Government for the additional data collection activities for MTFs is $29,678. This cost estimate includes instrument development, online programming, and pretesting. Costs for overall data collection, analysis, and reporting were built into the original request.

A.15. Changes in Burden

This is a new collection of information.

A.16. Plans for Publication, Analysis, and Schedule

The purpose of this additional data collection is to examine the outcomes of NSF–funded Noyce awards on Master Teaching Fellows (MTFs). This will help NSF respond to such questions as: How has the Noyce Program helped improve the ability of K-12 teachers to teach science and math for different types of recipients?

The survey data will yield important details about the supports MTFs receive and the leadership activities they engage in. The study team will also examine variation in program implementation (types and levels of financial support and of Noyce-supported activities), school and district characteristics, and recipient characteristics.

Analyses of all survey data will include a detailed summary that utilizes appropriate descriptive statistics. For survey items using continuous scales, the study will calculate means and standard deviations to describe both central trends and variation across the samples of schools. Frequency distributions and percentages will be used to summarize answers given on ordinal scales.

The strength of the observed associations between program and recipient characteristics and outcomes will be estimated using multiple regression techniques to appropriately control statistically for other factors that could potentially affect recipients’ plans to stay in teaching other than the recipients’ respective IHE’s level of financial support (i.e., contextual factors such as climate of K-12 school, districts’ hiring policies due to changes in state or district economic circumstances).

The results of the MTF findings will be made public as a part of the overall program evaluation.

A.16.1 Project Time Schedule

Spring 2011:	Study design
Summer 2011	Prepare online surveys
January-April 2012	Administer online surveys of Master Teaching Fellows

A.17. Approval to Not Display Expiration Date

The data collection instruments will display the expiration date.

A.18. Exceptions to Items 19 of OMB Form 83-I

No exceptions are sought.