Att_TFS Supporting Statement PART B

PART B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Respondent Universe

Background on the SASS 08 sample.

The SASS sample was a stratified sample. The schools were sampled first and then teachers were selected within each sampled school. Schools were first classified by school type (public/private). Public schools were stratified by the 50 States and the District of Columbia and then by three grade levels (elementary/secondary/combined). Sampled public schools were selected with a probability proportional to size (PPS), which is the square root of the number of teachers in a school within each of the 153 public school strata. A district that had at least one school selected for the public school sample was included in the district sample for this survey. Within each sampled public school stratum, an average teacher sample size of 3, 7, or 5 teachers was randomly selected for elementary, secondary, and combined schools, respectively.

Private schools were stratified by grade level, region, and religious orientation, including:

• Baptist

• Catholic-Parochial

• Catholic-Diocesan

• Catholic-Private

• Jewish

• Lutheran

• Nonsectarian-Regular

• Nonsectarian-Special Emphasis

• Nonsectarian-Special Education

• Seventh-Day Adventist

• Other Religious

Sampled private schools were selected with probability proportional to size, which is the square root of the number of teachers in a school within each of the private school strata. Within each sampled private school stratum, an average teacher sample size of 4, 5, or 3 was randomly selected for elementary, secondary, and combined schools, respectively.

The sampled schools were asked to provide a list of all teachers teaching in the school and the following information for each teacher on the list:

• Whether the teacher’s total teaching experience was 3 years or less, 4 – 19 years, or 20 or more years;

• Main subject taught (general elementary, special education, math, science, English/Language arts, social studies, vocational/technical, or other subjects);

• Full-time or part-time teaching status at the school; and

• Whether it was likely that the teacher would be teaching at the school for the next school year (2008-09).

The above information for each teacher in a selected SASS school comprised the teacher listing operation or the teacher sampling frame.

Within each selected school, teachers were classified as belonging in one of the following five categories:

1. New stayer (3 or fewer years in the teaching profession) and likely to be teaching at the same school;

2. Not new stayer (4 or more years in the teaching profession) and likely to be teaching at the same school;

3. New leaver/mover (3 or fewer years in the teaching profession) and not likely to be teaching at the same school;

4. Mid career leaver/mover (4 to 19 years in the teaching profession) and not likely to be teaching at the same school; and

5. Late career leaver/mover (20 or more years in the teaching profession) and not likely to be teaching at the same school.

Teachers not expected to be teaching at the same school (groups 3, 4, and 5) were oversampled to achieve a sufficient sample size (1,200 public and 800 private) to support national estimates and to achieve TFS sample design goals.

Private school teachers with 3 or fewer years in the teaching profession (group 1) were oversampled to achieve a sufficient total group sample size (2,400) to support regional and national estimates. (Oversampling of new teachers was also done in the 1990-91, 1993-94, 1999-2000, and 2003-04 SASS.)

Within each teacher stratum, teachers were sorted by teacher’s main subject taught (as reported by the principal on the SASS Teacher Listing Form). This method was used to assure a good distribution of teachers by main subject taught.

Within each school and teacher stratum, teachers were selected systematically with equal probability, meaning that each teacher within each stratum and school is given an equal chance of selection and sorted by subject taught.

The following table shows a summary of the SASS 08 sample sizes:

	District	School	Principal	Teacher	Library
Public	5,252	9,990	9,990	48,400	9,990
Private	-----	2,9371	2,937	8,300	-----

Teacher Follow-up Survey (TFS) Sample Design.

The TFS 09 sample will consist of a subset of the SASS 08 teacher sample. All SASS 08 respondents who were first-year teachers in public schools in the 2007-08 school year will be included in the TFS 09 sample. It is estimated that there will be approximately 2,000 such teachers.

Another 3,500 teachers will be selected according to a stratified design similar to earlier TFS collections and based on information obtained from the SASS schools about the status of the SASS 08 respondents. The TFS 09 sample will include both former teachers (SASS teachers who have left the teaching profession) and current teachers (SASS teachers who have remained in the teaching profession). The current teachers group is stratified further into two groups–those who moved to different schools and those who stayed in the same schools. This design provides analytic data on teachers who stay, move, or leave the teaching profession. (NOTE: Status for each SASS teacher is determined through a Teacher Status Form sent to the school in the fall of 2008. OMB clearance for the Teacher Status Form was obtained through an addendum to the SASS 08 clearance.)

Teachers will be further stratified by sector (traditional public, public charter, and private), grade level (elementary, middle, and high school), experience level (first-year, 2^nd or 3^rd year, and 4 + years of teaching), and minority status (minority or not minority).

Within each public TFS stratum, teachers will be sorted by teacher subject, Census region, urbanicity, school enrollment, and SASS teacher control number. Within each private TFS stratum, teachers will be sorted by teacher subject, association membership (list frame), orientation (area frame), urbanicity, school enrollment, and SASS teacher control number.

After the teachers are sorted, teachers will be selected within each stratum using a probability proportional to size sampling procedure. The measure of size will be the SASS 08 final teacher weight.

The target sample size for leavers/movers is 1,500. For stayers, 2,000 will be selected. The breakdown of leavers/movers and stayers in traditional public, public charter, and private schools is as follows:

	Leavers/Movers	Stayers	Total
Traditional Public	975	1,360	2,335
Public Charter	60	75	135
Private	465	565	1,030
Total	1,500	2,000	3,500

Within strata, teachers will be selected with probability proportional to the inverse of the teacher’s SASS probability of selection.

2. Procedures for Collection of Information

Based on Teacher Status Forms completed by sampled schools in Fall 2008, Census Bureau staff will select the sample of teachers for the TFS and prepare materials for data collection.

In the SASS 08, teachers were asked to provide the following information:

• Name

• Spouse’s name

• Home address

• Home telephone number and in whose name it’s listed

• Most convenient time to reach her/him

• Work email address

• Home email address

• Similar information for two other people who would know where to get in touch with her/him.

During December 2008, staff will conduct research in order to obtain contact information for teachers who provided incomplete or no contact information in the SASS interview.

Data collection will begin in February 2009. The first contact with sampled teachers will be through mail (and email, if email addresses are available). If home addresses were not provided in SASS 08 or cannot be obtained, the SASS school address will be used as the address. Each letter will be customized and will provide the sampled teacher with a username and password to access the web-based instrument. The letter also will explain the purpose of the survey and include a statement of authority and will discuss NCES’ policy on protecting personal information. About two weeks later, nonrespondents will be sent a second letter (and email, if email addresses are available). Throughout data collection, research will be conducted, as needed, to find current addresses and emails for sampled teachers whose letters and emails are returned as undeliverable.

In March 2009, telephone reminders will be made to those teachers who have not yet responded to the web-based collection. Every effort will be made to encourage self-administered web-based participation. However, telephone interviewers will be able to log into the web-based instrument and use it much like a CATI (computer-assisted-telephone-interview) instrument. Throughout the data collection period, teachers requesting a hardcopy questionnaire will be sent one.

In late April, all nonrespondents will be sent a hardcopy questionnaire—the version deemed to be appropriate given known information about experience (first-year v. other-than-first-year) and status (current v. former teacher). Data collection will end in late May.

Nonresponse Issue.

The Teacher Follow-up Survey was designed from its inception to sample only from the respondents to SASS in the prior year. It is possible to sample teachers for TFS from the pool of teachers initially selected for SASS rather than only from those who responded to SASS. By not sampling teachers from the nonresponse stratum, potential biases may be introduced into TFS. When considering unit nonresponse for the previous administration of SASS, there was no evidence to point to a substantial bias in teacher response rates in SASS estimates.

The main purpose of TFS is to measure various characteristics collected on SASS of teachers who move, leave, or stay in the teaching profession the following year. If SASS nonrespondents were to be contacted for the Teacher Follow-up Survey, there are several methodological problems that would need to be solved: 1) there would be no prior year data, so all of the SASS teacher questionnaire items would have to be asked retrospectively of the teacher before the TFS could be administered; 2) if the teacher was determined to be out-of-scope for SASS, then there would be no TFS interview and a sampling adjustment would be required; and 3) if the teacher was in-scope for SASS but out-of-scope for TFS (if the teacher moves out of the country or dies), then that would also require some adjustment of the sampling weights. Logistically, trying to combine prior-year SASS interviewing with current-year TFS interviewing is a complication that would slow down the field data collection, because it would probably take longer to wrap up the nonresponding SASS cases. For these reasons, TFS teachers will be sampled from responding SASS teachers only.

A response rate in the approximate range of 88-90 percent can be expected from the leavers after follow up, and a response rate of 88-92 percent can be expected from stayers and movers after follow-up, based on the prior administration of TFS.

3. Methods for Maximizing Response Rates

A variety of procedures will be employed to ensure high response rates at both the level of the responding unit (i.e., sample member) and at the level of the individual survey items in each survey questionnaire.

The entire survey process, starting with securing research cooperation from key public and private school groups and individual sample members and continuing throughout the distribution and collection of individual questionnaires, is designed to increase survey response rates. In addition, NCES believes that the following four elements of the data collection plan, in particular, will contribute to the overall success of the survey and will enhance the response rates.

1. Endorsements from key public and private school groups.

The level of interest and cooperation demonstrated by key groups can often greatly influence the degree of participation of survey respondents. Endorsements are viewed as a critical factor in soliciting cooperation from state and local education officials for obtaining high participation rates in the private sector. The TFS has been endorsed by the following organizations or agencies:

American Federation of Teachers

Association Montessori International

Association of Christian Schools International

Association of Christian Teachers and Schools

Bureau of Indian Affairs, Office of Indian Education Programs

Christian Schools International

Council for American Private Education

Evangelical Lutheran Church in America

Islamic School League of America

Jesuit Secondary Education Association

Jewish Community Day School Network

Jewish Education Services of North America

Lutheran Church - Missouri Synod

National Association of Elementary School Principals

National Association of Episcopal Schools

National Association of Independent Schools

National Association of Private Special Education Centers

National Association of Secondary School Principals

National Catholic Educational Association

National Coalition of Girls’ Schools

National Council for Private School Accreditation

National Education Association

National Independent Private Schools Association

National Indian Education Association

North American Division of Seventh-Day Adventists

Oral Roberts University Educational Fellowship

Solomon Schechter Day School Association

Wisconsin Evangelical Lutheran Synod

2. Stressing the importance of the survey and the respondents' participation in it.

Official letters (advance and follow-up) from our NCES Project Director will motivate respondents to return their surveys. Knowledge of support by various respected affiliations and organizations is intended to increase the respondent’s perception of the importance of the survey. The additional personalization of survey materials (e.g., cover letters and survey packets with teachers’ names) is expected to have a positive effect on the response rates.

3. Extensive follow-up (by mail, email, and telephone) of non-respondents.

The Census Bureau will use a variety of techniques to increase response levels to TFS, including:

(a) Using mixed survey modes—web-based, telephone, and self-administered mail instruments, as needed, to achieve high response levels.

(b) Allocating adequate time and resources to respondent tracking efforts to ensure that a high percentage of movers and leavers are successfully located and surveyed. A variety of techniques will be employed to locate survey respondents. Potential tracking sources include: (a) names, addresses, and phone numbers for respondents’ spouses and two other friends or relatives (if the respondents listed this information in the completed SASS teacher survey); (b) leads provided by school principals or their designees; (c) directory assistance and telephone directories; and (d) post office for possible forwarding addresses.

Good questionnaire design techniques will be employed to minimize item non-response. All completed questionnaires from the 2004-05 survey were analyzed carefully to determine which items had the highest levels of item nonresponse. This information guided NCES and the Census Bureau in reviewing the clarity of item wording, definitions, and instructions.

To permit sufficient time to locate sampled members who left their teaching position in the previous year and thorough follow-up of non-respondents to the mail survey, the Teacher Follow-up Survey should begin in February 2009. This will allow 4 months for all data collection activities related to teachers.

4. Tests of Procedures and Methods

The 1988-89 Teacher Follow-up Survey was field tested in 1987-88. Since then, TFS has been conducted on a full-scale basis five times: in 1988-89, 1991-92, 1994-95, 2000-01, and 2004-05. The results from those experiences have been used to clarify and revise questions.

After the 1994-95 TFS an extensive reinterview and reconciliation program was implemented. NCES used the results of this reinterview to revise the questions to minimize response error and improve the flow of the questionnaire. For the 2004-05 TFS, several items were revised or eliminated based on results from cognitive interviews. Items were tested, revised, and then retested with current and former teachers to determine whether the items were clearly and uniformly understood. In addition, cognitive interviews were conducted on new and modified items for the TFS 09.

An Internet and incentive experiment was conducted during the 2004–05 TFS. The goal was to use monetary incentives to increase overall response rates and responses via the Internet, when both mail and Internet choices were offered. The current and former teachers who were selected to be in the TFS sample were randomly divided into six different treatment groups. The groups varied on three dimensions, which included offering an Internet option, offering a prepaid $10 incentive, and notifying respondents in the Internet groups that they also would receive a paper questionnaire.

Key findings from the 2004–05 TFS Internet and incentive experiment can be summarized as follows:

• The best response to TFS was achieved by offering only the mailed paper questionnaires.

• While adding an Internet option negatively impacted the total number of responses, the negative effect was offset by not mentioning the forthcoming mail option.

• Offering a small, prepaid incentive increased responses by 5.5 percent, which is statistically significant (p < .05).

5. Reviewing Statisticians

Dennis Schwanz of the Census Bureau reviewed and approved the TFS sample design and related matters for statistical quality, feasibility, and suitability to the overall objectives of the survey.

1 Includes data comparable to public school districts.

9

9