TLF Analysis

November 6, 2006

Analysis of Changes to the 2006 SASS-16(X) Teacher Listing Form, and the TFS-1(X) Teacher Status Form and Recommendations for 2007-08 SASS and 2008-09 TFS

Issues for Analysis:

1. Did the new question on the TLF work - “Do you think it is likely that this teacher will be teaching in this school next year?”

2. Does the highly experienced (currently 20+) category need to be redefined, and is it beneficial for oversampling?

3. Did the rewrite of the categories on the TFS-1(X) have the desired effect of reducing the “Don’t Know” strata? The category corresponding to the Don’t Know strata was rewritten from “Left the School – No Other Information Given” to “Left the School – Occupational Status Unknown”.

Goal:

Be able to increase the number of leavers in TFS through changes in SASS sampling, with a particular focus on retiring teachers.

Background:

As part of the field test for the 2005-06 SASS, one question was modified and one question was added to the Teacher Listing Form. The experience question previously allowed for responses in only two categories: (1) three or fewer years, and (2) more than three years. For the Field test, the question was modified to split the more experienced category into (2) 4-19 years, and (3) 20 or more years. A new question was introduced asking if schools thought that it is likely that the teacher would be teaching in their school the following year.

Nearly all teachers, except where it would cause the school’s overall teacher sample size to exceed 20, who planned to leave and had 20+ years of experience were selected for sample in the 2005-06 SASS Pretest. Less experienced teachers who planned to leave as well as teachers who did not plan to leave were sampled at the usual rate of about 1-in-6 on average.

To determine the current status of all the Pretest sample teachers, the TFS-1(X) was mailed on August 29 to SASS Pretest schools, and was administered through October 16^th. Modified occupational status codes were introduced on the TFS-1(X) in an attempt to reduce the number of teachers classified as ‘Don’t Know”. A response rate of 99.3% was obtained.

Analysis:

Issue 1

The TFS-1 results were tabulated and analyzed by original teacher listing form experience and leaver expectation. Results are listed in Table 1. From the table, it appears that around 69% (assuming 80% of the teachers for which the school does not know their status, are leavers) of the highly experienced teachers who were expecting to leave really did leave. An additional 11% moved to another school. So, it does appear this group of teachers behaved in a way consistent with the school’s past-year expectations.

Of the less experienced teachers who were expecting to leave, about 38% were leavers and about 35% were movers (again assuming 80% of the don’t know teachers are leavers). So, while the expectation question was somewhat less effective for this group, it still did identify a substantial number of movers and leavers.

An additional issue related to the effectiveness of this question relates to the actual proportion of leavers that could be identified using this question. Pretest teachers were not weighted, so the actual proportion cannot be measured from Table 1, but taking into account that the highly experienced teachers not expected to leave were sampled at about 1-in-6 and the teacher expecting to leave were mostly sampled, it appears the question identified about 25-30% of highly experienced leavers, and about 30-35% of the less experienced leavers.

While this is not a terribly high proportion and will not improve the reliability of leaver estimates in TFS by a huge amount, it will allow for a substantial increase in the size of the retiree sample in TFS, enabling the publication of data. This expected increase in retirees assumes highly experienced teachers expected to leave the school are oversampled at a very high rate.

Recommendation: Keep this question on the SASS-16 for 2007-08 SASS. Oversample highly experienced teachers expecting to leave at a very high rate in SASS. Oversample less experienced teachers expecting to leave at a somewhat lower rate in accordance with design goals of TFS.

Issue 2

Tables 2 and 3 and the associated Figure address the question of where the most appropriate place is to draw the line between high and medium experience. As the graph shows, for public teachers, attrition rates bottom out at around 20 years of experience, and climb slowly until about 29 years of experience, when the increase sharply, but the attrition rates still don’t reach 25% until around 40 years of experience, at which point hardly any teachers are left teaching. The trend for private teachers is less clear-cut, attrition rates are generally higher than for public teachers until about 27 years of experience, then are consistently lower.

The optimum place to draw the line would appear to be about 30 years of experience; at this point, however, a strong majority of teachers do not leave in any given year, apparently no more than 15%. So, experience alone does not appear to be an effective means for oversampling SASS teachers to obtain more retirees for TFS.

Given that experience alone is not effective, the experience question then becomes, “what is the optimum place to draw the line in combination with the leave expectation question?”

Only minimal data were keyed from the SASS Pretest teacher questionnaires, so it is not easily possible to do this analysis from the available data. At this point, though, it does not appear that we would have any reason to question the placement of the cutoff at 20+ .

Recommendation: Keep the experience categories from the SASS-16(X) for the 2007-08 SASS Teacher Listing From. That is, use the categories: (1) 3 years or less, (2) 4-19 years, and (3) 20 years or more.

Issue 3

Did the rewriting of the TFS-1 occupational status codes work to reduce the number of teachers classified as status unknown?

From the pretest, if there had been no oversampling, it appears that about 283 teachers would have been classified as stayers and 261 would have been classified as unknown. While this seems to be a rather high number of unknowns, in 2005 TFS there were1637 leavers and 3129 unknowns – a much higher proportion, so it does appear that changing the categories, including rewording the unknown category from “Left the school – no other information given” to “left the school – occupational status unknown” was very effective in reducing the size of the unknown category.

Recommendation: Keep the revised occupational status codes for the 2008-09 TFS-1.

Table 1: Results of TFS-1 Status Operation from Sept-Oct 2006

Expecting to leave	Experience	stayer	stayer pct	mover	mover pct	leaver	leaver pct	dk	dk-pct
Yes	0-3	46	29.3%	39	24.8%	20	12.7%	52	33.1%
	4-19	41	23.8%	58	33.7%	30	17.4%	43	25.0%
	20+	65	20.4%	30	9.4%	199	62.6%	24	7.5%
no or DK	0-3	640	76.6%	73	8.7%	47	5.6%	75	9.0%
	4-19	2177	87.7%	158	6.4%	71	2.9%	75	3.0%
	20+	1032	88.4%	41	3.5%	82	7.0%	12	1.0%

Table 2: Public School Teacher Attrition Rates from 2004 SASS and 2005 TFS sampling frame

Yrs of Experience	leavers	don't know	stayers or movers	Attrition Rate
0	118	194	2658	9.2%
1	53	292	2066	11.9%
2	39	208	1855	9.8%
3	39	159	1970	7.7%
4	50	141	1855	8.0%
5	44	132	1898	7.2%
6	46	88	1695	6.4%
7	37	73	1592	5.6%
8	38	60	1471	5.5%
9	30	60	1283	5.7%
10	24	52	1444	4.3%
11	23	38	1179	4.3%
12	16	39	1023	4.4%
13	24	43	1101	5.0%
14	21	28	1057	3.9%
15	15	38	973	4.4%
16	26	36	889	5.8%
17	25	41	873	6.2%
18	17	19	884	3.5%
19	12	19	802	3.3%
20	19	26	960	4.0%
21	13	19	807	3.4%
22	18	17	702	4.3%
23	16	26	824	4.2%
24	27	27	770	5.9%
25	18	23	894	3.9%
26	24	26	811	5.2%
27	30	38	773	7.2%
28	29	37	777	7.0%
29	26	36	689	7.3%
30	52	54	805	10.5%
31	42	39	622	10.4%
32	46	49	544	13.3%
33	53	50	455	16.7%
34	44	42	344	18.0%
35	43	34	252	21.3%
36	26	24	172	20.4%
37	12	17	115	17.8%
38	8	11	83	16.5%
39	7	7	60	17.0%
40	7	8	32	28.5%
41	7	4	28	26.2%
42-43	4	3	26	19.4%
44 or more	6	6	30	25.7%

Table 3: Private School Teacher Attrition Rates from 2004 SASS and 2005 TFS sampling frame

	Yrs of Experience		leavers		don't know		stayers or movers		Attrition Rate
	0		27		62		387		16.1%
	1		45		110		530		19.4%
	2		44		116		521		20.1%
	3		35		77		525		15.2%
	4		22		62		363		16.0%
	5		20		37		353		12.1%
	6		15		29		276		11.9%
	7		12		28		268		11.2%
	8		14		24		219		12.9%
	9		9		17		210		9.6%
	10		14		22		233		11.7%
	11		9		15		169		10.9%
	12		7		11		157		9.0%
	13		7		13		170		9.2%
	14		3		4		180		3.3%
	15		4		9		159		6.5%
	16		5		7		140		7.0%
	17		4		8		150		6.4%
	18		4		7		137		6.5%
	19		2		9		133		6.4%
	20		6		3		123		6.4%
	21		5		7		98		9.6%
	22		1		2		105		2.4%
	23		5		3		105		6.5%
	24		5		4		112		6.8%
	25		2		7		92		7.5%
	26		4		4		106		6.3%
	27		1		4		94		4.2%
	28		4		1		91		5.0%
	29		2		6		73		8.4%
	30		1		2		93		2.7%
	31		1		4		61		6.4%
	32		0		2		69		2.3%
	33		5		3		47		13.5%
	34		0		4		48		6.2%
	35		0		4		51		5.8%
	36		4		3		33		16.0%
	37		1		3		33		9.2%
	38		1		4		29		12.4%
	39		1		1		22		7.5%
	40		4		3		17		26.7%
	41		0		0		15		0.0%
	42-43		1		2		12		17.3%
	44 or more		7		3		30		23.5%