REL Southwest OMB Supporting Statement (B#2) - Studies 1.1.1 1.1.2

REL Southwest Contract No. ED-06-CO-0017

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection methods to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection has been conducted previously, include the actual response rate achieved during the last collection.

The respondents participating in this survey will consist of educators/educational administrators, parents and business leaders from the REL Southwest five state region (including Arkansas, Louisiana, New Mexico, Oklahoma, and Texas). According to 2006 estimates from Market Data Retrieval’s (MDR) 2005-2006 Educator Database, a prominent national level educator data collector, the total count of educators (having the specific titles selected for this study) is 563,947.

The chart below details the potential respondent universe as a whole as well as for each educator title that will be targeted in the proposed study. The response rate for the data collection is estimated to be at 5%. Because this is a new project, no previous collection existed, so no statistical response rate achieved during a previous collection can be provided.

Market Data Retrieval’s Educator 2005-2006 Population Database	AR	LA	NM	OK	TX	Total Populations
Superintendents	253	75	92	566	1,070	2,056
Asst. Superintendent	49	31	29	52	400	561
Special Education Director	245	85	92	509	976	1,907
Curriculum Director	288	156	83	417	1,061	2,005
Testing/Assessment Director	138	43	35	251	467	934
ELL/Bilingual Director	75	23	36	118	443	695
Principal/Assistant Principal	1,815	2,725	1,152	2,296	16,311	24,299
Teachers	36,737	55,473	20,297	46,011	338,178	496,696
Librarian	925	1,131	567	1,033	6,326	9,982
Higher Education 2 YR (Deans, Administrators and Dept. Heads)	520	568	483	645	3,326	5,542
Higher Education 4 YR (Deans, Administrators and Dept. Heads)	1,111	1,776	522	1,360	5,633	10,402
Higher Education Grad (Deans, Administrators and Dept. Heads)	756	1,570	497	1,127	4,918	8,868
Total Count						563,947

*Note that true population counts for parents and business leaders are estimates only. However, these are groups that we will be targeting on a scale that is sufficient to reliably project to their respective populations across REL Southwest’s five state region.

2. Describe procedures for the collection of information including: statistical methodology for stratification and sample selection; estimation procedure; degree of accuracy needed for the purpose described in the justification; unusual problems requiring specialized sampling procedures; and any use of periodic (less frequent than annual) data collection cycles to reduce burden.

A combination of secondary and primary research efforts will be undertaken for this study. The secondary research will be conducted prior to launching the primary research. Through secondary research, most current estimates of the actual constituent populations will be identified for each of the 5 REL Southwest states. This secondary research will be used to guide the stratification and development of quotas to be used in the primary methodology. Data collection will occur every year. In the primary phase, a survey will be deployed to fulfill project research objectives. The survey will be deployed via the internet. For identified constituents who do not have access to the internet, a phone survey may be deployed to meet quota targets if necessary. For example, a phone survey may be deployed to meet survey quota targets in hard to reach places in hurricane-impacted Louisiana or rural schools/districts or with parents who do not have access to the internet and thus cannot complete an online survey.

• Stratified random sampling will be done of the entire survey population to ensure collection of statistically projectable data that is representative of all educators involved in elementary, secondary and higher education in the southwest.

• Stratified random sampling will also be done to ensure collection of statistically projectable data that is representative of all parents and business leaders involved in elementary, secondary and higher education in the southwest.

• Survey sampling and quotas will be set to fulfill 95% confidence level limits (at +/-10% confidence interval).

• Sampling of the populations will be done using the below listed two part formula. The first part calculates the sample size as though the population is infinite. This is defined as n. The second part corrects for the population size.
  1) n=(z/m)^2*p*(1-p) where:

• z is the standard normal statistic

• m is the desired error

• p is the probability of the outcome

• ^2 means "the square of the resulting number"

2) n'=n/(1-n/N) where:

• n is the result from the equation in part 1) above

• N is the population size

• Accuracy of sampling will be maintained by using educator contact information that is accurate and reflects accurate information for 2006, from renowned and prestigious list providers such as Quality Education Data (QED) and Market Data Retrieval (MDR).

• Below are stratification details for each targeted educator title. All sample counts have been based on Market Data Retrieval’s Educator 2005-2006 Population Count.

(Chart note: *MDR Population = Market Data Retrieval’s 2005-2006 Educator Database)

Segment	Sub Segment	Title	AR	LA	NM	OK	TX	Sub Total	Total Sample Required
Pre K-12	Rural	Superintendents	67	31	44	80	85	_	307
	Rural	*MDR Population	p=217	p=45	p=80	p=481	p=742	_
	Urban/Suburban	Superintendents	25	22	11	45	74	_	177
	Urban/Suburban	*MDR Population	p=34	p=29	p=12	p=83	p=321	_
	Rural	Special Education Director	66	34	44	79	84	_	307
	Rural	*MDR Population	p=207	p=51	p=81	p=426	p=669	_
	Urban/Suburban	Special Education Director	26	25	10	44	73	_	178
	Urban/Suburban	*MDR Population	p=36	p=33	p=11	p=80	p=298	_
	Rural	Curriculum Director	68	48	40	74	83	_	313
	Rural	*MDR Population	p=231	p=95	p=67	p=326	p=620	_
	Urban/Suburban	Curriculum Director	35	37	14	47	79	_	212
	Urban/Suburban	*MDR Population	p=54	p=60	p=16	p=91	p=431	_
	Rural	Testing/Assessment Director	53	21	23	67	74	_	238
	Rural	*MDR Population	p=118	p=27	p=30	p=222	p=318	_
	Urban/Suburban	Testing/Assessment Director	17	14	5	22	58	_	116
	Urban/Suburban	*MDR Population	p=20	p=16	p=5	p=28	p=145	_
	Rural	ELL/Bilingual Director	36	12	23	48	73	_	192
	Rural	*MDR Population	p=56	p=13	p=30	p=94	p=306	_
	Urban/Suburban	ELL/Bilingual Director	15	9	6	19	56	_	105
	Urban/Suburban	*MDR Population	p=17	p=10	p=6	p=23	p=133	_
	Rural	Principal/Asst. Principal	88	88	83	89	94	_	442
	Rural	*MDR Population	p=992	p=1,042	p=592	p=1,175	p=3,558	_
	Urban/Suburban	Principal/Asst. Principal	86	91	80	89	95	_	441
	Urban/Suburban	*MDR Population	p=790	p=1,681	p=487	p=1,116	p=12,541	_
	Rural	Teachers (all grades, all subject areas, all pre-school, elementary, middle, high)	96	96	95	96	96	_	479
	Rural	*MDR Population	p=20,988	p=19,985	p=9,620	p=23,216	p=73,760	_
	Urban/Suburban	Teachers (all grades, all subject areas, all pre-school, elementary, middle, high)	95	96	95	96	96	_	478
	Urban/Suburban	*MDR Population	p=15,212	p=35,462	p=9,355	p=22,692	p=260,189	_
	Rural/Urban/Suburban	Librarian	87	89	82	88	95	_	441
	Rural/Urban/Suburban	*MDR Population	p=913	p=1,131	p=535	p=1,030	p=6,242	_
Higher Education	Rural/Urban/Suburban	Higher Education 2 Year Colleges (College Department Chair, Dean, College Admissions Director, College Student Affairs Director)	81	82	79	84	93	_	419
	Rural/Urban/Suburban	*MDR Population	p=503	p=568	p=426	p=645	p=3,291	_
	Rural/Urban/Suburban	Higher Education 4 Year and Graduate Colleges (College Department Chair, Undergraduate/ Graduate Dean, College Admissions Director, College Student Affairs Director)	89	93	88	92	95	_	457
	Rural/Urban/Suburban	*MDR Population	p=1,291	p=3,318	p=1,019	p=2,227	p=10,041	_
Pre K-12 and Higher Education	Rural	Parents						1,125
	Urban/Suburban	Parents						1,125
	Ethnicity - Caucasian	Parents	75	75	75	75	75	375
	Ethnicity- Black/African-American	Parents	75	75	75	75	75	375
	Ethnicity- Asian/Pacific Islander	Parents	75	75	75	75	75	375
	Ethnicity- Hispanic	Parents	75	75	75	75	75	375
	Ethnicity- Native American	Parents	75	75	75	75	75	375
	Ethnicity- Multiracial	Parents	75	75	75	75	75	375
	Education Level - Less than High School	Parents						1,125
	Education Level - High School or more	Parents						1,125
	Total Parents	Parents							2,250
	Business Leaders in Rural/Urban/Suburban	Business Leaders	100	100	100	100	100	_	500
Total Surveys Completed									8,052

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield “reliable” data that can be generalized to the universe studied.

Expected response rate is 5%.

Maximizing response rates:

• Multiple email blasts will be sent if response rate is low.

• Email invitation will be kept short and will provide only one link to the survey.

• Email invitation will identify who we are and briefly define:

  • Purpose of study

  • Length of survey

  • Privacy statement

  • Short/concise instructions

  • Explanation of how we will use responses

  • Clarity of instructions, purpose and questions

• Respondents will be informed that research is being conducted to aid the Department of Education in fulfilling its education research objectives.

• Respondents will also be informed that participation in the survey is voluntary.

• All respondents will be informed about the confidential nature of this survey and data collection and will be assured of the usage of this information for research purposes only.

• Survey will include a limited number of questions to reduce time respondents will need to complete.

• Majority of questions will be closed/multiple choice with limited number of open-ended questions.

• The primary survey will be deployed via the internet. For identified constituents who do not have access to the internet, a phone survey may be deployed to meet quota targets. All data will be presented at 95% confidence level with +/-10% confidence interval.

Non-response issues:

• If response rates for any specific population segment are found to be low, then those specific segments will be contacted by a phone survey to collect their responses and complete survey quotas.

Accuracy and Reliability:

• Stratified random sampling will be done of the entire survey population to ensure collection of statistically projectable data that is representative of all educators, parents and business leaders involved in elementary, secondary and higher education in the southwest.

• Accuracy of sampling will be maintained by using educator contact information data from renowned and prestigious list providers such as Quality Education Data (QED) and Market Data Retrieval (MDR).

• Accuracy of data collection will be maintained by embedding multiple screeners and skip patterns within the survey. Doing so will help to ensure that only targeted respondents, who meet our selection criteria (such as professional title, current involvement in pre-K through Higher Education, etc.) are able to complete the survey.

• “Data cleaning” will be done when all data collection has been completed, and inconsistent or incomplete responses will not be included in the research analysis.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of data.

The pilot is tentatively scheduled for January 4, 2007.

No pilot or data collection with more than 9 people will occur until OMB project approval has been granted. Therefore, pilot date is subject to change. (Please see pilot and data collection schedule on page 13 of Supporting Statement, Part A.)

A pilot of the survey will be conducted prior to launching the survey to the entire sample. In this pilot, data will be collected from less than 9 respondents and results will be reviewed to “minimize burden and improve utility” by testing the average time it takes to complete the survey using the online version and testing among a range of titles, to ensure that questions are clear and understandable to the person taking the survey; and to help ensure that the online survey taking process runs smoothly and without technical problems.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Vendor name: Galloway Research

Vendor contact: John Ucab

Title: Director, Telephone Survey Department

Role: Responsible for monitoring client phone survey and ensuring that data collection is done according to client guidelines.

Phone: 210-734-4346

Vendor name: Market Data Retrieval

Vendor contact: Beth Shields

Title: Account Manager, Market Data Retrieval

Role: Responsible for providing contact information of educators, for data collection.

Phone: 312-345-4355

5