Part B Gallup survey Supporting statement final

C ontract #ED-06-CO-0023

SUPPORTING STATEMENT FOR REQUEST

FOR OMB APPROVAL OF

DATA COLLECTION/nEEDS ASSESSMENT

Part b

February 20, 2007January 31, 2007

Donna Chlopak
The Gallup Organization

Submitted to:		Submitted by:
	Institute of Education Sciences U.S. Department of Education 555 New Jersey Ave., N.W. Washington, DC 20208		Mid-continent Research for Education and Learning 4601 DTC Blvd., #500 Denver, CO 80237 Phone: 303-337-0990 Fax: 303-337-3005
Project Officer		Project Director:
	Sandra Garcia, Ph.D.		Louis F. Cicchinelli, Ph.D.

Collections of Information Employing Statistical Methods

1. Describe potential respondent universe and any sampling selection method to be used.

Respondents will be Curriculum Coordinators and Principals of schools from Pre-K through High School from any of the following 7 states: Colorado, Kansas, Missouri, Nebraska, North Dakota, South Dakota and Wyoming.

The contractor will create a sampling frame from a representative sample listing of all Curriculum Coordinators and Principals from these seven states and attempt to contact a sample of the universe through random sampling techniques. Each state will have a quota for both Principals and Curriculum Coordinators (25 per each per state). Potential respondents will be contacted at their schools and a convenient time for conducting the interview will be arranged. Most potential respondents are anxious to participate. The contractor will order the sample from Quality Education Data (QED)

The expected response rate for this survey must be calculated individually for both the Superintendent and Principal groups. In similar surveys we experienced a 31% response rate and expect similar or better this time. The 31% rate was obtained as follows: the contact rate was 80%; cooperation rate 39%; completion rate 99%; yielded overall response rate of .8 X .39 X .99 = 31%. 

2. Describe procedures for collection information, including statistical methodology for stratification and sample selection, estimation procedures, degrees of accuracy needed, and less than annual periodic data cycles.

Statistical methodology for stratification and sample selection: There will be random sample selection within each type of respondent (principal and curriculum coordinator) within each state. The contractor will order sample from Quality Education Data (QED).

The contractor will receive the entire population of Curriculum Coordinators and Principals from QED and then create random replicates of 50 for each population for each state. Each replicate will be fielded using a 5x5 design until completion, at which time the next replicate will be released.

Estimation procedures: The traditional means and proportions tests will be used to assess differences between the two populations and between states.

Degree of accuracy: The sample size design of 175 Principals will provide a degree of accuracy of ±7% and the sample of 175 Curriculum Coordinators will provide a degree of accuracy of ±7% .

Unusual problems requiring specialized sampling: There are no unusual problems associated with this data collection effort.

Use of periodic data collection: The same data collection process will be repeated in 2009.

3. Methods to maximize response rates

The contractor undertakes several procedures to ensure maximum response rates.

• The contractor will institute a 5x5 design such that respondents are called at least five times to identify and ensure that they are connecting with the correct respondent and then, up to an additional five calls are made to speak with the actual respondent. These calls will be made at various times and days or at a time the contact has designated.

• Replicates will be used so that people are not just chosen and then thrown away. With replicates, the interviewers can only call people assigned to the current replicate and the entire replicate must be exhausted before any “new” numbers are added into the system.

• Gallup’s interviewers are some of the best in the country. The contractor has one of the lowest turnover rates for interviewing staff and interviewers are hired for their ability to work very professionally and confidentially with respondents. As such, a dedicated team of interviewers will be assigned to this project. They will be the contractor’s experienced interviewers and they will go through a two-hour training session on this project. These interviewers are skilled at converting soft refusals. These interviewers do this by being flexible with the times that they schedule interviews to meet the respondent’s preferences and by understanding and communicating the value of the individual’s feedback.

• Phone surveying allows more control of non-response and a greater understanding of any potential non-response bias. Surveying by replicate supports the random sampling process , making the resulting information generalizable to the broader population.

• Gallup expects to show substantial improvement in the response rates we have obtained in similar studies of this population (and which were used as a basis of response rate baseline for this study).  We will send a prenotification letter to inform potential respondents that they will be contacted.  This is a well-established method of improving response rates and was not done in the past with the similar efforts Gallup has done on this population. For this effort we have suggested an increased call design from our standard, simple 5 call design to a 5+5 call design with up to five calls to reach the targeted respondent and then up to five additional calls to complete the interview with that respondent.  Since there is a well-established relationship between number of call attempts and response rates we expect increased response.  Finally, the field period will be moved from when schools are mostly out of session (as was the case in the past) to when schools are in session.  This should increase the probability of reaching the targeted respondents.

• In the event that a nonresponse bias investigation is needed, the sampling frame is fairly rich in the number of variables it provides so that the interviewed sample can be compared to the frame information.  For example, at the school district and school building level, we know the percentage of students that are Caucasian, African-American, Hispanic-American, Asian-American, and Native American as well as the percentage that are college bound, qualifying for free lunch, the grade span, enrollment, number of computers, etc.  At the personnel level, we know the tenure both in the profession and at their current institution, and job functions.  The interviewed sample can be compared to these frame variables for evidence of bias.  If necessary, weighting adjustments could be made to correct for disparities.  Also, any trustworthy data that is external to the frame could also be used to make comparisons with the interviewed sample (and if necessary, adjustments made).  Finally, we can also investigate the late responders and reluctant responders to see the extent to which they differ from the interviewed sample, for any evidence of bias as well.

• The interviewing progress will be monitored to see what type of response rate is obtained.  In the event that the response rates fall below expectations, Gallup can in consultation with McREL increase the number of call attempts to each record, extend the field period, consider offering incentives, consider additional mailing materials, refusal conversion calls, etc.  Another option (that is also useful for purposes of investigating nonresponse bias) would be to take a random subsample and target that group with additional efforts

4. Test of procedures or methods

The Gallup organization regularly conducts and has conducted similar surveys in the past and has tested the procedures and methods. Therefore, no additional testing will be conducted.

5. Contact names

Dr. Donna Chlopak
Project Director
The Gallup Organization
609-279-2261.

Dr. Manas Chattopadhyay
Chief Statistician
The Gallup Organization
202-715-3030.

Supporting Statement/McREL Study/Part B Page 1