Att_1875-NEW 4559 OMB Supporting Statement B 6-10-11

E

POLICY STUDIES ASSOCIATES, INC.

POLICY STUDIES ASSOCIATES, INC.

VALUATION OF THE EDUCATION

FOR HOMELESS CHILDREN AND
YOUTH PROGRAM–SUPPORTING

STATEMENT FOR PAPERWORK REDUCTION

Part B: Collection of Information Employing

Statistical Methods

Prepared by:

Policy Studies Associates

1718 Connecticut Ave., NW

Suite 400

Washington, DC 20009

Prepared for:

U.S. Department of Education

Policy and Program Studies Service

Office of Planning, Evaluation, and Policy Development

March 29, 2011

Supporting Statement for Paperwork Reduction Act Submissions

B. Collection of Information Employing Statistical Methods

B.1. Respondent Universe and Sample Size

The U.S. Department of Education (ED) contracted with Policy Studies Associates (PSA) to administer two surveys: one to State Coordinators in each of the 50 states, the District of Columbia, and Puerto Rico and one to a sample of liaisons in local education agencies (LEAs) both that receive and do not receive Education for Homeless Children and Youth (EHCY) Program subgrants.

The population of LEAs that received regular EHCY Program subgrants in FY 2011 will provide the population from which the evaluators will draw the sample. The list of FY 2010 ARRA EHCY Program awardees will help to validate state survey responses on questions related to ARRA funding, and it will serve as an independent variable in statistical analyses to measure the effect on implementation of the additional ARRA funding.

In FY 2009, 2,477 LEAs reported receiving EHCY Program subgrants (EDFacts). Assuming a similar number of subgrantees in FY 2010-2011, PSA will draw a disproportionately stratified, representative sample of approximately 350 LEAs¹ to which evaluators will administer the LEA survey. Using disproportional stratification will ensure the inclusion of sufficient numbers of large LEAs in the analyses. Achieving a response rate of 85 percent requires responses from approximately 300 participating LEAs. Although evaluators hope to achieve a response rate of 100 percent, an 85-percent response rate will allow an estimate of population values with confidence intervals of approximately 5 percent and allow inferences about the population values with a confidence level of 95 percent and power of 75 percent. As necessary, the sample size will be adjusted for the number of subgrantees in FY 2010-2011.

Based on the selection of these 350 LEAs, evaluators will select a matched-comparison group of approximately 350 LEAs that did not receive EHCY Program subgrants in FY 2010-2011. The estimated total universe of non-grantee LEAs is 10,835 (see Exhibit 1).

Exhibit 1: Universe and Sample Sizes of Respondent Groups

	Universe	Sample	Expected response rate, in percents	Response rate of 2002 data collection, in percents
State Survey	52	52	95	95
LEA Survey Subgrantees Non-grantees	2,477* 10,835*	350 350	85 85	N/A N/A

* FY 2009 data from EDFacts.

PSA will use the National Center of Education Statistics’ Common Core of Data (CCD) Local Education Agency Universe Survey to stratify the population by geographic location and area population density (using the ULOCAL code) prior to drawing the sample. To match LEAs, the evaluators will use covariates identified through the CCD, including size and urbanicity of the LEA, poverty level, and percent of students in the LEA who belong to a racial minority. Using the EDFacts database, PSA will identify the number of children and youth experiencing homelessness in the LEA and use that figure in the matching process. Finally, PSA will attempt to match LEAs within states to the extent possible; however, in some cases this may not be possible because of the limited number and type of LEAs in particular states. Given the unique nature of some large LEAs, there will be few or no LEAs not receiving EHCY Program funds with which to match them. While surveying these large LEAs, the evaluators will exclude them from the process of matching with LEAs receiving no EHCY Program funds.

B.2. Procedures for Collection of Information

ED and the National Center on Homeless Education (NCHE) will assist in creating a list of state coordinators and their relevant contact information to take the state survey and, once a sample of LEA subgrantees and matched non-grantees have been selected, will assist with identifying the LEA liaisons and their contact information in subgrantee LEAs. PSA will collect contact information for the non-grantee LEAs from existing data collected through program monitoring and through searches of state websites. In the estimated 10 to 15 cases where neither of these searches provides the needed contact information, PSA will contact the state coordinators for liaison contact information in their state. This process will take about an hour per state coordinator.

Survey administration will begin in September 2011 after OMB clearance has been confirmed and after the school year has begun. Prior to conducting the surveys, state coordinator and district liaison respondents will be notified about the study via an introductory letter. PSA will also mail a study notification letter to Chief State School Officers and LEA superintendents to make them aware of the evaluation study and to enlist their support in ensuring a high response rate. The introductory and notification letters, signed by an ED official, will explain the evaluation and its importance to ED. They will also indicate that appropriate OMB clearance has been obtained to administer the survey and that ED’s EHCY program office supports this survey. In the case of states and grantee districts the letter will note that participation is mandatory. In the case of non-grantee districts the letter will note that ED’s EHCY program office encourages LEAs to respond.

Each recipient will receive a follow-up e-mail that again explains the study and contains an individualized link to the web-based survey. To secure the cooperation of respondents and to achieve a high response rate (85 percent or more for LEAs and 95 percent or more for states), respondents will be able to complete the state and LEA survey in approximately 35 minutes.

A week before the survey is due, PSA will mail a reminder postcard to non-respondents, with instructions for requesting another individualized link or a paper survey. A week after the due date, PSA again e-mail individualized links to all non-respondents, with instructions for requesting paper surveys. Two weeks later, PSA will mail paper surveys to non-respondents and will begin telephone follow-up. Each non-respondent will receive up to three telephone calls, asking him or her to complete the survey and offering to send replacement links or paper surveys. PSA staff will offer to walk respondents through the survey and collect responses by telephone. For those LEAs that still have not responded, PSA will contact the relevant state coordinators to enlist their assistance in urging LEA administrators to participate.

In February 2011 ED discussed the survey and evaluation at the state coordinators meeting to explain what is expected of state coordinators. State level grantees and district level subgrantees are required to participate in ED evaluations under ESEA, Sec. 9304(a)(4) and ESEA, Sec. 9306(a)(4), respectively.

B.3. Methods for Maximizing Response Rate and Dealing with Nonresponse

The data collection procedures described in Section B.2. include a follow-up plan to ensure a high response rate.

Each recipient will receive a follow-up e-mail that again explains the study and contains an individualized link to the web-based survey. To secure the cooperation of respondents and to achieve a high response rate (85 percent or more for LEAs and 95 percent or more for states), respondents will be able to complete the state and LEA survey in approximately 35 minutes.

A week before the survey is due, PSA will mail a reminder postcard to non-respondents, with instructions for requesting another individualized link or a paper survey. A week after the due date, PSA will again e-mail individualized links to all non-respondents, with instructions for requesting paper surveys. Two weeks later, PSA will mail paper surveys to non-respondents and begin telephone follow-up. Each non-respondent will receive up to three telephone calls, asking him or her to complete the survey and offering to send replacement links or paper surveys. PSA staff will offer to walk respondents through the survey and collect responses by telephone. For those LEAs that still have not responded, PSA will contact the relevant state coordinators to enlist their assistance in urging LEA administrators to participate.

In February 2011 ED discussed the survey and evaluation at the state coordinators meeting to explain what is expected of state coordinators. State level grantees and district level subgrantees are required to participate in ED evaluations under ESEA, Sec. 9304(1)(4) and ESEA, Sec. 9306(a)(4), respectively.

B.4. Test of Procedures and Methods

ED and NCHE identified six state coordinators to provide initial feedback on the state and district surveys for the evaluators. The evaluators spoke with these state coordinators at the state coordinators’ meeting in February 2011. During March and April 2011 evaluators piloted the LEA survey with less than nine district liaisons identified by ED and NCHE, while the survey was in the first stage of OMB review.

Participants in the survey pilot test were asked to track the time required to complete the survey and respond briefly to several questions related to item clarity and structure. The evaluators debriefed the participants by telephone to learn more about the respondent’s comprehension of each item’s text, definitions of key terms, precision and completeness of existing response options, and whether important questions are missing. The evaluation team analyzed participant responses item by item in a database and made survey revisions. They were incorporated into the final draft of the survey.

B.5. Consultations on Statistical Aspects of the Design

Name	Organization	Title	Phone
Mark Low	Policy Studies Associates	Research Associate	202-939-5316
Stephen Coleman	Policy Studies Associates	Senior Research Associate	202-939-5306

1 The exact number will depend on the number of LEAs receiving EHCY Program subgrants and the number of large districts that cannot be matched with a district not receiving EHCY Program funds.