This information collection requests clearance approval from the Office of Management and Budget (OMB) for the collection of data under the Evaluation of the Effectiveness of the Scholarships for Opportunity and Results (SOAR) Act Program. The foundation of this evaluation is a randomized control trial (RCT) comparing outcomes of eligible applicants (students and their parents) assigned by lottery to receive or not receive a scholarship. This design is consistent with the requirement for a rigorous evaluation as well as the need to fairly allocate the scholarships if the program is oversubscribed. Because the law also specified other kinds of comparisons and analyses, the planned evaluation study includes both quantitative and qualitative components. In order for the evaluation to have sufficient statistical power to detect policy relevant impacts, the sample consists of 1,771 eligible program applicants in spring 2012 (cohort 1; n=536), spring 2013 (cohort 2; n=718), and in spring 2014 (cohort 3, n=517. This is an extension of an approved information collection. The extension is requested to cover follow up data collection for cohort 2 as well as the addition of cohort 3 needed to obtain the sample size needed for the evaluation. There are no changes to the study design, data collection plans and instruments from what is approved by OMB in the current collection #1850-0800.
Evaluation data will be collected for the three cohorts of program applicants from a variety of sources listed below. Each cohort will have baseline data as well as three years of follow up (post-lottery) data collection; 2013-2015 for cohort 1, 2014-2016 for cohort 2, and 2015-2017 for cohort 3. In addition to estimating program impact, we will use this experimental study to conducted research about interim outcomes.
Data sources include:
--Student assessments: The Terra Nova assessment will be administered each spring following the lotteries for 3 years (spring 2013-2015 for the 2012 cohort, spring 2014-2016 for the 2013 cohort, and spring 2015-2017 for the 2014 cohort). The follow up assessments will be administered in studentsâ school and will provide the primary outcome measure for the impact evaluation. School records Administrative records will be collected from DCPS, the District of Columbia Public Charter School Board and participating private schools in the fall of each year to obtain data on prior year attendance, persistence, disciplinary actions, and grades for members of the treatment and control groups.
--Parent surveys: Annual surveys of evaluation sample membersâ parents in each follow up year. These surveys will examine such issues as reasons for continued participation or withdrawal, involvement in school, satisfaction with school choices, and perceptions of school safety, leadership, and offerings. The survey will be mixed mode. (Web with phone or paper follow up).
Student surveys: The surveys will be administered to each evaluation sample student in grades four and above in each of the follow up years at the same time (and place) as the student assessments.
--Principal surveys: Surveys to be administered to principals in the DC traditional public school, charter school, and private school systems in 2013-2017. Data from principals of students in the treatment and control groups will provide information about school organization and offerings for descriptive analyses of studentsâ school environments and for use as mediators in the impact analysis. The web-based principal surveys will also be used to examine how aware public and private schools are of the DC Opportunity Scholarship Program and whether they are making any changes in response to it.
PL:
Pub.L. 112 - 10 309
Name of Law: Scholarships for Opportunity and Results Act (SOAR) Program
This is an extension of a currrent collection. All of the forms and documents are the same as the currently approved collection. However, there are changes to the number of responses and burden hours due to a third cohort being added to the data collection in order to obtain the sample size needed to carry out this evaluation.
On behalf of this Federal agency, I certify that the collection of information encompassed by this request complies with 5 CFR 1320.9 and the related provisions of 5 CFR 1320.8(b)(3).
The following is a summary of the topics, regarding the proposed collection of information, that the certification covers:
(i) Why the information is being collected;
(ii) Use of information;
(iii) Burden estimate;
(iv) Nature of response (voluntary, required for a benefit, or mandatory);
(v) Nature and extent of confidentiality; and
(vi) Need to display currently valid OMB control number;
If you are unable to certify compliance with any of these provisions, identify the item by leaving the box unchecked and explain the reason in the Supporting Statement.
1850-0800 Appendix K OSP Consent Form.pdf
1850-0800 Appendix N_Confidentiality Agreement.docx
1850-0800 revOMB Part A (2-17-16).doc
Evaluation of the Effectiveness of the Scholarships For Opportunity and Results (SOAR) Program