Supporting_Statement_Part_B 060515

P art B: Supporting Statement for Paperwork Reduction Act Submission

Effectiveness of Promising Strategies in Federal College Access Programs: Study Student Messaging in GEAR UP

Prepared for:

Marsha Silverberg

U.S. Department of Education

555 New Jersey Ave, NW

Room 502I

Washington, DC 20208-5500

Submitted by:

Abt Associates Inc.

55 Wheeler Street

Cambridge, MA 02138

Part B: Supporting Statement for Paperwork Reduction Act Submission

Table of Contents

Part B: Supporting Statement for Paperwork Reduction Act Submission i

B. Collection of Information Employing Statistical Methods B-2

Introduction B-2

B.1 Respondent Universe and Sampling Methods B-7

B.1.1 Respondent Universe B-7

B.1.2 Sampling Methods B-7

B.1.3 Expected Response Rates B-7

B.2 Statistical Methods for Sample Selection and Degree of Accuracy Needed B-8

B.2.1 Sample Selection B-8

B.2.2 Estimation Procedures B-8

B.2.3 Degree of Accuracy Needed B-10

B.2.4 Unusual Problems Requiring Specialized Sampling Procedures B-12

B.2.5 Use of Periodic Data Collection Cycles to Reduce Burden B-12

B.3 Methods to Maximize Response Rates and Deal with Nonresponse B-12

B.4 Test of Procedures and Methods to be Undertaken B-13

B.5 Individuals Consulted on Statistical Aspects of the Design B-14

References B-15

2. Collection of Information Employing Statistical Methods

The U.S. Department of Education (ED) will conduct a demonstration to test the effectiveness of promising strategies to improve college-related outcomes in the federal college access program Gaining Early Awareness and Readiness for Undergraduate Programs (GEAR UP). The demonstration will use a randomized controlled trial (RCT) design to test the effectiveness of sending customized messaging to students, during the summer after high school graduation and in the fall and spring of their expected first year of college.

This study is being sponsored by ED’s Institute of Education Sciences (IES), in collaboration with the Office of Postsecondary Education, and implemented by Abt Associates Inc. and its partners, Survey Research Management (SRM), Program and Policy Insight (PPI), Digital Design Group (DDG), and consultant Ben Castleman (together, the “study team”).

Overview of the GEAR UP Program

Chapter 2 of Subpart 2 of Part A of Title IV of the Higher Education Act of 1965, as amended (HEA) authorizes the GEAR UP program. The program provides funding – over $300 million in FY 2015 – to state agencies and local partnerships to support disadvantaged students in obtaining a high school diploma and to prepare them for success in postsecondary education. Grantees are expected to provide a comprehensive set of intervention services such as academic support, counseling, mentoring, and outreach to disadvantaged middle and high school students, as well as information about college financial aid options and the benefits of postsecondary education to students and their families. State grantees may, and partnership grantees must, focus on a cohort of students (typically beginning in 7^th grade) and provide services to them through high school, though funds can be used to serve other students as well. State grantees are required to, and local partnership can, also provide scholarships to help participating students pay for college attendance.

Importantly, starting with the 2008 Higher Education Opportunity Act, GEAR UP grantees may now apply for a seventh year of funding to support students into their first year of college in order to improve college enrollment and persistence (referred to as the 7^th year). The first group of grantees with this funding, those awarded grants in FY 2011, will begin providing extended services to 2016-2017 seniors transitioning to postsecondary education the following year.

Overview of the Student Messaging in GEAR UP Demonstration

This demonstration is designed to address a new need among GEAR UP grantees for strategies to serve students during the 7^th year. According to recent focus groups conducted by Abt with GEAR UP grantees, grantees find it difficult to reach every GEAR UP student on a one-on-one basis while in high school, and to track and support students as they leave school and transition to college, particularly since students may be scattered across a variety of postsecondary institutions both in and out of state. Nationally, as many as 40 percent of first-generation, low-income high school seniors intending to go to college fail to enroll the following fall, with information gaps, logistical complexities, and concerns about fitting in identified as primary explanations of this “summer melt” (Castleman and Paige, 2014). Of those who matriculate, many students drop out during their first year or fail to re-enroll the second year (NSC, 2014b), and persistence is lower among students from low-income and high minority high schools (NSC, 2014a).

The demonstration will investigate promising approaches emerging from recent rigorous research that has investigated low-cost student messaging to improve college matriculation (Castleman & Page 2013) and college persistence (Castleman & Page, 2014), and targeted interventions to develop growth mindset or individuals’ beliefs that abilities, particularly academic abilities, can be developed through effort (Yeager, Walton & Cohen 2013). Outcomes for this study will include college enrollment, FAFSA completion, and persistence, which will be drawn from administrative data.

To test whether these approaches enhance the existing GEAR UP services, customized messages, delivered via a digital format (phone texting), will be sent to students identified as seniors, who intend to go to college, in high schools that are part of the GEAR UP program. Two independent treatments of student messaging will be tested; both treatments will consist of text messages sent to students during the summer that follows their senior year in high school and extending into their freshman year in college. Treatment 1 (milestone messages) will consist of text messages to remind students of key milestones related to college matriculation and college persistence, and direct students to where they can receive additional support, including from GEAR UP advisors. Treatment 2 (enhanced messages) will include the milestone messages plus messages designed to reinforce a growth mindset or an understanding that intelligence can be developed. In both treatments, students will be able to respond to the message to receive additional support from a counselor. All students will continue to be eligible for any regular GEAR UP services provided.

Students randomly assigned to Treatment 1 (milestone messages) will receive a minimum of 10 messages over the summer designed to relay information that students may not be aware of or may not have easy access to. The messages will inform students about key enrollment-related tasks they need to complete such as submitting a deposit, filling out the housing form, submitting their proof of health insurance form, registering for classes, etc. Students will receive a minimum of 12 messages during their freshman year designed to support students’ completion of their first semester and persistence into the second semester. These messages will focus on key tasks to complete such as paying tuition and registering for the next semester as well as common challenges faced in the first semester such as academic unpreparedness and lack of knowledge about campus supports.

Treatment 2 (enhanced messages) will consist of the milestone messages, plus additional messages that involve growth mindset messaging. These messages will consist of an activity that engages students with the idea that intelligence is malleable. Students randomly assigned to Treatment 2 will receive the milestone messages, plus approximately three additional growth mindset messages over the summer and an additional five during their freshman year.

All messages will be grounded in both social psychological and behavioral economic research suggesting that the messages be aligned to key time points such as housing deadlines, registration deadlines, tuition payment due dates, the add/drop period, and mid-term exams and that the messages be action oriented such as submit your housing form and go to this webpage to register for classes. Students will also be able to respond to the messages to receive support from a GEAR UP advisor.

To the extent possible, messages will be customized for each student based on the information they provide on the baseline survey about college intentions. For example, deadlines for submitting particular forms and links to find more information will be specific to the college a student indicated as the planned college of enrollment on their baseline survey.

Exhibit B-1 contains a preliminary outline of key prompts that are timed to key milestones or pressure points in enrolling and persisting in college.

Exhibit B-1. Draft Outline of Student Messages

Topic	Personalization
Treatment 1
Summer messages
Log into college's web portal to access important paperwork and deadlines	Link to specific website
Register for orientation and placement tests	Institution-specific registration deadlines
Complete housing forms	Link to specific forms, include deadlines
Sign up or waive health insurance (if relevant)	Link to specific forms, include deadlines
Submit deposit for fall enrollment	Institution-specific deposit deadlines
Register for fall classes	Institution-specific registration deadlines
Reminder of orientation and/or welcome activities dates	Institution-specific dates/activities
College messages
Provide links to campus resources for student support	Links to specific resources
Reminder about upcoming midterms	Link to campus specific academic support center
Mindset booster on overcoming challenges in college	Link to campus counseling center
Check financial aid eligibility for second year	Link to requirements
Register for spring classes	Institution-specific registration deadlines
Treatment 2 (all messages above, plus the enhancements below)
Summer message to introduce intelligence as something that can grow
Links to activities that promote growth mindset
Pre-midterm exams reminder that intelligence can grow through dedicated study
Post-midterm check on how exams went
Pre-final exams link to video about growing intelligence

Overview of the Evaluation

The study will use a rigorous random assignment design to test the effectiveness of sending customized messages to GEAR UP students that provide students with key information and allow students to respond for additional support. GEAR UP projects that were awarded awards in Federal Fiscal Year 2011 will be invited to volunteer for the demonstration (see Appendix A for invitation), because they are the first cohort of grantees to have funds to support a 7^th year of services. To test the effectiveness of the customized messaging, the study will assign students within participating high schools selected by volunteer GEAR UP projects to a control condition (eligible for all GEAR UP services), or one of two treatment conditions (eligible for all GEAR UP services, plus receiving customized text messages). Students will be randomly assigned to a condition at the end of their senior year of high school. This two-treatment design maximizes our ability to investigate the effects of messaging associated with key college milestones, as well as messaging that includes a social-psychological enhancement.

At the end of the 2016-2017 school year, students in high schools served by GEAR UP projects who declare their intention to enroll in college will be randomly assigned to receive only regular GEAR UP 7^th year services (Control), milestone messages in the summer after high school and freshman year of college (Treatment 1), or milestone messages plus growth mindset messages in the summer after high school and freshman year of college (Treatment 2). The comparison of the Control group to each Treatment group will test the impacts of messaging over and above the regular 7^th year GEAR UP activities. The comparison of Treatment 1 to Treatment 2 will test whether the growth mindset addition to the messaging has an effect above that of the milestone messages alone. Exhibit B-2 displays the groups into which students will be randomly assigned to test the impacts of different messaging on college outcomes.

Exhibit B-2: Randomization into Treatment Groups

Randomization	College Intending Seniors
End of 2016-17 school year	C1	T1	T2
	- Regular GEAR UP services	- Regular GEAR UP services	- Regular GEAR UP services
		- Milestone messages	- Milestone messages
			- Growth mindset messages

The Annual Performance Reports (APR) that GEAR UP grantees submit to ED do not contain student-level data. Thus, baseline data will be collected from all students via a survey, and college-related outcome data will be extracted from national datasets (National Student Clearinghouse Data (NSC) and the Federal Student Aid (FSA) database).

The study will address the three research questions:

1. To what extent do the messages—above and beyond the services GEAR UP may already provide—improve student college-related outcomes? Is the effect of milestone messaging different with the addition of growth mindset messages?

2. What types of college advising are typically received by GEAR UP students?

3. Is there variation in the impact of the messages, and to what extent is the variation associated with student characteristics or features of the GEAR UP project or school?

To answer these questions we will conduct both impact and descriptive analyses. The first report, which will address each research question with a focus on college enrollment-related outcomes, will be published in 2018. The final report, which will also include college persistence, will be available in 2020.

Exhibit B-3 presents the research questions along with the data sources for each question, the analytic approach and outcomes of interest.

Exhibit B-3. Evaluation Questions, Data Sources, Analytic Approach, and Outcomes of Interest

Research Question	Data Sources	Analytic Approach	Outcomes of Interest
To what extent do the messages—above and beyond the services GEAR UP may already provide—improve student college-related outcomes? Is the effect of milestone messaging different with the addition of growth mindset messages?	Student rosters Student baseline survey data National Student Clearinghouse Data Federal Student Aid (FSA) database	Impact analysis of RCT (HLM) Baseline survey sample: 16,000 students in 80 schools. Analytic sample size: approximately 5,600 students in 80 high schools that intend to enroll in college, equally divided across treatment and control groups.	College enrollment College persistence into sophomore year FAFSA completion
What types of college advising are typically received by GEAR UP students?	Student baseline survey data	Descriptive analysis Survey sample: 16,000 students in 80 schools.	Description of GEAR UP program components provided and received
Is there variation in the impact of the messages, and to what extent is the variation associated with student characteristics or features of the GEAR UP project or school?	Student baseline survey data National Student Clearinghouse Data Federal Student Aid (FSA) database	Descriptive (subgroup) analysis Sample size: 5,600 students in 80 high schools that intend to enroll in college	College enrollment College persistence into sophomore year FAFSA completion

Altogether, we expect that 80 GEAR UP high schools will participate in the research project, with an estimated total of 16,000 seniors.¹ All seniors will be surveyed, with the consent process giving them (or the parents) an option to opt out of completing the baseline survey or other data collection activities. The survey will gather information from students for participation in the student messaging intervention (see Appendix C for the consent letter and Appendix D for the student survey).

1. Respondent Universe and Sampling Methods

   1. Respondent Universe

The respondent universe includes the FY 2011 GEAR UP grantees and associated schools that volunteer for the demonstration. Up to 12 GEAR UP grantees will be recruited involving approximately 80 schools. To participate in the demonstration, schools will need to agree to allow students to be randomly assigned to the intervention. The seniors in these schools in the 2016-2017 academic year who intend to go to college and who do not opt-out through the consent process will be the target population.

As noted above, an estimate of the number of seniors in each school is 200, given that the average secondary school enrollment in 2011-2012 was 788 students (NCES, 2013).² Of these students, we expect 70 percent to express intent to attend college, ³ and 50 percent of these to assent to receive text messages, resulting in a total of 5,600 students in the study with 1,866 students in each group.

Exhibit B-4. Population Size Estimates

Estimated number of GEAR UP high schools	80 schools
Estimated number of 12th graders in GEAR UP cohort per school	200 students
TOTALS High schools	80
12th graders in GEAR UP high schools	16,000
12th grade GEAR UP demonstration participants	5,600

2. Sampling Methods

There will be no sampling of schools for the evaluation. The entire universe of GEAR UP schools within the demonstration grants who have agreed to random assignment will participate in the evaluation study. All 12^th grade students in these schools will be asked to participate in the evaluation. Those who assent to participate in the intervention and intend to go to college will be involved in the impact study.

3. Expected Response Rates

We expect to obtain a 100 percent response on the rosters from schools. We expect an 85 percent response rate on the baseline survey. Outcome data including college matriculation and persistence will be drawn from extant NSC data. The NSC includes more than 3,600 institutions enrolling 98 percent of all students in public and private U.S. institutions (NSC 2014c). The primary reason enrolled students are not found in the NSC is because the student suppressed his/her data; for the academic years of 2006-2007, 2008-2009, and 2010-2011, this rate was 4.7 percent (NSC 2012).

2. Statistical Methods for Sample Selection and Degree of Accuracy Needed

   1. Sample Selection

Random assignment will take place at the end of the 2016-2017 school year; students in high schools served by GEAR UP projects who declare their intention to enroll in college will be randomly assigned to receive milestone messages in the summer after high school (Treatment 1) or to receive milestone messages and growth mindset messages (Treatment 2) or to receive no messages (Control 1).

2. Estimation Procedures

Impact and descriptive analyses will be conducted to answer the study research questions, as described below.

1. To what extent do the messages—above and beyond the services GEAR UP may already provide—improve student outcomes? Is the effect of milestone messaging different with the addition of growth mindset messages?

Given that participating schools are volunteering to participate in the study (rather than being selected at random) and inferences will be focused on students in these or similar schools, the study will use a model with fixed treatment effects to assess the effect of messaging on student outcomes (Schochet, 2008).⁴The study team will control for student demographic characteristics, including gender, race/ethnicity, and first generation college student status. Reflecting the randomization of students within schools, the study team will include a set of dummy variables for schools and a treatment indicator to provide an estimate of the average impact of the messaging demonstration on students across all schools.

The study team will conduct a test of the null hypothesis of no treatment impact for each treatment on each outcome variable. ⁵ The study team will interpret a positive and statistically significant impact at the 5-percent level (using a two-tailed test) as compelling scientific evidence that the intervention improves the targeted outcome (e.g., college enrollment, college persistence, FAFSA completion). For continuous outcome variables, the study team will estimate multilevel linear models; for dichotomous outcomes (e.g., college enrollment, FAFSA completion), the study team will estimate multilevel logistic regression models.

1. Multilevel Logistic Regression

For dichotomous outcomes (e.g., enrollment, persistence, and FAFSA completion), a logistic regression will be used. This model is:

1. 

Where is the covariate-adjusted log-odds of the outcome occurring (e.g., enrolled in college, completed FAFSA) versus not occurring (e.g., did not enroll in college, did not complete FAFSA) for students in school j and is a regression coefficient indicating the difference in log-odds of the outcome between the treatment and control group, controlling for all other variables in the model..

To convert estimates produced by logistic regression to an impact measured in percentage point increases in probability of the binary outcome, we calculate the individual level probability of the outcome given treatment implied by individual level covariates and model estimates, we repeat this calculation for individuals assigned to the control group, and we find the mean difference between these two probabilities.

2. Hierarchical Linear Model

The following model will be used to estimate program impacts on continuous outcome variables. Given that our core outcome measures (i.e., confirmatory contrasts) are all dichotomous (i.e., college enrollment, FAFSA completion, and persistence into the second year of college), this HLM describes our methodology for assessing exploratory contrasts.

Y_ij = ₀ + ₁ (T_ij) + ₂ (Y^*_ij) + _{2+j j} + _{2+(J-1)+m mij} +

where Y_ij is the value of the outcome (e.g., amount of Federal financial aid received) for the i^th student in the j^th GEAR UP school; T_ij is 1 if student i is randomized to the treatment condition (the impact of each treatment will be modeled) in schools j and 0 if assigned to the control condition in school j; Y^*_ij is the baseline score for the outcome of interest for student i in school j; Sch_j = 1 if student is in school j, =0 else; _mij are additional covariates representing demographic characteristics of student i at school j (e.g., race/ethnicity, English language learner status); ₀ is the conditional mean follow-up score for the outcome of interest for control students; is the treatment effect i.e. the mean difference of the outcome between intervention and control students. The overall treatment estimate ( ₁) will be a precision-weighted estimate, where the weights are inversely proportional to the variance of the treatment effect in each school. is the residual error term assumed to be normally distributed with mean 0 and variance ².

2. What types of college advising are typically received by GEAR UP students?

Descriptive analyses will be used to provide information on the types of college advising GEAR UP students typically receive. It is important to understand the existing college advising to which the messaging will be added. Further, this collection and analysis has the added benefit of filling a gap in program information about the intensity of and emphasis on college advising. The study team will describe the types and intensity of college advising regularly received by GEAR UP students at the high school level as reported on the student survey. One example of the type of college advising that might be received by students is one-on-one counseling or advising sessions about applying to college; the study team would describe the percent of students attending such sessions. These results will provide another important source of contextual information for interpreting impact findings.

2. Is there variation in the impact of the messages, and to what extent is the variation associated with student characteristics or features of the GEAR UP project or school?

To examine variation in program impacts by student characteristics or features of the GEAR UP project or school, the study team will include an interaction of the treatment indicator with the subgroup variable of interest in the impact models described above.⁶ The study team will include interaction terms in the model for our core dichotomous outcomes of college enrollment, FAFSA completion, and college persistence (i.e., equation 1 above), allowing the treatment impact to vary by student characteristics, such as knowledge of the college application process, understanding of financial aid options and college costs, or amount of college advising received (e.g., one-on-one counseling or advising sessions about applying to college ) – or by program characteristics such as whether the GEAR UP program was funded under a state grant or a partnerships grant. If significant interaction effects are found, the sample will be broken into subgroups, the treatment effect will be estimated for each subgroup, and the means and variances of the subgroup will be compared.

3. Degree of Accuracy Needed

This section presents the power analyses conducted to estimate the minimum detectable effect sizes (MDES) for the treatment/control differences on student outcomes.

The power analysis conducted utilizes a slightly modified⁷ version of the MDE formula provided by Schochet (2008) for an RCT design that entails random assignment of students within schools and the corresponding 2-level impact model:

Where

: estimated minimum detectable effect size for the treatment impact;

: a constant that is a function of the significance level (α), statistical power (β), and the number of degrees of freedom (df);

s: number of schools;

m: average number of students in each school;

p: proportion of students assigned to the treatment condition;

: student-level variance in the outcome;

: proportion of student-level variance in the outcome explained by covariates;

σ: standard deviation of the outcome measure.

Using the equation above we make the following assumptions:

1. Randomization of students within schools to treatment or control conditions

2. Student level outcome of interest is measured on a continuous scale

3. 80 percent power

4. Two-tailed testing at the 5 percent level (alpha level=0.05)

5. The demonstration includes s=80 schools

6. The demonstration includes 70 students per school

7. 23 students are assigned to each group (p=0.5)

8. The variance of the outcome across students (σ²_students) = σ=1 (because of standardization)

9. The proportion of variation explained by student level covariates ( ) is 0.1

Because our key outcomes (i.e., college enrollment, FAFSA completion, and college persistence) are all dichotomous variables, the power calculations present the minimum detectable effect size (MDES) expressed as a difference in percentage points between successes in the treatment and control groups using the following formula:

Conversion of MDES to MDE.PP

Since the standard deviation of a proportion (P) is and MDES are expressed in standard deviation units, we used the following formula to convert MDES to MDE.PP (the MDE expressed as a difference in percentage points between successes in the treatment and control groups):

; where the BaseRate is the rate (percent) of (percent) of success in absence of treatment (i.e. success in the control condition.

Exhibit B-3 shows the minimum detectable effect size (MDES) we would be powered to detect with 80 schools and 70 students per school (23 in each group) would be approximately 2-4 percentage points depending on the base rate. Assuming a college matriculation rate of high school seniors of 50 percent⁸, the MDE is 4.6 percentage points. ⁹

Exhibit B-5: Power Calculations for Dichotomous Outcomes

Base Rate (success in absence of treatment i.e. success in the control condition)	MDE expressed as a difference in percentage points between successes in the treatment and control groups
10	0.028
25	0.040
50	0.046
60	0.045
75	0.040
86	0.032

4. Unusual Problems Requiring Specialized Sampling Procedures

Unusual problems that require specialized sampling procedures are not anticipated.

5. Use of Periodic Data Collection Cycles to Reduce Burden

The data collection plan reflects sensitivity to issues of efficiency and respondent burden. In this study, the study team planned data collection at the fewest intervals possible to reduce burden and ensure the quality of the data. The student rosters will only be collected once. Student surveys will be fielded once during this study. Administrative records, rather than surveys, will be used to gather data on outcomes.

3. Methods to Maximize Response Rates and Deal with Nonresponse

In order to obtain responses from the study sample, the study team has developed strategies to facilitate communication with respondents during data collection activities and to maximize response rates. These strategies have proven successful in the study team’s extensive experience conducting large-scale evaluation studies (e.g., Reading First Impact Study, Evaluation of the U.S. Department of Education’s Student Mentoring Program, Evaluation of the Massachusetts Expanded Learning Time Initiative; Enhanced Reading Opportunities Study, Career Academies Evaluation, The Teacher Incentive Fund Evaluation, and Study of Enhanced College Advising in Upward Bound).

The study team will follow several procedures to maximize response rates and handle nonresponse.

Student rosters. During the August 2016 meeting of the association of GEAR UP grantees organized by the National Council for Community and Education Partnerships, the study team will provide an evaluation session for participating project directors and school coordinators that will clearly explain the importance of the study and the roles and responsibilities of project directors and school coordinators, including the process for submitting student rosters. During the 2016-2017 school year, each school will be assigned a study liaison to answer questions and explain the demonstration requirements. Next, the liaison will send an email to each site coordinator explaining the request that will be made of site coordinators (see Appendix B). School coordinators will be asked to submit rosters of 12^th grade students through a password protected secure file transfer portal (SFTP). One week after sending out the email request, a member of the study team will reach out to each coordinator via a telephone call to answer any questions about the roster submission. The submission of student rosters via a password protected SFTP will allow coordinators to submit rosters at a time most convenient for them and will assure coordinators that student confidentiality is protected as permitted by law.

Baseline student surveys. The baseline student surveys will be administered in electronic or paper format. To assist data collection, the site liaison will work with each GEAR UP school to determine whether in-school administration is feasible and to: facilitate the logistics for the baseline student survey data collection; track response rates per project and communicate with project directors, as needed; remind students about the survey; and distribute individualized thank you or reminder letters to students near the end of the survey field period. The study team will make the survey available in multiple modes (electronic and paper) and individual follow-up will be conducted to reach students who were absent during in-person administration.

4. Test of Procedures and Methods to be Undertaken

We do not believe pretesting is necessary for the collection of student rosters, as the study team will be following a procedure that has been successful in studies such as the Evaluation of Citizen Schools: School-Level Expanded Learning Time and the Study of Enhanced College Advising in Upward Bound.

The baseline survey was pilot tested during the 60-day comment period with five respondents who were seniors in GEAR UP high schools. The survey took nine minutes on average to complete. In response to review and pilot testing, the questions about educational expectations were moved to after those about college plans, the number of question on academic preparation was reduced, and a question related to beliefs about growth mindset was added.

5. Individuals Consulted on Statistical Aspects of the Design

The following individuals were consulted on the statistical aspects of the study:

Name	Title/Affiliation	Telephone
Mr. Cristofer Price	Principal Scientist, Abt Associates	301-634-1852
Dr. Alina Martinez	Principal Associate, Abt Associates	617-349-2312
Dr. Rob Olsen	Principal Scientist, Abt Associates	301-634-1716
Dr. Allan Porowski	Principal Associate, Abt Associates	301-634-1765
Dr. Anne Wolf	Associate, Abt Associates	301-634-1738
Ms. Amanda Parsad	Senior Scientist, Abt Associates	301-634-1791

The following individuals will be responsible for the Data Collection and Analysis:

Name	Title/Affiliation	Telephone
Dr. Alina Martinez	Principal Associate, Abt Associates	617-349-2312
Mr. Cristofer Price	Principal Scientist, Abt Associates	301-634-1852
Dr. Rob Olsen	Principal Scientist, Abt Associates	301-634-1716
Ms. Amanda Parsad	Senior Scientist, Abt Associates	301-634-1791
Dr. Tamara Linkow	Associate, Abt Associates	617-520-2978
Ms. Linda Kuhn	President, Survey Research Management	303-247-0140

References

Aronson, J., Fried, C., & Good, C. (2002). Reducing the effects of stereotype threat on African American college students by shaping theories of intelligence. Journal of Experimental Social Psychology, 38, 113-125.

Aud, S., Hussar, W., Johnson, F., Kena, G., Roth, E., Manning, E., Wang, X., & Zhang, J. (2012). The condition of education 2012 (NCES 2012-045). U.S. Department of Education, National Center for Education Statistics. Washington, DC. Retrieved June 27, 2012 from http://nces.ed.gov/pubsearch.

Bailey, M. J., & Dynarski, S. M. (2011). Inequality in postsecondary education. In G. J. Duncan and R. J. Murnane (Eds.). Whither opportunity? Rising inequality and the uncertain life chances of disadvantaged children. New York, NY: Russell Sage Foundation.

Bowen, W. G., Chingos, M.M., & McPherson, M.S. 2009. Crossing the Finish Line: Completing College at America’s Public Universities. Princeton, NJ: Princeton University Press.

Castleman, B. L. & Page, L. C. (2013). The not-so-lazy days of summer: Experimental interventions to increase college entry among low-income high school students. New Directions for Youth Development, 140, 77-97.

Castleman, B.L., & Page, L.C. (2014). Freshman year financial aid nudges: An experiment to increase FAFSA renewal and college persistence. EdPolicy Works Working Paper. Available http://curry.virginia.edu/uploads/resourceLibrary/29_Freshman_Year_Financial_Aid_Nudges.pdf

Castleman, B.L., Page, L.C., & Schooley, K. (2014). The forgotten summer: Mitigating summer attrition among college-intending low-income high school graduates. The Journal of Policy Analysis and Management. DOI: 10.1002/pam.21743

Darling, R. & Smith, M. (2008). First-generation college students: First-year challenges. In M. S. Hunter, B McCalla-Wriggins, & E. White (Eds) Academic Advising: New Insights for Teaching and Learning in the First Year. National Resource Center for the First-Year Experience and Students in Transition, Columbia SC.

Goyette, K. A. (2008). College for some to college for all: Social background, occupational expectations, and educational expectations over time. Social Science Research, 37(2), 461-484.

Haskins, R. (2008). Education and economic mobility. In J. B. Isaacs, I. V. Sawhill, and R. Haskins (Eds.). Getting ahead or losing ground: Economic mobility in America. Washington, DC: The Brookings Institution. Retrieved June 27, 2012 from http://www.brookings.edu/research/reports/2008/02/economic-mobility-sawhill.

Kao, G., & Tienda, M. (1998). Educational aspirations of minority youth. American Journal of Education, 106(3), 349-384.

Liu, A., Sharkness, J., & Pryor, J. (2008). Findings from the 2007 Administration of Your First College Year (YFCY): National Aggregates. Higher Education Research Institute, University of California, Los Angeles. Available http://www.heri.ucla.edu/PDFS/YFCY_2007_Report05-07-08.pdf

National Center for Education Statistics (2012). Digest of Education Statistics, 2011. Washington, DC: US Department of Education.

National Center for Education Statistics (2013). Common Core of Data (CCD), "Public Elementary/Secondary School Universe Survey," 2011-12. http://nces.ed.gov/programs/digest/d13/tables/dt13_216.80.asp

National Student Clearinghouse (2012). Impact of Directory Information Blocks on StudentTracker Results. Available http://nscresearchcenter.org/wp-content/uploads/NSC_Directory_Block_Rates.pdf

National Student Clearinghouse (2014a). Report: High School Benchmarks 2014, National College Progression Rates for high schools participating in the National Student Clearinghouse Student Tracker Service. Available http://nscresearchcenter.org/hsbenchmarks2014

National Student Clearinghouse (2014b). Snapshot Report: First-Year Persistence and Retention Rates by Starting Enrollment Intensity: 2009-2012. Available: http://nscresearchcenter.org/snapshotreport-persistenceretention14/

National Student Clearinghouse (2014c). Clearinghouse Facts. Available http://www.studentclearinghouse.org/about/clearinghouse_facts.php

Roderick, M., Nagaoka, J., Coca, V., & Moeller, E. (2009). From High School to the Future: Making Hard Work Pay Off. Chicago: Consortium on Chicago School Research.

Schochet, P. (2008). Statistical Power for Random Assignment Evaluations of Education Programs. Journal of Educational and Behavioral Statistics, 33(1), 62-87.

U.S. Department of Education. (2014). Gaining Early Awareness and Readiness for Undergraduate Programs (GEAR UP) Funding Status. Retrieved on January 13, 2015 from: http://www2.ed.gov/programs/gearup/funding.html

Walton, G. & Cohen, G. (2011). A brief social-belonging intervention improves academic and health outcomes of minority students. Science, 331, 1447-1451.

Yeager, G., Walton, G., & Cohen, G. (2013). Addressing achievement gaps with psychological interventions. Kappa, 94, 62-65.

Yeager, D.S., Johnson, R., Spitzer, B., Trzesniewski, K., Powers, J., & Dweck, C.S. (2014). The far-reaching effects of believing people can change: Implicit theories of personality shape stress, health, and achievement during adolescence. Journal of Personality and Social Psychology. 106(6), Jun 2014, 867-884.

1 An estimate of the number of seniors in each school is 200, given that the average secondary school enrollment in 2011-2012 was 788 students (NCES, 2013).

2 The estimate of the number of seniors in GEAR-UP schools is based on the average enrollment of 788 in regular secondary schools in 2011-2012, assuming four grades per school (NCES 2013).

3 Among 2010 seniors, 82 percent of graduating students planned to attend a postsecondary institution and the percent among students whose parents had no college education was 71 percent (Aud et al. 2012).

4 If we suspect that treatment effects differ by school, we will interact schools with the treatment indicator.

5The study team will also test the null hypothesis of no difference between Treatment 1 (milestone messages) versus Treatment 2 (enhanced messages).

6 This approach requires the assumption that each characteristic is not correlated with the marginal effect of the other model covariates. We will evaluate whether this assumption is tenable during preliminary data exploration.

7The Schochet (2008) formula was modified to include the term.

8 This is the immediate college enrollment rate in the first fall after high school graduation for students graduating from low income schools according to the National Student Clearinghouse Research Center (National Student Clearinghouse 2014a).

9 This detectable difference is similar to the 4 percentage point difference found in college enrollment (Castleman, Page & Schooley 2014) and smaller than the 12 percentage points difference in persistence rates into the fall of sophomore year between community college freshmen who received a text intervention and those that did not receive the intervention, (Castleman & Page 2014).