ED Response to OMB Passback

PPSS Response to OMB Comments

High School Reform Study

February 4, 2015

1. How will the survey contribute to supporting evidence-based decision-making regarding high school strategies to increase graduation rates?

High schools use a diverse array of strategies to prevent dropouts and support students in completing high school. These strategies vary widely in types of programs offered, services provided, types of staff who provide the services, how students are targeted or selected for the program or service, average caseloads or class sizes, and frequency of contact with target students. As discussed by the expert panel that advised on the design of this survey, currently there is a paucity of information available on the prevalence and characteristics of various high school strategies to support at-risk students; this survey will help to fill this knowledge gap with consistent and reliable information based on a nationally representative sample of high schools. By enabling the Department, policymakers, researchers, and practitioners to understand the prevalence of various strategies and basic information about how schools are implementing the strategies, this study will help to inform policy related to high school program design, identify technical assistance and program priorities, and inform the design of future research on the effectiveness of frequently used strategies.

2. The survey seems very broad. The Department should consider ways to provide more in-depth information on the strategies.

In response to OMB’s comments, we have revisited the survey questions with the goal of obtaining more comprehensive information on the range of services and programs that schools are providing to high school students who are struggling academically or who may be at risk of dropping out. As part of this process, we revisited the input provided by the expert panel, met with staff for the Department’s High School Graduation Initiative, reviewed questions asked on the most recent NCES survey of dropout prevention services and programs (2010-11), and held multiple discussions of options between PPSS staff and the SRI research team.

In order to achieve high response rates for this respondent population, it is important to keep the survey length to approximately 30 minutes. Therefore, we looked for ways to eliminate some of the survey questions in order to make room for additional questions and keep the survey within a 30 minute timeframe.

First, we prioritized the strategies and dropped questionnaire sections on 3 of the original 16 strategies — positive school climate programs, flexible scheduling, and smaller learning communities — because these 3 strategies are often used for more general purposes rather than being focused on dropout prevention.

Second, we added questions for 8 of the remaining 11 strategies in order to obtain a more detailed understanding of their implementation characteristics and to gather more consistent information across strategies as appropriate. For example, the following types of questions were added:

Proportion of all high school students receiving service

• Flagged by early warning system

• Receiving personalized learning plan

• Participating in competency-based advancement

• Participating in career-themed curriculum

Staff providing services

• Who is involved in developing personalized learning plans

Services provided/needs addressed

• Types of services provided by case managers

• Information included in personalized learning plans

• How students demonstrate mastery for competency-based advancement

• Components of a career-themed curriculum approach

Table 1 below summarizes the 16 survey questions that were added to provide additional information on 8 of the strategies. Some of these questions were previously considered but had not been included due to concerns about the survey length (which we have now addressed differently by deleting 3 of the strategies from the survey).

Table 1: Summary of Questions Added to Survey, by Topic and Strategy

Topic	Case manager	Academic support classes	Academic tutoring	Student support teams	Early warning systems	Personalized learning plans	Competency-based advancement	Career-themed curriculum
Typical caseload/ class size		Class size (Q57)
How service allocated (all or some)							All or some students (Q105)
% students receiving service					% flagged (Q84)	% students receiving (Q99)	% students receiving (Q106)	% students receiving (Q110)
Staff/agency that provide service						Who develops (Q101)
Services provided/ needs addressed*	What case mgrs do (Q13)					What’s included in plans (Q102)	How students demonstrate competency (Q108)	Components of approach (Q112)
How often service is provided	How often (Q14)		How often provided (Q71)	How often they meet (Q82)	How often checked (Q91)	How often reviewed (Q103)

The net impact of these revisions increases the total number of survey questions from 108 to 113 (by dropping 11 items and adding 16 new items). However, respondents will not have to respond to all survey questions, only those for the specific strategies that they are currently implementing. Overall, we expect that the average survey response time will remain close to 30 minutes.

Table 2 below shows all of the implementation topics that are covered for all 13 strategies that are included in the revised survey. Survey additions are highlighted in blue.

In addition, we have also changed the name of the survey from “High School Reform Study” to “Study of High School Strategies to Improve Graduation Rates” to more closely reflect the purpose and content of the survey.

Topic

Case manager

Adult mentoring

Access to Social Services

Credit recovery

Accelerated academic programs

Academic support classes

Academic tutoring

Student support teams

Early warning systems

Middle to high school transition

Personalized learning plans

Competency-based advancement

Career-themed curriculum

Have/don’t have

Q5

Q15

Q25

Q33

Q42

Q53

Q62 (mandated)

Q75

Q83

Q93

Q98

Q104

Q109

Typical caseload/ class size

Q6

Q24

N/A

Q41

N/A

Class size (Q57)

Q69

N/A

N/A

N/A

N/A

N/A

N/A

How service allocated (all or some; voluntary or mandatory), participation requirements

Q7

Q16, Q22

N/A

Only some students

Q43, Q50

Q54

Only some students

Only some students

Q85 (derive all or some students)

Q95

Q99 (derive all or some students)

All or some students (Q105)

Q110

% of student offered service

N/A

N/A

N/A

Q34

Q44

N/A

N/A

N/A

NA

N/A

N/A

N/A

N/A

% students receiving service

Q8

Q17

Q28, Q32

Q40

Q46

Q55

Q63, Q74

Q76

% flagged (Q84)

Q96

% students receiving (Q99)

% students receiving (Q106)

% students receiving (Q110)

Which students targeted, what grades

Q9

Q18

N/A

Q35

Q45

Q56

Q64

Q77

Q85

Q97

Q100 Add what grades?

Q107 Add what grades?

Q111 Add what grades?

Staff/agency that provide service, info to match student to provider

Q10 Qualifications, Q12

Q19, Q21

Q27

Q39

Q49

Q59

Q65, Q67

Q80

Q86, Q87, Q89, Q90

N/A

Who develops (Q101)

N/A

N/A

Services provided/needs addressed*

What case mgrs do (Q13)

Q20

Q26, Q29, Q30, Q31

Part of the definition

Q51, Q52

Q60

Q72, Q73

Q78, Q79, Q80, Q81

Q85, Q86, Q88, Q92

Q94

What’s included in plans (Q102)

How students demonstrate competency (Q108)

Components of approach (Q112)

Location of service delivery/how delivered

Q11

N/AN/A

Q27, Q31

Q36, Q37

Q47, Q48

Q61

Q68, Q69

Q79

N/A

N/A

N/A

N/A

N/A

When service offered, how often

How often (Q14)

Q22, Q23

N/A

Q38

N/A

Q58

Q70

How often they meet (Q82)

How often checked (Q91)

Q94

How often reviewed (Q103)

N/A

N/A

How often provided (Q71)

Who pays for service

N/A

N/A

N/A

N/A

N/A

N/A

Q66

N/A

N/A

N/A

N/A

N/A

N/A

Table 2: Survey Coverage of Specific Implementation Topics for Various High School Strategies

Blue shading indicates new questions. All question numbers have been updated to reflect revised survey dated 2-3-15. Some questions may address more than one topic.