1875-NEW PPSS_MEP evaluation OMB Submission 30 Day-Part A_CLEAN_09-07-17

The Policy and Program Studies Service (PPSS), of the U.S. Department of Education’s Office of Planning, Evaluation and Policy Development, requests Office of Management and Budget (OMB) clearance for the design of the Study of the Implementation of the ESEA Title I—Part C Migrant Education Program (MEP). The study is being carried out under the authority of the Elementary and Secondary Education Act (ESEA) as reauthorized by the Every Student Succeeds Act (ESSA), Public Law 114-95 (12/10/2015), section 8042.

PPSS is requesting clearance to (a) collect data about subgrantees that receive MEP funds from state education agencies; (b) collect data from state MEP directors and local MEP coordinators about program implementation using an electronic, online survey; and (c) collect data via interviews from MEP leaders from 10 states, 20 districts within those 10 states, and 40 schools or projects in the 20 districts.

A previous information collection package has been submitted to OMB and is currently in clearance for the recruitment phase and study design for this MEP collection. This second information collection is requesting clearance for the study instruments for the MEP study.

This package contains two major sections:

Supporting Statement for Paperwork Reduction Act Submission

Justification (Part A)
Description of Statistical Methods (Part B)

Appendices

Appendix A—Notification letters
Appendix B—State director survey instrument
Appendix C—Local coordinator survey instrument
Appendix D— State director interview protocol
Appendix E—Regional coordinator interview protocol
Appendix F—Local coordinator interview protocol
Appendix G—School coordinator interview protocol

A. Justification

1. Circumstances that make the collection of information necessary

Migratory students represent one of the most disadvantaged student groups in the United States. Migratory students are more likely to live in poverty, experience disconnected educational experiences, and have health problems often associated with poverty that can have a negative impact on students’ educational progress and success. Congress established the Migrant Education Program (MEP) in 1966 under an amendment to the 1965 Elementary and Secondary Education Act (ESEA). Since that time, the program has evolved. For example, program changes now require states to conduct and use comprehensive needs assessments and statewide evaluations to inform their service delivery plans, and the program replaced the Migrant Student Record Transfer System with the Migrant Student Information Exchange (MSIX) to facilitate the seamless transfer of critical student background and performance information across district and state lines. The 2015 reauthorization of the ESEA under the Every Student Succeeds Act (ESSA) brought still more changes to program requirements. For example, new language on Priority for Services (PFS) is likely to affect how many and which migratory students are identified and served under PFS criteria. Earlier ESEA language on PFS referred to children whose education was interrupted during the school year; new ESSA language refers to those who have moved within the previous one-year period or have dropped out of school.

As the program changed over time, the population served also changed. Between 2001–2002 and
2013–2014, the number of eligible migrant students dropped 69 percent from 872,732 to 347,634;
out-of-school youth (OSY) receiving MEP services fell from nine percent of the eligible migrant student population to six percent; and students identified as PFS fell from 42 percent of migrant students served to 22 percent. At the same time, the percentage of all migrant students identified in the three states with the largest migratory populations remained constant at 53 percent. Enrollment declines reflect changes in migratory patterns and in agricultural and labor practices, including increased use of H2-A visas for agricultural work and tighter identification and recruitment practices in response to both Office of Inspector General (OIG) findings^¹ and Office of Migrant Education (OME) guidance. The declining percentage of migrant students identified as PFS reflects increased accountability for prioritizing services to students identified as PFS.

State and local services and supports for migratory children and youth have reflected the changes in MEP program requirements and the migrant student population. An internal review of MEP evaluation reports from a variety of states (e.g., Florida, Kentucky, New York, Maine, and Mississippi) identified numerous changes in states’ approaches to MEP. New York, for example, reconfigured its regional projects to focus on specific strategies for meeting its Measurable Program Outcomes. Kentucky reconfigured its regional and statewide summer programs to focus on the statewide academic needs identified through its Comprehensive Needs Assessment. Iowa modified its credit accrual and interstate communication practices to address increased return migration from Texas. For systematic analysis of such responses, it is necessary to go beyond current federal data collections (e.g., Comprehensive State Performance Reports, MSIX). New requirements also deserve analysis, notably the ESSA requirement that state and subgrantees align MEP strategies with evidence-based practices. This data collection request supports a national study of MEP that will describe how state and local MEP-funded programs have evolved in response to changing requirements, and the extent to which and how they are serving the needs of a changing migratory student population.

Purpose of the Study

The Migrant Education Program provides funding to assist migratory children in meeting the same standards as other children and help them graduate from high school or a high school equivalency program with an education that prepares them for responsible citizenship, further learning, and productive employment. This study is designed to examine and describe how states, districts, and schools implement programs for migratory children and youth, including coordination of services to address the unique needs of migratory students.

Exhibit 1 shows the study questions and the data sources planned to answer each question. The study is using existing data and information when available. The study team has completed a review and analysis of the literature and of extant data to support the study and found that available data and literature do not provide complete answers to any of the study questions and provide no information about others. The study will address these gaps in the research through surveys of State Directors of Migrant Education and local MEP coordinators and case study interviews with program coordinators and staff in MEP-funded states, local grantee sites, and schools or local MEP-funded projects. The original data collection activities are designed to be complementary. The surveys will provide information on the full population of state and local MEP grantees while the case study interviews will provide detailed and nuanced data from a small subset of states, local sites, and schools or projects.

Exhibit 1. Study questions and data sources

	Literature and extant data review	State survey	Local survey	Case study
(RQ1) How do state and local grantees respond to federal requirements for serving migratory children?
How does ESSA’s expanded definition of migratory students influence state and local practices to identify eligible migratory students?				
How are grantees prioritizing services to migratory students who moved within the year, are failing, or dropped out of school?				
How are states addressing the unique needs of migratory students in designing their accountability systems, both for MEP and for ESSA overall?				
How do migrant programs identify and serve eligible out-of-school youth?				
How do state and local MEP staff coordinate with other OME programs?				
(RQ2) What services are provided to migratory students?
2.1. What strategies do programs implement to serve migratory students?				
2.2. What influences inform the strategies/services provided?				
2.3. What evidence do programs use to make implementation decisions?				
2.4. How do student outcome data influence program services?				
(RQ3) How do entities receiving MEP grant funds collaborate with other programs and agencies to address the needs of highly mobile students?
How do entities receiving grant funds work with staff from other programs to address the needs of migratory students?				
To what extent do MEP grantees facilitate partnerships/coordinate with nongovernmental organizations (NGOs), schools, and local operating agencies (LOAs) in serving migratory students?				
How are these collaborations documented, and how to they influence plans for other programs serving migratory students?				

Exhibit 1. Study questions and data sources, concluded

	Literature and extant data review	State survey	Local survey	Case study
(RQ4) How do migrant programs support students in earning high school diplomas or high school equivalency credentials and preparing them for postsecondary education and the workforce?
4.1. What instructional and support services are targeted to migratory students who are working toward high school completion?				
4.2. What instructional and support services are offered during the summer months to keep students on track to graduate?				
4.3. How do state and local policies and systems, including MSIX, facilitate record transfers and credit accrual across states and districts?				

To carry out the study, the research team from SRI International (SRI), Policy Studies Associates (PSA), and Arroyo Research Services (ARS) will first contact State Directors of Migrant Education to collect the names and contact information for their current subgrantees, including regions, local education agencies (LEAs), and local operating agencies (LOAs) that receive MEP funding. This second data collection will include surveys of State Directors of Migrant Education and local MEP coordinators, and case studies of 10 states, 20 subgrantee sites within those 10 states, and 40 schools or projects in the 20 subgrantee sites. The surveys and case study interview protocols can be found in Appendices B—G.

Authorizing Legislation

Authorizing legislation for this study is provided by the Elementary and Secondary Education Act (ESEA) as reauthorized by the Every Student Succeeds Act (ESSA), Public Law 114-95 (12/10/2015), section 8042.

2. Indicate how, by whom, and for what purpose the information is to be used

The study will result in a report and a short brief designed for federal, state, and local program coordinators and their partners who are implementing the Migrant Education Program. The U.S. Department of Education (the Department) plans to use the study results to increase its knowledge and understanding of program implementation. The Department will disseminate both study publications on the Web. The results of the study will inform MEP leaders about changes in program implementation following ESSA, and how these changes facilitate or hinder MEP-funded programs’ ability to meet migratory students’ educational and nonacademic support needs. The study will also facilitate sharing of promising practices among MEP-funded programs in support of continuous program improvement.

The study team will use a variety of technologies and methods to maximize data collection efficiency and minimize respondent burden. To collect information about MEP subgrantees, including each subgrantee organization’s name, address, primary point of contact, email address, and phone number, the study team will request that state MEP directors submit information via email in the format they have readily accessible (Excel is preferred when multiple formats are available).

Survey data collection will take place using Qualtrics, an online surveying software. The surveys will allow for multiple skip patterns that will save respondents time by presenting them only with the questions that are relevant to them. This approach also will minimize data entry error and the need to follow up with respondents.

As part of the case study methods, collection of key artifacts and documents will be limited to public information. Artifact and document collection will take place during site visits, although grantees and subgrantees may choose to send documents via email if it is more convenient.

During data collection periods, a toll-free phone number and an email address will be available to permit respondents to contact the survey administrator and case study site leads with questions or requests for assistance.

4. Describe efforts to identify duplication. Show specifically why any similar information already available cannot be used or modified for use for the purposes described in Item 2 above

The last study of the MEP was conducted in 1999. The Study of the Implementation of the ESEA Title I—Part C Migrant Education Program Serving Children of Agricultural Workers and Fishers, for which clearance is requested, is the only national study of the program currently funded. The study team completed an analysis of extant data and a review of academic literature, public reports, and available state program evaluations to identify existing information to answer the study questions. The study team identified some existing information that contributed to the design of the study instruments, to the case study sampling criteria, and to study findings, but information is not available to answer all the study questions. All information collected for this study aligns directly with the study questions, and all survey and interview protocols focus on critical information not available from other sources.

5. If the collection of information impacts small businesses or other small entities (Item 5 of OMB Form 83-I), describe any methods used to minimize burden

Known MEP grantees are state education agencies, which are not small businesses or small entities. No current information is available about the number of small businesses or entities that are subgrantees, however. A review of a sample of 2014–15 state evaluation reports suggests that the majority of subgrantees are either regional education service providers or local education agencies. The study team hypothesizes, therefore, that a very small proportion of subgrantees requested to complete a survey will be small entities (e.g., community-based organizations). To minimize burden on these entities, the surveys will be administered only once, are short (requiring only 15 to 20 minutes for program staff to complete), are free of jargon and inappropriate assumptions about respondents’ background knowledge and experience, and include primarily closed-response questions that do not present cognitive challenges to complete. The case study sample will not include a substantial number of small businesses or entities.

6. Describe the consequence to federal program or policy activities if the collection is not conducted or is conducted less frequently, as well as any technical or legal obstacles to reducing burden

Students and youth eligible for MEP-funded services are among the most at-risk populations in the United States, yet there has been no comprehensive, national study of the MEP since 1999. Failure to collect information on implementation of the MEP will prevent the Department of Education, Congress, and others from evaluating the factors that facilitate and hinder state-, regional-, local-, and school-level efforts to provide migratory students and out-of-school youth with effective, evidence-based academic and support services intended to help them achieve to the same standards as other children and prepare them for responsible citizenship, further learning, and productive employment. Without this information, the Department will lack key information needed to judge whether the program is being implemented as intended and how it could be improved, and to determine how to guide states and subgrantees as they implement new MEP requirements under ESSA.

7. Special circumstances

None of the special circumstances listed apply to this data collection.

8. Federal Register comments and persons consulted outside the agency

The 60-day Federal Register notice was published on May 9, 2017, Vol. 82, page 21532. It yielded no public comments. Additionally, a technical working group comprising four state and local program directors and a research methods expert reviewed and provided input on the study design and methods during the 60-day public comment period. The Department and the research team reviewed and considered all input from this group and revised the design and methods as appropriate.

To gather more detailed feedback on the survey and interview protocols, the research team conducted a pilot test of each data collection instrument. For the state MEP director and local MEP coordinator surveys, the study team piloted the instruments with two state MEP directors and two local MEP coordinators. This pilot test helped ensure that all questions are clear and are measuring the concepts the study intends and that each survey takes no more than 20 minutes to complete. Survey pilot participants completed the survey and responded briefly to several questions related to item clarity and structure. The survey tool tracked the time it took participants to complete the survey. The study team incorporated the revisions from the pilot phase into the draft of the survey and made substantial cuts to the length of the instrument.

For the case study interview protocols, the study team conducted pilot interviews with a sample of four volunteers representing state MEP directors, local MEP coordinators, and school administrators. Study team members conducted the pilot interviews by using a think-aloud technique through which researchers asked the interview questions and pilot respondents provided answers but also commented on any confusing, inappropriate, or leading questions. The study team gave special attention to the response options of the structured questions to help ensure that they were clearly understood and that they provide options that reflect pilot respondents’ experiences. Researchers took detailed notes throughout this process, consolidated the findings, and made corresponding changes to the protocols as appropriate. The researchers also made substantial cuts to the length of the instrument and identified high-priority questions to ensure the interviews can be completed within 60 minutes.

9. Payment or gift

The study team will provide no payment or gift to state, district, school, or project staff who participate in this study.

10. Assurances of confidentiality

Study participants will be provided with a study description before responding to the survey or case study interview. The study description will indicate that the researchers will make every effort to protect the confidentiality of the information provided, to the extent provided by law. It will also advise that respondents names and state as well as institution's or school's name will be disassociated from the data and pseudonyms will be used for each grantee. It will also inform respondents that responses will be used to summarize findings in an aggregate manner (across groups of grantees), or will be used to provide examples of program implementation in a manner that does not associate responses with a specific site or individual. The study description will also inform respondents that the study team may include direct quotes in the study report, but that no identifying information will accompany these quotes. The study team plans to report results in aggregate to describe MEP implementation practices at the state, district, and school or project levels. States, districts, schools and projects will be grouped by factors such as the size of the eligible migrant population served or approach to delivering services to minimize the risk of revealing the identity of a state, district, school/project, or individual through deductive reasoning. Before each individual interview, the site visitors will explain the purpose of the study, the topics to be covered in the interview, and the confidentiality assurances discussed above.

The study team has extensive experience in protecting the privacy and confidentiality of interview respondents. Safeguards to protect the privacy and confidentiality of all respondents—in addition to the ones discussed above—include the following:

All team members will participate in data collection training that includes a focus on methods to maintain participant confidentiality and data security.
The study team will provide secure environments for all data collected for the study.
The study team will immediately deidentify all data collected during the study that can potentially be linked to an individual and will delete temporary files that are stored on encrypted hard drives during on-site data collection activities.
Only authorized members of the study team will have direct access to deidentified study databases. Study team members will maintain a high level of focus on ensuring the confidentiality of both quantitative and qualitative data.
After completion of the study, the study team will destroy all data files.
The team will not share data obtained in this research with any entity or individual other than the Department and will not use the data for purposes other than this study.

11. Justification for questions of a sensitive nature

The survey and interview protocols will not include questions of a sensitive nature.

12. Estimates of the hour burden

The study team estimates that surveys will take no more than 20 minutes to complete. Surveys will be sent to one respondent in each of 47 states and approximately 813 subgrantee sites. Case study interviews will take no more than 60 minutes to complete and will take place in 10 states, with two subgrantees per state (20 total) and 40 schools or local projects per subgrantee site (two in each district).

Exhibit 2 provides estimates of the number of surveys and interviews and the amount of time required to conduct them. Document collection, which may be completed before, during, or after the site visit, may require case study sites up to 30 minutes and up to two individuals to complete in each site. The total number of responses needed to collect information for this study is estimated to be 1,350 responses and the total number of labor hours needed to collect information for the study is estimated to be 707 hours. There are no monetary costs to respondents for this activity. At an estimated 707 labor hours at an average of $45 per hour in labor, the overall cost burden for this information collection is $31,800.

Exhibit 2. Estimated number of respondents and labor hours for each information collection

Respondent category	Number of respondents per site	Number of sites	Total responses	Time to complete data collection	Total labor hours
Survey participants
State Directors of Migrant Education	1	47 (estimated based on 2015 grant awards)	47	20 minutes	16
Local subgrantee program coordinators	1	813 (estimated based on program office information)	813	20 minutes	271
Case study participants
State staff (e.g., State Director of Migrant Education, other MEP staff, other SEA staff) (10 state grantees)	Up to 4	10	40	1 hour	40
Directors of other state-level agencies and organizations collaborating with the state MEP (10 state grantees)	Up to 3	10	30	1 hour	30
Local MEP program staff (MEP coordinator, other local MEP staff) (20 subgrantees or 2 grantees per state)	Up to 3	20 subgrantees (2 subgrantees per state)	60	1 hour	60
Directors of other local or regional agencies and organizations collaborating with the local MEP (20 subgrantees or 2 grantees per state)	Up to 3	20 subgrantees (2 subgrantees per state)	60	1 hour	60
School or project staff (e.g., school principals, instructional personnel, counselors, and others who provide services to migratory children and youth) (40 schools/projects or 4 schools per subgrantee)	Up to 4	40 schools/projects or 4 schools per local subgrantee	160	1 hour	160
Key artifact and document collection	Up to 2	70, includes all sites visited	140	30 minutes	70
Total to complete the data collection			1350		707
Annualized (over three years)			450		236

13. Total annual cost burden for this activity

There is no capital or start-up cost component to these data collection activities, nor is there any operations, maintenance, or purchase cost associated with the study.

14. Annualized costs to the federal government

The estimated cost to the federal government is $1,099,668. Annualized over three years, this cost is $366,566 per year. This total includes costs already invoiced, plus budgeted future costs charged to the government for contractual services from SRI, PSA, and ARS to finalize the study design, site selection, data collection (including travel for site visits), data analysis, and reporting.

15. Program changes in burden/cost estimates

This request is for a new information collection, so no changes apply.

16. Plans for tabulation and publication

The study will result in a comprehensive report and a short brief designed to inform the Department staff and key MEP stakeholders, including State Directors of Migrant Education and local MEP program coordinators, about promising practices and potential challenges of MEP implementation. The report will use a combination of narrative text, tables, and charts to present findings. The report will describe how states, districts, and schools are identifying and serving the educational and other needs of highly mobile migrant students; how states address the needs of migrant students in designing their accountability systems; how migrant programs are identifying and serving hard-to-reach migratory populations, including out-of-school youth; how states and districts receiving MEP grant funds collaborate with other agencies and organizations to deliver services to migratory children and their families; and what instructional and support services MEP-funded programs deliver to help migrant students complete high school.

Data collection is scheduled to begin November 2017 and end in February 2018. The analysis will begin in February 2018, and the final report is scheduled for completion in September 2018.

17. Expiration date omission approval

Not applicable. All data collection instruments will include the OMB data control number and data collection expiration date.

18. Exceptions to the certification statement

Not applicable. There are no exceptions requested.

1 OIG has identified issues related to state Identification and Recruitment practices and whether MEP funds are adequately focused on students who are PFS (see OIG Management Information Report Control Number ED-OIG/X06-D0021).

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
Author	Deborah Jonas
File Modified	0000-00-00
File Created	2021-01-22