1875-NEW MEP evaluation SS OMB Submission 1-Part A

The Policy and Program Studies Service (PPSS), of the U.S. Department of Education’s Office of Planning, Evaluation and Policy Development, requests Office of Management and Budget (OMB) clearance for the design of the Evaluation of the ESEA Title I—Part C Migrant Education Program (MEP). The study is being carried out under the authority of the Elementary and Secondary Education Act (ESEA) as reauthorized by the Every Student Succeeds Act (ESSA), Public Law 114-95 (12/10/2015), section 8042.

PPSS is requesting clearance to (a) collect data about subgrantees that receive MEP funds from state education agencies; (b) collect data from state MEP directors and local MEP coordinators about program implementation using an electronic, online survey; and (c) collect data via interviews from MEP leaders from 10 states, 20 districts within those 10 states, and 40 schools or projects in the 20 districts.

This request is for the study design and sampling strategy and a request to collect information about local grantees receiving MEP funding. This collection request is for recruitment activities associated with the evaluation. A future request will request clearance for the data collection instruments for the MEP evaluation.

This package contains two major sections:

Supporting Statement for Paperwork Reduction Act Submission

Justification (Part A)
Description of Statistical Methods (Part B)

Appendix

Appendix A—Notification letters

A. Justification

1. Circumstances that make the collection of information necessary

Migratory students represent one of the most disadvantaged student groups in the United States. Migratory students are more likely to live in poverty, experience disconnected educational experiences, and have health problems often associated with poverty that can have a negative impact on students’ educational progress and success. Congress established the Migrant Education Program (MEP) in 1966 under an amendment to the 1965 Elementary and Secondary Education Act (ESEA). Since that time, the program has evolved. For example, program changes now require states to conduct and use comprehensive needs assessments and statewide evaluations to inform their service delivery plans, and the program replaced the Migrant Student Record Transfer System with the Migrant Student Information Exchange (MSIX) to facilitate the seamless transfer of critical student background and performance information across district and state lines. The 2015 reauthorization of the ESEA under the Every Student Succeeds Act (ESSA) brought still more changes to program requirements. For example, new language on Priority for Services (PFS) is likely to affect how many and which migratory students are identified and served under PFS criteria. Earlier ESEA language on PFS referred to children whose education was interrupted during the school year; new ESSA language refers to those who have moved within the previous one-year period or have dropped out of school.

As the program changed over time, the population served also changed. Between 2001–2002 and
2013–2014, the number of eligible migrant students dropped 69 percent from 872,732 to 347,634;
out-of-school youth (OSY) receiving MEP services fell from nine percent of the eligible migrant student population to six percent; and students identified as PFS fell from 42 percent of migrant students served to 22 percent. At the same time, the percentage of all migrant students identified in the three states with the largest migratory populations remained constant at 53 percent. Enrollment declines reflect changes in migratory patterns and in agricultural and labor practices, including increased use of H2-A visas for agricultural work and tighter identification and recruitment practices in response to both Office of Inspector General (OIG) findings^¹ and Office of Migrant Education (OME) guidance. The declining percentage of migrant students identified as PFS reflects increased accountability for prioritizing services to students identified as PFS.

State and local services and supports for migratory children and youth have reflected the changes in MEP program requirements and the migrant student population. An internal review of MEP evaluation reports from a variety of states (e.g., Florida, Kentucky, New York, Maine, and Mississippi) identified numerous changes in states’ approaches to MEP. New York, for example, reconfigured its regional projects to focus on specific strategies for meeting its Measurable Program Outcomes. Kentucky reconfigured its regional and statewide summer programs to focus on the statewide academic needs identified through its Comprehensive Needs Assessment. Iowa modified its credit accrual and interstate communication practices to address increased return migration from Texas. For systematic analysis of such responses, it is necessary to go beyond current federal data collections (e.g., Comprehensive State Performance Reports, MSIX). New requirements also deserve analysis, notably the ESSA requirement that state and subgrantees align MEP strategies with evidence-based practices. This data collection request supports a national evaluation of MEP that will describe how state and local MEP-funded programs have evolved in response to changing requirements, and the extent to which and how they are serving the needs of a changing migratory student population.

Purpose of the Study

The Migrant Education Program provides funding to assist migratory children in meeting the same standards as other children and help them graduate from high school or a high school equivalency program with an education that prepares them for responsible citizenship, further learning, and productive employment. This evaluation is designed to examine and describe how states, districts, and schools implement programs for migratory children and youth, including coordination of services to address the unique needs of migratory students.

Exhibit 1 shows the evaluation questions and the data sources planned to answer each question. The study is using existing data and information when available. The study team has completed a review and analysis of the literature and of extant data to support the study and found that available data and literature do not provide complete answers to any of the evaluation questions and provide no information about others. The study will address these gaps in the research through surveys of State Directors of Migrant Education and local MEP coordinators; case study interviews with program coordinators and staff in MEP-funded states, local grantee sites, and schools or local MEP-funded projects; and a review of key artifacts and documents collected during site visits. The original data collection activities are designed to be complementary. The surveys will provide information on the full population of state and local MEP grantees while the case studies (including interviews and artifact and document collection) will provide detailed and nuanced data from a small subset of states, local sites, and schools or projects.

Exhibit 1. Evaluation questions and data sources

	Literature and extant data review	State survey	Local survey	Case study
(RQ1) How do state and local grantees respond to federal requirements for serving migratory children?
How does ESSA’s expanded definition of migratory students influence state and local practices to identify eligible migratory students?				
How are grantees prioritizing services to migratory students who moved within the year, are failing, or dropped out of school?				
How are states addressing the unique needs of migratory students in designing their accountability systems, both for MEP and for ESSA overall?				
How do migrant programs identify and serve eligible out-of-school youth?				
How do state and local MEP staff coordinate with other OME programs?				
(RQ2) What services are provided to migratory students?
2.1. What strategies do programs implement to serve migratory students?				
2.2. What influences inform the strategies/services provided?				
2.3. What evidence do programs use to make implementation decisions?				
2.4. How do student outcome data influence program services?				
(RQ3) How do entities receiving MEP grant funds collaborate with other programs and agencies to address the needs of highly mobile students?
How do entities receiving grant funds work with staff from other programs to address the needs of migratory students?				
To what extent do MEP grantees facilitate partnerships/coordinate with nongovernmental organizations (NGOs), schools, and local operating agencies (LOAs) in serving migratory students?				
How are these collaborations documented, and how to they influence plans for other programs serving migratory students?				

Exhibit 1. Evaluation questions and data sources, concluded

	Literature and extant data review	State survey	Local survey	Case study
(RQ4) How do migrant programs support students in earning high school diplomas or high school equivalency credentials and preparing them for postsecondary education and the workforce?
4.1. What instructional and support services are targeted to migratory students who are working toward high school completion?				
4.2. What instructional and support services are offered during the summer months to keep students on track to graduate?				
4.3. How do state and local policies and systems, including MSIX, facilitate record transfers and credit accrual across states and districts?				

To carry out the study, the research team from SRI International (SRI), Policy Studies Associates (PSA), and Arroyo Research Services (ARS) will first contact State Directors of Migrant Education to collect the names and contact information for their current subgrantees, including regions, local education agencies (LEAs), and local operating agencies (LOAs) that receive MEP funding. This first data collection will inform the second data collection, which will include surveys of State Directors of Migrant Education and local MEP coordinators, and case studies of 10 states, 20 subgrantee sites within those 10 states, and 40 schools or projects in the 20 subgrantee sites. The surveys and case study interview protocols can be found in Appendices C—F.

Authorizing Legislation

Authorizing legislation for this study is provided by the Elementary and Secondary Education Act (ESEA) as reauthorized by the Every Student Succeeds Act (ESSA), Public Law 114-95 (12/10/2015), section 8042.

2. Indicate how, by whom, and for what purpose the information is to be used

The study will result in a report and a short brief designed for federal, state, and local program coordinators and their partners who are implementing the Migrant Education Program. The U.S. Department of Education (the Department) plans to use the study results to increase its knowledge and understanding of program implementation. The Department will disseminate both study publications on the Web. The results of the study will inform MEP leaders about changes in program implementation following ESSA, and how these changes facilitate or hinder MEP-funded programs’ ability to meet migratory students’ educational and nonacademic support needs. The study will also facilitate sharing of promising practices among MEP-funded programs in support of continuous program improvement.

The study team will use a variety of technologies and methods to maximize data collection efficiency and minimize respondent burden. To collect information about MEP subgrantees, including each subgrantee organization’s name, address, primary point of contact, email address, and phone number, the study team will request that state MEP directors submit information via email in the format they have readily accessible (Excel is preferred when multiple formats are available).

Survey data collection will take place using Qualtrics, an online surveying software. The surveys will allow for multiple skip patterns that will save respondents time by presenting them only with the questions that are relevant to them. This approach also will minimize data entry error and the need to follow up with respondents.

As part of the case study methods, collection of key artifacts and documents will be limited to public information. Artifact and document collection will take place during site visits, although grantees and subgrantees may choose to send documents via email if it is more convenient.

During data collection periods, a toll-free phone number and an email address will be available to permit respondents to contact the survey administrator and case study site leads with questions or requests for assistance.

4. Describe efforts to identify duplication. Show specifically why any similar information already available cannot be used or modified for use for the purposes described in Item 2 above

The last evaluation of the MEP was conducted in 1999. The Evaluation of the ESEA Title I—Part C Migrant Education Program, for which clearance is requested, is the only national study of the program currently funded. The study team completed an analysis of extant data and a review of academic literature, public reports, and available state program evaluations to identify existing information to answer the evaluation questions. The study team identified some existing information that contributed to the design of the study instruments, to the case study sampling criteria, and to study findings, but information is not available to answer all the evaluation questions. All information collected for this study aligns directly with the evaluation questions, and all survey and interview protocols focus on critical information not available from other sources.

5. If the collection of information impacts small businesses or other small entities (Item 5 of OMB Form 83-I), describe any methods used to minimize burden

Known MEP grantees are state education agencies, which are not small businesses or small entities. No current information is available about the number of small businesses or entities that are subgrantees, however. A review of a sample of 2014–15 state evaluation reports suggests that the majority of subgrantees are either regional education service providers or local education agencies. The study team hypothesizes, therefore, that a very small proportion of subgrantees requested to complete a survey will be small entities (e.g., community-based organizations). To minimize burden on these entities, the surveys will be administered only once, are short (requiring only 15 to 20 minutes for program staff to complete), are free of jargon and inappropriate assumptions about respondents’ background knowledge and experience, and include primarily closed-response questions that do not present cognitive challenges to complete. The case study sample will not include a substantial number of small businesses or entities.

6. Describe the consequence to federal program or policy activities if the collection is not conducted or is conducted less frequently, as well as any technical or legal obstacles to reducing burden

Students and youth eligible for MEP-funded services are among the most at-risk populations in the United States, yet there has been no comprehensive, national evaluation of the MEP since 1999. Failure to collect information on implementation of the MEP will prevent the Department of Education, Congress, and others from evaluating the factors that facilitate and hinder state-, regional-, local-, and school-level efforts to provide migratory students and out-of-school youth with effective, evidence-based academic and support services intended to help them achieve to the same standards as other children and prepare them for responsible citizenship, further learning, and productive employment. Without this information, the Department will lack key information needed to judge whether the program is being implemented as intended and how it could be improved, and to determine how to guide states and subgrantees as they implement new MEP requirements under ESSA.

7. Special circumstances

None of the special circumstances listed apply to this data collection.

8. Federal Register comments and persons consulted outside the agency

The 60-day Federal Register Notice was published on March 6, 2017, Vol. 82, page 12576. A technical working group comprising of four state and local program directors and a research methods reviewed and provided input on the study design and methods during the 60-day public comment period. The Department and the research team reviewed and consider all input from this group and revised the design and methods as appropriate.

To gather more detailed feedback on the survey and interview protocols, the research team will conduct a pilot test of each data collection instrument. For the state MEP director and local MEP coordinator surveys, the study team will seek out four state MEP directors and four local MEP coordinators to volunteer to participate in the pilot test. This pilot test will help ensure that all questions are clear and are measuring the concepts the study intends and that each survey takes no more than 20 minutes to complete. Survey pilot participants will complete the survey and respond briefly to several questions related to item clarity and structure. The survey tool will track the time it takes participants to complete the survey. Follow-up phone calls with pilot participants will help the team learn more about the respondents’ understanding of the text of each item, definitions of key terms, precision and completeness of the response options, and whether important questions are missing. The study team will incorporate the revisions into the final draft of the survey.

For the case study interview protocols, the study team will conduct pilot interviews with a sample of nine volunteers representing state MEP directors, local MEP coordinators, and school administrators. Study team members will conduct the pilot interviews by using a think-aloud technique through which researchers will ask the interview questions and pilot respondents will provide answers but also comment on any confusing, inappropriate, or leading questions. The study team will give special attention to the response options of the structured questions to help ensure they are clearly understood and provide options that reflect pilot respondents’ experiences. Researchers will take detailed notes throughout this process, consolidate the findings, and make corresponding changes to the protocols as appropriate. The pilot will also assess the time it takes to complete the interview to verify that the information can be collected in less than 60 minutes.

9. Payment or gift

The study team will provide no payment or gift to state, district, school, or project staff who participate in this study.

10. Assurances of confidentiality

Study participants will provide informed consent before responding to the survey or case study interview. The consent form will indicate that the study team will keep all responses confidential and aggregate data in all reports to minimize the risk that any individual may be identified through a deductive reasoning process. The consent form will also inform respondents that the study team may include direct quotes in the study report, but that no identifying information will accompany these quotes. The study team plans to report results in aggregate to describe MEP implementation practices at the state, district, and school or project levels. States, districts, schools and projects will be grouped by factors such as the size of the eligible migrant population served or approach to delivering services to minimize the risk of revealing the identity of a state, district, school/project, or individual through deductive reasoning. Before each individual interview, the site visitors will explain the purpose of the study, the topics to be covered in the interview, and the confidentiality assurances discussed above.

The study team has extensive experience in protecting the privacy and confidentiality of interview respondents. Safeguards to protect the privacy and confidentiality of all respondents—in addition to the ones discussed above—include the following:

All team members will participate in data collection training that includes a focus on methods to maintain participant confidentiality and data security.
The study team will provide secure environments for all data collected for the study.
The study team will immediately deidentify all data collected during the study that can potentially be linked to an individual and will delete temporary files that are stored on encrypted hard drives during on-site data collection activities.
Only authorized members of the study team will have direct access to deidentified evaluation databases. Study team members will maintain a high level of focus on ensuring the confidentiality of both quantitative and qualitative data.
After completion of the study, the study team will destroy all data files.
The team will not share data obtained in this research with any entity or individual other than the Department and will not use the data for purposes other than this evaluation.

11. Justification for questions of a sensitive nature

The survey and interview protocols will not include questions of a sensitive nature.

12. Estimates of the hour burden

The study team estimates that surveys will take no more than 20 minutes to complete. Surveys will be sent to one respondent in each of 47 states and approximately 813 subgrantee sites. Case study interviews will take no more than 60 minutes to complete and will take place in 10 states, with two subgrantees per state (20 total) and 40 schools or local projects per subgrantee site (two in each district).

Exhibit 2 provides estimates of the number of surveys and interviews and the amount of time required to conduct them. Document collection, which may be completed before, during, or after the site visit, may require case study sites up to 30 minutes and up to two individuals to complete in each site. The total number of estimated labor hours needed to collect information for this study is 707 hours.

There are no monetary costs to respondents for this activity. At an estimated 707 labor hours at an average of $45 per hour in labor, the overall cost burden for this information collection is $31,800.

Exhibit 2. Estimated number of respondents and labor hours for each information collection

Respondent category	Number of respondents per site	Number of sites	Number of respondents	Time to complete data collection	Total labor hours
Survey participants
State Directors of Migrant Education	1	47 (estimated based on 2015 grant awards)	47	20 minutes	16
Subgrantee program coordinators	1	813 (estimated based on program office information)	813	20 minutes	271
Case study participants
State staff (e.g., State Director of Migrant Education, other MEP staff, other SEA staff) (10 state grantees)	Up to 4	10	40	1 hour	40
Directors of other state-level agencies and organizations collaborating with the state MEP (10 state grantees)	Up to 3	10	30	1 hour	30
Local MEP program staff (MEP coordinator, other local MEP staff) (20 subgrantees or 2 grantees per state)	Up to 3	20 subgrantees (2 subgrantees per state)	60	1 hour	60
Directors of other local or regional agencies and organizations collaborating with the local MEP (20 subgrantees or 2 grantees per state)	Up to 3	20 subgrantees (2 subgrantees per state)	60	1 hour	60
School or project staff (e.g., school principals, instructional personnel, counselors, and others who provide services to migratory children and youth) (40 schools/projects or 4 schools per subgrantee)	Up to 4	40 schools/projects or 4 schools per local subgrantee	160	1 hour	160
Key artifact and document collection	Up to 2	70, includes all sites visited	140	30 minutes	70
	Total Number of respondents		1,350
	Annual Number of Respondents		450
			Total labor hours to complete the data collection		707
			Annual labor hours to complete the data collection		236

13. Total annual cost burden for this activity

There is no capital or start-up cost component to these data collection activities, nor is there any operations, maintenance, or purchase cost associated with the evaluation.

14. Annualized costs to the federal government

The estimated cost to the federal government is $366,556. This total includes costs already invoiced, plus budgeted future costs charged to the government for contractual services from SRI, PSA, and ARS to finalize the study design, site selection, data collection (including travel for site visits), data analysis, and reporting.

15. Program changes in burden/cost estimates

This request is for a new information collection, so no changes apply.

16. Plans for tabulation and publication

The study will result in a comprehensive report and a short brief designed to inform the Department staff and key MEP stakeholders, including State Directors of Migrant Education and local MEP program coordinators, about promising practices and potential challenges of MEP implementation. The report will use a combination of narrative text, tables, and charts to present findings. The report will describe how states, districts, and schools are identifying and serving the educational and other needs of highly mobile migrant students; how states address the needs of migrant students in designing their accountability systems; how migrant programs are identifying and serving hard-to-reach migratory populations, including out-of-school youth; how states and districts receiving MEP grant funds collaborate with other agencies and organizations to deliver services to migratory children and their families; and what instructional and support services MEP-funded programs deliver to help migrant students complete high school.

Data collection is scheduled to begin in September 2017 and end in December 2017. The analysis will begin in January 2018, and the final report is scheduled for completion in September 2018.

17. Expiration date omission approval

Not applicable. All data collection instruments will include the OMB data control number and data collection expiration date.

18. Exceptions to the certification statement

Not applicable. There are no exceptions requested.

1 OIG has identified issues related to state Identification and Recruitment practices and whether MEP funds are adequately focused on students who are PFS (see OIG Management Information Report Control Number ED-OIG/X06-D0021).

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
Author	Deborah Jonas
File Modified	0000-00-00
File Created	2021-01-22