Att_1875-NEW 4763 omb_a_2.8.12

Contract # ED-CFO-10-A-0110-0001

21^st Century Community Learning Centers: Lessons Learned Guides

Deliverable 3.5 (1 of 2): Final OMB Package

Supporting Statement for Paperwork Reduction Act “A”

January 16, 2011

Revised January 31, 2012

Revised February 6, 2012

Prepared for

Prepared by

Supporting Statement for Paperwork Reduction Act

A. JUSTIFICATION

1. Circumstances making collection of information necessary

The U.S. Department of Education’s 21^st Century Community Learning Centers (21^st CCLC) program is authorized under Title IV, Part B, of the Elementary and Secondary Education Act (ESEA), as amended by the No Child Left Behind Act of 2001. The program’s purposes are to create community learning centers that provide students with academic enrichment opportunities as well as additional activities designed to complement their regular academic program. The 21^st CCLC program is designed to provide activities to address the educational, health, social services, cultural, and recreational needs of students after regular school hours, on weekends, and during the summer at school sites. By providing more time for enrichment and academics, the 21^st CCLC program plays an important role in ensuring that all students are college- and career-ready.

Building on its investment in these centers nationwide, the Department has continuously sought to identify and promote those practices that best support the program’s goals of complementing participants’ regular academic program and helping them become college- and career-ready. The Department commissioned this study to learn from the field about programmatic successes and challenges for four topic areas of afterschool programming: (1) Science, Technology, Engineering, and Math (STEM), (2) English Learners (ELs), (3) Career and Technical Education (CTE), and (4) structures to increase learning time (ILT). Knowledge of the curricula, enrichment activities, and strategies related to these topics that are currently used within 21^st CCLC sites remains limited, and it is important to determine how these topics are being implemented by individual subgrantees. (Please note that 21^st CCLC subgrantees may operate a single program site or may implement a program across multiple sites. This study is designed to concentrate its research and observations at the site, rather than the program, level.) It should be noted that subgrantees have an obligation to respond to research requests such as this study in order to retain a benefit (ESEA, Sec. 9306(a)(4)).

Specifically, the Department is interested in learning about particular practices related to each of the topic areas as they are currently being implemented by 21^st CCLC sites. A practice is defined here as any instructional approach, programmatic structure, or other activity or strategy enacted by a program as part of its organization or operation. The topic-area specific practices in which the Department is most interested are as follows:

• STEM. The Department is looking for practices to give students the knowledge, skills, and experiences to be prepared to go into a STEM field or know whether they are interested in a STEM field. This may involve practices including inquiry-based learning, partnerships (e.g., with community-based organizations or businesses), and professional development for staff to understand inquiry-based learning and know where to find information on STEM topics. It may also involve practices aimed at engaging students in scientific discourse and inquiry, designing engineering projects, using technology, or practicing math concepts in creative ways.

• EL. The Department is looking for practices (either in programs with dedicated EL activities or programs that have a substantial population of EL participants) that help make ELs proficient in English and equip them with the skills needed to participate fully in society, both academically and socially. This may include oral English practice, division of students by fluency level, use of English in meaningful and creative ways, or activities that address the socioeconomic needs of EL students. It will also include strategies to engage and support EL families. The Department aims to examine practices aimed at Spanish-language students as well as a diversity of other language communities.

• CTE. The Department is looking for practices that give students the knowledge, skills, and experiences to prepare them for careers. This may include practices around partnerships, internships (particularly internships that provide a learning experience), instruction in job-related and general employability skills, and practice in higher-order reasoning and problem-solving skills.

• ILT. The Department is looking for practices that support a programmatic focus on aligning with the school day. This may include practices such as shared professional development, shared curriculum, or shared planning time between regular school day and afterschool staff. It may also include reinforcement of concepts learned during the school day and explicit alignment of activities with school day learning standards.

2. Purposes and uses of the data

The field data collection portion of the study will involve case studies conducted in 60 21^st CCLC programs nationwide (15 programs per topic area) to document practices related to the topic areas. Through site visits and subsequent case study analysis, four “Lessons Learned Guides” will be produced. The guides will provide specific information on programmatic and instructional practices related to each of the topic areas. For STEM, this will include information on practices aimed at engaging students in scientific discourse and inquiry, designing engineering projects, using technology, or practicing math concepts in creative ways. It will also encompass inquiry-based learning practices, identification and utilization of partnerships, and the use of professional development activities to aid staff in learning STEM instructional techniques and provide access to background information on STEM topics. For EL, this will include information on practices aimed at making EL students across a diversity of language communities more proficient in English and better able to function academically and socially; it will also include strategies around communicating with and supporting EL families. For CTE, this will include information on effective partnerships, appropriate use of internships, and instructional practices around instilling and reinforcing practical job skills, general employability skills, and higher-order reasoning and problem-solving skills. For ILT, this will include programmatic and instructional strategies for aligning to the school day through techniques such as shared professional development and resources, reinforcement of school day concepts, and alignment to school day learning standards. In addition to information on the specific topic area practices described above, the study is designed to identify new or emergent practices within the sites. To this end, aspects of the interview protocols and the observation protocols are designed to remain open-ended to allow site visitors to identify new examples of programmatic and instructional practices in each topic area rather than to evaluate the program’s adherence to a preexistent list of practices. Ultimately, the guides will be designed to include information that 21^st CCLC subgrantees can use to improve their program. The Department’s staff will also use the guide to provide technical assistance to grantees. The guides will be posted on the U.S. Department of Education website.

3. Use of technology to reduce burden

Both the pre-selection screening tool and the information survey will be available online. The use of multiple skip patterns, integrated into the online survey, will greatly reduce burden on respondents by presenting them only with relevant questions. In order to further reduce burden, the information survey will be pre-populated wherever possible with information from the screening tool. All respondents will have the option to request the survey in hard copy format. The web-based survey will be password protected.

4. Efforts to identify duplication

This study of lessons learned from 21^st CCLC activities in the four topic areas represents the Department’s first and only study aimed at compiling data in these areas for the 21^st CCLC program. The Department has not conducted any other research studies on these topics since the start of the program.

5. Methods to minimize burden on small entities

Some of the 21^st CCLC subgrantees selected may be community-based organizations, and some of these may be small entities. These programs will experience the same level of burden as all other programs and will be able to use technology to reduce burden. We are not able to take further steps to minimize the burden on these programs as we need to ensure consistency in the data collected across all subgrantees.

6. Consequences of not collecting the data

The 21^st CCLC program is a government-funded program and one of the only federal funding sources for out-of-school time education, including afterschool and summer programs. To date, the Department has not captured or documented the range of curricula, enrichment activities, and strategies related to the topic areas that are currently used within 21^st CCLC sites. If this data is not collected, the Department will have an incomplete understanding of current practices in these topic areas and will be limited in its capacity to provide useful technical assistance to grantees in areas that are projected to grow in upcoming years.

7. Special circumstances

No special circumstances apply to this study.

8. Adherence to 5CFR 1320.8 guidelines and consultation outside the agency

A 60-day notice about this study was published in the Federal Register Notice Vol. 76, page 76393 on 12/7/11. A 30-day notice about this study will be also published in the Federal Register to provide the opportunity for public comment. No public comments have been received to date.

The contractor has convened a 12-member Technical Work Group (TWG) of individuals with expertise on 21^st CCLC programs, including researchers, practitioners, and administrators. The purposes of such consultations are to ensure the technical soundness and user relevance of study findings; to verify the importance, relevance, and accessibility of the information sought through the field data collection; to assess the clarity of interview items; and to minimize respondent burden.

Experts will serve either as part of a core TWG or as part of a topic-area specific TWG. Members of the core TWG will provide comments on site visit protocols and selection criteria, suggest possible 21^st CCLC programs for site visits, and provide feedback on the Lessons Learned Guides. Members of the topic-area specific TWGs will participate only in reviewing materials for their content areas. The panel members of both TWGs, with their roles indicated in parentheses, are as follows:

• Fred Doolittle, Vice President and Director, K-12 Education Policy Area, MDRC (Core)

• Lorraine Thoreson, Michigan Department of Education (Core)

• Megan Beckett, Behavioral/Social Scientist, RAND Corporation (Core)

• Nancy Deutsch, Asst. Professor of Education, University of Virginia (Core)

• Carol Tang, Director of the Coalition for Science After School (STEM)

• Gil Noam, Founder and Director of the Program in Education, Afterschool, & Resiliency (STEM)

• Laurie Olsen, Director of the Sobrato Early Academic Literacy Initiative (EL)

• Julie Maxwell-Jolly, Managing Director of the Center for Applied Policy in Education, UC Davis School of Education (EL)

• Richard Tagle, CEO, Higher Achievement (CTE)

• Kim Green, National Association of State Directors for Career and Technical Education (CTE)

• Ayeola Fortune, former Director, Extended Learning Opportunities and Development Project, Council of Chief State School Officers (ILT)

• Sarah M. Deschenes, Independent Consultant, formerly Senior Researcher, Harvard Family Research Project (ILT)

Manhattan Strategy Group (MSG) is organizing one in-person meeting and four phone calls to consult TWG members to ensure an appropriate study design and provide input on the data collection instruments and Lessons Learned Guides. In addition, project staff will also use outside experts for consultation on an as-needed basis throughout the study.

9. Payment or gifts

No payments or gifts will be used over the course of this study.

10. Assurances of confidentiality

MSG is concerned with maintaining the confidentiality and security of its records. The Lessons Learned Guides will name each selected site and provide program profile summaries of each. Program profiles will discuss positive practices observed in each site; weaknesses and shortcomings will be discussed in generalized sections that do not name individual sites. Individuals associated with the sites will not be identified in the reports at any point. It would not be difficult for individuals reading the report to research project directors or other individuals associated with each site and thus determine the identity of our respondents. We will endeavor to protect the privacy of our interviewees, and we will avoid using their names in the guides and attributing any quotes to specific individuals. The team will further ensure the confidentiality of the data to the extent possible through a variety of measures. The following specific confidentiality and data protection procedures will be in place:

• All interviewees in the study will be informed that they will not be mentioned by name in the study and that they can refuse to answer any question that they feel uncomfortable answering. Permission to audio record the interview will be verbally requested, and participants are free to decline being recorded.

• Project team members will be educated about the confidentiality assurances given to respondents. Each person assigned to the study will be cautioned not to discuss project data outside of official research team meetings.

• Respondents’ or interviewees’ names will be disassociated from the data as they are entered into the database. Data will only be coded according to the position/title of each respondent or interviewee during the data analysis process.

• We will shred all interview protocols, forms, and other hardcopy documents containing identifiable data as soon as the need for this hard copy no longer exists. We will also destroy any media containing interview responses.

• We will not provide information that associates responses or findings with the names of individual respondents or interviewees to anyone outside of the study team except if required by law.

• We will store all electronic recordings of interviews, interview notes, and other project-related documents in secure areas that are accessible only to authorized staff members. We will duplicate all basic computer files on computer-based backup servers to allow for file restoration in the event of unrecoverable loss of the original data. We will store these backup files under secure conditions in an area separate from the location of the original data.

11. Justification of sensitive questions

No questions of a sensitive nature will be included in this study. Respondents are reporting on program-level activities only.

12. Estimates of hour burden

The online screening tool will be completed by 21^st CCLC project directors as part of the final round of site selection. Total burden associated with participation in the online screening tool is estimated at 60 minutes per respondent, or 120 total hours. Interviews with state coordinators will be completed by telephone prior to the scheduled date of each site visit. Total burden associated with participation in telephone interviews for state coordinators is estimated at 30 minutes per respondent, or 30 total hours. (In cases where multiple sites are selected within a single state, the state coordinator will participate only in a single phone interview covering all of the sites in his or her state, which will further reduce the hour burden. However, the total amount of reduction cannot be determined until the completion of site selection.) The program information survey will be completed online or in hard copy by the 21^st CCLC project director prior to the site visit. Total burden associated with completion of the information survey is estimated at 30 minutes per respondent, or 30 total hours.

Total burden for site personnel includes estimates of time spent by administrative support personnel coordinating site visit schedules and time spent by personnel participating in in-person interviews. Total burden for administrative support staff is estimated at one hour per site, or 60 total hours. Project director burden associated with interview participation is estimated at 60 minutes per director, for a total of 60 hours across the 60 study sites. The burden for 21^st CCLC site coordinators associated with this activity is estimated at a maximum of 90 minutes per coordinator, for a total of 90 hours. The burden for 21^st CCLC staff is estimated for interviews at a maximum of 60 minutes per staff member for up to four staff members, resulting in a total burden of 240 hours. The burden for non-21^st CCLC personnel, including principals, classroom teachers, and partners, is estimated at a maximum of 30 minutes per individual, for a total burden of 150 hours. Total burden per site is estimated at 13 hours. (Please note that there is no burden associated with the site visit observations. Burden is already included for a staff member to set up and coordinate the site visit, and there is no additional staff burden related to the observations.)

Table 1 shows the total burden calculation for information collected by category of study participant. The total in hours for all participants and all portions of the study is 780 total hours (or 46,800 minutes). The hourly burden breakdown is as follows:

Table 1 Total Estimated Burden in Hours
	Total per Site (A)	Total Respondents (B)	Time in Minutes (C)	Total Time in Minutes (B x C)	Total Time in Hours
Project Director (Online Screening Tool)	1	120	60	7200	120
State Education Agency Representative	1	60	30	1800	30
Project Director (Information Survey)	1	60	30	1800	30
21st CCLC Administrative Assistant	1	60	60	3600	60
Project Director	1	60	60	3600	60
Site Coordinator	1	60	90	5400	90
21st CCLC Staff	4	240	60	14400	240
Principal	1	60	30	1800	30
Classroom Teacher	2	120	30	3600	60
Partner	2	120	30	3600	60
Total Burden	--	960	--	46,800	780

13. Estimate of cost burden to respondents

There are no respondent costs associated with this data collection other than the hour burden, accounted for in Item 12.

14. Estimate of annual cost to the federal government

The estimated cost for the 110-week study, including development of data collection instruments, justification package, data collection, data analysis, and preparation of Lessons Learned Guides, is $2,121,657.80. This averages to an annual cost to the government of $1,002,965.51.

15. Program changes or adjustments

The request is for a new data collection. This is a program change.

16. Plans for tabulation and publication of results

Table 2 presents the time schedule for conducting site visits to 21^st CCLC sites and for the publication of study results. The proposed timeline assumes OMB clearance in April 2012.

Table 2

Data Collection and Reporting Tasks and Timelines

Tasks	Timeline
Contact short-listed project directors for online screening tool	April 2012
Recruit school districts and school sites and schedule site visits	May 2012
Conduct field visits to 60 sites (including state interviews and information surveys)	May–December 2012
Analyze site visits	December 2012–January 2013
First Draft of Lessons Learned Guides	January 2013
Final Draft of Lessons Learned Guides	July 2013

In drafting the Lessons Learned Guides, MSG will prepare an outline of report contents. The contents will include, but are not limited to, the following:

1. Executive summary

2. Study purpose

3. Research methodology

   1. Interview development

   2. Data collection procedures

   3. Analysis methods

4. Cross-site analysis across topic areas

5. Cross-site analysis within each topic area

   1. Lessons learned on programmatic practices to support topic area

   2. Associated implementation challenges and strategies to overcome them

   3. Lessons learned on instructional practices to support topic area

   4. Associated implementation challenges and strategies to overcome them

6. Program profile summaries within each topic area

7. Implications of study findings and future directions

The Lessons Learned Guides will include results for the entire project based on the original study plan and any subsequent modifications to the plan agreed upon by the Department. Each guide will include a description of the methodology employed, findings, and implications for the study. Each will also include a nontechnical executive summary, a cross-site analysis that cuts across all four topic areas, cross-site analyses within each topic area, and individual site summaries within each topic area. Most centrally, they will offer detailed descriptions of the practices identified as being critical to successful implementation of each topic area. Included in these descriptions will be a discussion of the associated implementation challenges and the strategies that programs have used to overcome them. The guides will also provide in-depth descriptions of the practices in place at the study sites so that the reader can see what a practice “looks like in action.” We will write the guides and the executive summaries in a manner suitable for distribution to a broad audience. The Lessons Learned Guides will be provided in a 508-compliant PDF file for the PPSS website.

17. Approval to not display OMB expiration date

All data collection instruments will include the OMB expiration date.

18. Explanation of exceptions

No exceptions are requested.