Att_OMB Part A REL Appalachia Task 2C.7-16-07

IPR 12367

Part A

Supporting Justification Request for

OMB Clearance of Information Collection Forms for:

An Impact Evaluation of Early Literacy Programs

August 29, 2007

Submitted to: Submitted by:

U.S. Department of Education REL Appalachia / CNAC

Institute of Education Sciences 4825 Mark Center Drive

555 New Jersey Ave., NW, Rm. 506 Alexandria, VA 22311

Washington, DC 20208 703-824-2828

202-219-1597

Project Officer: Principal Investigator:

Sandra Garcia Steven Ross

TABLE OF CONTENTS

A. JUSTIFICATION 5

1. Circumstances That Make Data Collection Necessary 5

2. How, by Whom, and for What Purpose the Information is to be Used 6

3. Use of Information Technology 10

4. Efforts to Identify and Avoid Duplication 10

5. Impacts on Small Businesses and Other Small Entities 10

6. Consequences to Federal Programs or Policies if Data Collection is Not Conducted 11

7. Special Circumstances 11

8. Solicitation of Public Comments and Consultation with People Outside the Agency 11

9. Respondents Payments 12

10. Confidentiality Assurances 12

11. Sensitive Questions 15

12. Total Annual Cost Burden to Respondents 15

13. Estimate of the Total Annual Cost Burden to Respondents or Record Keepers 16

14. Annualized Cost to the Federal Government, Including a Description of the Methods

Used to Estimate Costs 16

15. Program Changes or Adjustments 16

16. Tabulation, Analysis and Publication of Results 16

17. Approval Not to Display the Expiration Date for OMB Approval 18

18. Exception to the Certification Statement 18

References 19

Appendices 21

Appendix A 22

Education Sciences Reform Act of 2002 (web link provided) 22

Appendix B 23

Early Literacy Study Consent Form 24

Appendix C 25

Draft of 60-Day Federal Register Notice 26

Appendix D 27

Draft of 30-Day Federal Register Notice 28

Appendix E 29

IRB Approval Letter for Study 30

Attachments 31

Exhibit A: Memphis City Schools OWL Program Teacher Questionnaire 32

Exhibit B: Memphis City Schools Pre-K Program Teacher Questionnaire 33

Exhibit C: Memphis City Schools OWL Program Paraprofessional Questionnaire 34

Exhibit D: Memphis City Schools Pre-K Program Paraprofessional Questionnaire 35

List of Tables

Table 1. Research Questions and Data Collection 8

Table 2. Estimated Hours of Burden for Respondents 15

Table 3. Schedule of Study Activities 16

A. JUSTIFICATION

1. Circumstances that make the collection of information necessary

The Regional Educational Laboratory Program (REL) consists of a network of ten laboratories that serve the educational needs of a designated region by providing access to high quality, scientifically valid education research through applied research and development projects, studies, and other related technical assistance activities. The REL program is authorized under the Education Sciences Reform Act of 2002, Part D, Section 174, (20 U.S.C. 9564) and administered by the Institute of Education Sciences' National Center for Education Evaluation and Regional Assistance. The current priority for the 2006-2010 contract period is providing policymakers and practitioners with expert advice and training and technical assistance on how to interpret the latest findings from scientifically valid research that pertain to requirements that they must meet under the No Child Left Behind Act. And, in instances where scientific evidence is not available and schools need appraisal and analysis of alternative strategies to improve learning, the RELs fill the void with applied research and development projects that include rigorous studies. Rigorous studies are planned to address high priority issues that need testing to establish the effects of proposed policies, programs, or practices on academic achievement and other related high-priority needs of the each region. These studies are designed to provide causally valid answers and must meet IES standards for field tests based on experimental designs. The website where this legislation authorizing the present study may be found is provided in Appendix A.

The proposed collection by REL Appalachia seeks to extend research to determine the impact of an early literacy program on preschool children. The Appalachian Region was cited in 2003 for failing to provide a comprehensive preschool curriculum to children. The report cites all four of the states in the REL Appalachian region, Kentucky, Tennessee, Virginia, and West Virginia, for their failure to meet the Federal benchmark for a comprehensive preschool curriculum, and three of the four (KY, VA, WV) for their failure to meet the Federal benchmark for teacher education and training. Clearly, in the Appalachian Region, preschool teachers need training in: (a) the importance of early literacy; (b) the role childcare has in supporting children’s early literacy development; and (c) effective scientifically based practices that support early literacy development. This collection meets a significant need to examine the impact of a scientifically based program in preparing children to be ready for school. It specifically seeks to identify effects and whether those effects vary among different types of children.

A growing literature suggests that quality preschool programs significantly improve children’s school readiness, particularly in the areas of cognitive development and early literacy skills. One of the most comprehensive studies demonstrating the positive effects of quality child-care on later child development is the NICHD Study of Early Child Care and Youth Development (NICHD Early Child Care Research Network (ECCRN), 2001). Findings from this study consistently indicate that quality of child-care significantly predicts children’s later cognitive and linguistic development (Duncan, 2003; NICHD ECCRN, 2000; NICHD ECCRN, 2001; NICHD ECCRN, 2002). The effect sizes found in the NICHD SECCYD were generally small. In a commentary on the NICHD 2001 study findings, Todd (2001) noted that, compared to home-based caregivers, center-based caregivers typically provided more verbal stimulation, which, in turn promoted the acquisition of early literacy skills. Furthermore, the greater frequency of structured activities in center-based preschools also led to the children's performing at higher intellectual and linguistic levels. Similarly, Duncan (2003) found that participation in center-based preschool programs did predict later cognitive development, although the effect sizes were small. Positive effects have also been observed after controlling for maternal vocabulary score, family income, child gender, observed quality of the home environment, and observed maternal cognitive stimulation (NICHD Child Care Research Network, 2000). Thus, improving teacher education and training is likely to enhance overall program quality, which in turn, should enhance children’s school readiness. It is noteworthy that the findings from the NICHD SECCYD are based on correlational analyses; it therefore seems important to determine whether a more highly controlled, randomized study might yield larger effects.

Teaching reading and writing to young children in America has always been an area of controversy and debate (Teale & Yokota, 2000), and it remains so today. Studies (Lee & Burkam, 2002; NAEP, 2004) show that without powerful intervention, children from economically disadvantaged settings are likely to start school behind their middle-class peers and stay behind, with the gap becoming ever larger in each subsequent year. To prevent the gap, research has shown that these children need content-rich instruction that blends meaningful learning with foundational skills (Makin, 2003).

2. Explain how, by whom, how frequently, and for what purpose the information will be used. Except for a new collection, indicate the actual use the agency has made of the information received from the current collection

This study will assess the impact of an early literacy curriculum in Memphis City Schools (MCS). Caution should be used in extending results interpretation beyond the Memphis City Schools. Some bias will result due to significant variance in population characteristics among the four Appalachian states, ranging from primarily poverty level, white, rural children to primarily urban, poverty level, African American children. The sample is not a probability sample, and is therefore not intended to represent all elementary students in the region. Of the research questions shown below, Questions 1 and 5 address the random experimental design and utilize Hierarchical Linear Modeling, which is described in greater detail in Part B. Its purpose is to obtain correct standard errors of measurement (SEs) due to clustering. Questions 2, 3, and 4 involve descriptive and pre-test/posttest only designs, utilizing MANOVA, MANCOVA and qualitative analyses of the observational and survey data.

1. What are program effects on the literacy readiness and skills of MCS children, particularly those from low-income families, prior to entering kindergarten?

2. To what degree have teachers improved instruction and classroom environments by implementing scientific research-based practices in language, cognition, and early reading?

3. What is the impact of the Early Literacy Education program on the quality of instructional practice?

4. To what extent have all teachers been provided with the training and support to implement high-quality Early Literacy Education for children?

5. To what degree are program effects on student achievement associated with variations in school characteristics, student characteristics, and implementation fidelity?

Memphis City Schools (MCS) was selected for the study because of its large number of preschool classrooms (138) and district-wide investment in early literacy instruction. The low reading and language arts state achievement test scores in its Title I elementary schools also prompted a strong district focus on early literacy instruction. The large number of preschool classrooms available for the study (50) provided the necessary statistical power to implement a randomized study with control classrooms comprised of very similar children (i.e., ethnic, socio-economic and community variables). The extensive costs of conducting an intensive, multi-year evaluation dictated the size of the targeted population and restricted the study to a single, large school district.

A total of 21 schools, consisting of 26 preschool classrooms, were randomly assigned to the program treatment group, and 20 schools, consisting of 24 preschool classrooms, to the control group. The program treatment schools received Opening the World of Learning (OWL) training and materials in August 2006. To measure impact of this curriculum, within each classroom we will assess all student enrollees (having parent permission), for a total of approximately 1000 children. In Year 2, a second cohort will be added to the study. Because our primary focus is on the impact of the Early Literacy program intervention on MCS children’s school readiness, all analyses will be at the individual student level, with school variables controlled via the hierarchical linear model (HLM) analyses to be employed.

Tennessee’s public preschool programs are voluntary. Parents make application and provide written permission for their children’s participation. Memphis City Schools is already collecting information on its preschool classrooms to comply with requirements for classrooms receiving state funding. For example, the district administers the Early Language and Literacy Classroom Observation (ELLCO), which is a comprehensive set of observation tools for describing the extent to which classrooms provide children optimal support for their language and literacy development (Smith & Dickinson, 2002). In selecting assessment instruments for this study, every practical effort has been made to consolidate requirements on teachers, parents and students who are the respondents.

The study utilizes data collected by Memphis City Schools in the first year of implementation of the study (Cohort 1). The school district requested that REL Appalachia and its subcontractor, Education Innovations, LLC (EI), collect data for the second year of implementation (Cohort 2) and for the follow-up assessment of the children in Cohort 1 in their kindergarten and later grades. EI will collect data through direct assessments of individual children, teacher and paraprofessional surveys, and classroom observations. The district will continue to collect parent survey data, which it will provide to EI. In the second year of implementation (Cohort 2 and Year 3 of study), REL Appalachia / Education Innovations, LLC (EI) will collect preschool student achievement data twice a year, teacher and paraprofessional surveys in the spring, and classroom observations twice a year.

OMB approval is being sought only for the administration of teacher and paraprofessional surveys in Spring 2008. Memphis City Schools disseminated the consent form for teacher and paraprofessional participation. Therefore, the consent form is not part of the documents to be approved. All assessment instruments yielding data for the study and descriptions of data collection are presented in Table 1.

In summary, the instruments being used to collect data for the study are the Peabody Picture Vocabulary Test – Third Edition, Phonological Awareness Literacy Screening for Preschool, Dynamic Indicator of Basic Early Literacy Skills, Woodcock-Johnson Tests of Achievement – Third Edition, Early Language and Literacy Observation, Early Literacy Observation Tool, OWL Teacher Survey, Pre-K Teacher Survey, OWL Paraprofessional Survey, Pre-K Paraprofessional Survey, and Parent Survey. Of these, OMB approval is only required for the OWL Teacher Survey, Pre-K Teacher Survey, OWL Paraprofessional Survey, and Pre-K Paraprofessional Survey (see Exhibits A, B, C, and D).

Table 1. Research Questions and Data Collection

Data Source and Purpose	How and By Whom Data are Collected	Research Question
Peabody Picture Vocabulary Test- Third Edition (PPVT-III). The PPVT-III is an assessment of receptive vocabulary and listening comprehension of spoken words.	Assessments are conducted by EI researchers and the data are collected by EI staff from the PPVT-III test forms. The PPVT-III is individually administered to all participating students by trained EI site researchers in the fall and spring of the preschool year. All participating students who are present in their classroom are tested during the scheduled day of testing. Site researchers return to the classrooms to test students who were absent during the original testing date. EI staff provides training, supervision, and monitoring.	1, 4, 5
Phonological Awareness Literacy Screening for Preschool (PALS-Pre-K). The PALS-Pre-K is an assessment of phonological awareness and literacy basics (e.g., Name writing ability, Letter sound and Beginning sound production).	Trained EI site researchers conduct assessments, and the data are collected by EI staff from the PALS-Pre-K test forms. The PALS-Pre-K is individually administered to all participating students by site researchers in the fall and spring of the preschool year. All participating students who are present in their classroom are tested during the scheduled day of testing. Site researchers return to the classrooms to test students who were absent during the original testing date. EI staff provides training, supervision, and monitoring.	1, 4, 5
Dynamic Indicator of Basic Early Literacy Skills (DIBELS). The DIBELS assessment measures alphabetic principles, phonemic awareness, phonics, and oral reading fluency. The five subtests utilized for kindergarten through second grade students are: Initial Sound Fluency, Letter Naming Fluency, Phoneme Segmentation Fluency, Nonsense Word Fluency and Oral Reading Fluency	The DIBELS is individually administered to all participating students by trained EI site researchers in the fall and spring of the kindergarten and first grade years and the fall of the second grade year. All participating students who are present in their classroom will be tested during the scheduled day of testing. Site researchers return to the classrooms to test students who were absent during the original testing date. EI staff provides training, supervision, and monitoring.	1, 5
Woodcock-Johnson Tests of Achievement- Third Edition (WJ-III ACH). The WJ-III ACH is an individually administered norm-referenced assessment comprised of 22 tests measuring five curricular areas—reading, mathematics, written language, oral language, and academic knowledge. Only 4 tests related to reading and oral language will be administered: Spelling, Passage Comprehension, Oral Comprehension, and Reading Vocabulary.	The WJ-III ACH is individually administered to all participating students by trained EI site researchers in the fall and spring of the kindergarten and first grade years. All participating students who are present in their classroom will be tested during the scheduled day of testing. Site researchers return to the classrooms to test students who were absent during the original testing date. EI staff provides training, supervision, and monitoring.	1, 5
Early Language and Literacy Classroom Observation (ELLCO). The ELLCO is an assessment of classroom environment using observation to complete the following subscales: Literacy Environment, General Classroom Environment, Language, Literacy and Curriculum, and Literacy Activities Rating Scale.	Trained EI site researchers conduct classroom observations. Data are not collected from individuals and therefore do not require OMB clearance. The ELLCO will be conducted in the fall and spring for treatment schools and only in the spring for control schools. An EI site researcher spends one full academic day observing in the classroom to complete the rating scale.	2, 3, 4, 5
Early Literacy Observation Tool (E-LOT). The E-LOT is an evaluation of teacher training, using classroom observation (Literacy Observation Tool Notes and Data Summary).	The E-LOT can be conducted simultaneously with the ELLCO and provides additional teacher data. Data are not collected from individuals and therefore do not require OMB clearance. Trained EI site researchers complete an observational rating scale and checklist. EI will conduct the E-LOT in the fall and spring for treatment schools and only in the spring for control schools. A site researcher spends one full academic day observing in the classroom.	2, 3, 4, 5
Teacher Surveys. The Memphis City Schools OWL Program Teacher Questionnaire (MCSOPTQ) and the Memphis City Schools Pre-K Program Teacher Questionnaire (MCSPTQ) will be used to collect Treatment and Control teacher perceptions of professional development, resources, pedagogical change, outcomes, and support.	The Memphis City School district conducts the MCSOPTQ and the MCSPTQ for Cohort 1 with technical assistance and support provided by EI staff. Surveys are completed online during the end of the spring semester and results are provided to EI. OMB approval is requested to conduct teacher surveys for Cohort 2 (Year III of the study).	2, 3, 4, 5
Paraprofessional Surveys. The Memphis City Schools OWL Program Paraprofessional Questionnaire (MCSOPPQ) and the Memphis City Schools Pre-K Program Paraprofessional Questionnaire (MCSPPQ) will be used to collect Treatment and Control paraprofessional perceptions of professional development, resources, outcomes, and support.	The Memphis City School district conducts the MCSOPPQ and the MCSPPQ with technical assistance and support provided by EI staff. Surveys are completed online during the end of the spring semester and results are provided to EI. OMB approval is requested to conduct paraprofessional surveys for Cohort 2 (Year III of the study).	2, 3, 4, 5
Parent Survey. The Memphis City Schools Pre-K Program Parent Questionnaire (MCSPPPQ) will be used to collect parent perceptions of student progress, literacy and the preschool program.	The MCS district conducts annually a paper-based, state-mandated parent survey and the results are provided to EI.	3, 4

REL Appalachia will use this information to inform Memphis City Schools and the U. S. Department of Education regarding the impact of a scientifically based preschool curriculum on early literacy development and cognitive development among Appalachian Region children. Information will be shared continuously over the four-year period of the project with monthly progress reports provided.

3. Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological techniques or other forms of information technology

On-line surveys will be administered to teachers and paraprofessionals. Instructions are printed on the questionnaire form. Memphis City Schools Administrative staff distributes the consent form for participation in the study to each teacher and paraprofessional at the beginning of the school year. Copies are provided to Education Innovations, LLC. See Appendix B for a copy of the consent form.

4. Describe efforts to identify duplication. Show specifically why any similar information already available cannot be used or modified for use for the purposes described in item 2 above

No other information is available from teachers or paraprofessionals regarding perception of the early literacy program, its implementation and impact in the second cohort year. No state or district achievement tests are administered in kindergarten or first grade, so similar academic achievement information is not available. Duplicate questions are not present among the instruments of the proposed collection.

5. If the collection of information involves small businesses or other small entities, describe the methods used to minimize burden

Schools are small entities. Creating online instruments for the teachers and paraprofessionals has reduced the burden to schools. These questionnaires may be completed anytime and anywhere, given the accessibility of a computer. Ample computers with Internet capability are available at the schools, and many staff members have their own personal computers at home.

6. Describe the consequences to the Federal program or policy activities if the collection is not conducted or is conducted less frequently

This one-time collection by REL Appalachia via EI will be valuable for informing future decision-making regarding curriculum and classroom practices. The consequences of not conducting the collection are substantial. Large numbers of grants have been provided to school districts in Tennessee to implement preschool programs, including the funding of 50 classrooms in Memphis City Schools. Without the ability to conduct a randomized study, information would not be available as to the impact of an early literacy program in these classrooms, and, more generally, the impact of scientifically based curricula on Appalachian Region preschool children of varying backgrounds and experiences. 

7. Explain any special circumstances that require the collection to be conducted in a manner inconsistent with OMB guidelines

There are no special circumstances that require the collection to be conducted in a manner inconsistent with OMB guidelines.

8. Provide a copy of the PRA Federal Register notice that solicited public comments on the information collection prior to this submission. Summarize the public comments received in response to that notice and describe the actions taken by the agency in response to those comments. Describe the efforts to consult with persons outside the agency to obtain their views on the availability of data, frequency of collection, the clarity of instructions and recordkeeping, disclosure, or reporting format (if any), and on the data elements to be recorded, disclosed, or reported

We will publish 60-day and 30-day Federal Register Notices to allow public comment.

Consultation has been undertaken prior to formulating the proposed collection. Meetings have been held over a period of one year with Memphis City Schools’ administrative staff as well as preschool teachers. Dr. Linda Kennard, Director of Early Childhood and Elementary Literacy for MCS, and Dr. Brenda Taylor, Coordinator of the MCS Early Childhood Education Program, are experienced educators with doctoral degrees in Curriculum and Instruction with an early childhood and literacy focus. Expertise in early literacy has also been provided by the College of Education at The University of Memphis (Dr. Anna Grehan, Research Associate Professor). Topics of discussion included views on the need for data, availability of data, frequency of collection, clarity of instructions, recordkeeping, disclosure, reporting format, and on the data elements to be recorded, disclosed, or reported. Comments and suggestions from these discussions were incorporated as the present study design was developed.

9. Explain any decisions to provide payments or gifts to respondents, other than remuneration of contractors or grantees

EI will be providing professional development services worth up to $500 to Memphis City Schools for each control school’s participation in the study (20 schools). Additionally, 1 staff person per district is permitted to attend the professional development training. These services are provided for the control schools’ effort to provide data (assistance in scheduling and coordinating children’s assessment and classroom observations as well as the establishment of appropriate times for teachers and paraprofessionals to respond to an online survey). The control school preschool classrooms will not be receiving the intervention of the early literacy curriculum. Treatment schools are receiving the benefits of the district-provided early literacy curriculum and accompanying professional development in lieu of payment.

All participating children (treatment and control groups) will receive a small token of appreciation (special pencil, pad, etc.) for their participation in the testing activities, budgeted at approximately 50 cents per child.

10. Describe any assurance of confidentiality provided to respondents and the basis for assurance in statute, regulation, or agency policy

Assurance of confidentiality is provided in writing to all respondents. A statement to this effect is included on the school district administered teacher and paraprofessional participant consent form. (Please see Appendix B.)

The Privacy Act of 1974 (Public Law No. 93-579, 88 Stat. 1897) serves as the basis for assurance of confidentiality. The CNA Corporation, Education Innovations, LLC and all REL Appalachia partners follow the confidentiality and data protection requirements of IES (The Education Sciences Reform Act of 2002, Title I, Part E, Section 183). The CNA Corporation and REL Appalachia will protect the confidentiality of all information collected for the study and will use it for research purposes only. No information that identifies any study participant will be released. Information from participating institutions and respondents will be presented at aggregate levels in reports. Datasets with individual identifiers required for the study include student assessments and administrative records. All individual level identifiable information will be used only for purposes of linking records across datasets. Once the files are merged, they will be stripped of the original identifiers. Original datasets with identifiers will be kept in secured locations and destroyed as soon as they are no longer required.  All institution-level identifiable information will be kept in secured locations and identifiers will be destroyed as soon as they are no longer required.

All members of the study team having access to study data have been certified by The University of Memphis Internal Review Board (IRB) as having received training in the importance of confidentiality and data security. The following confidentiality language appears on all letters, brochures, consents, and other study materials:

Responses to this data collection will be used only for statistical purposes.  The reports prepared for this study will summarize findings across the sample and will not associate responses with a specific district or individual.  We will not provide information that identifies you or your district to anyone outside the study team, except as required by law.

Both the CNA Corporation (CNAC), the REL Appalachia contracted organization, and Education Innovations, their subcontractor conducting this research, execute and maintain a rigorous policy on Human Subjects Research, which will be the standard for this study. These policies result from CNAC and EI insistence upon full compliance with governing statues and regulations and from their commitment to safeguard the rights and welfare of human participants in all research with which they are associated. Both CNAC and EI have a designated Human Subjects Officer (HSO) to review research involving human subjects and determine whether that research needs to be assessed by a formal Institutional Review Board (IRB) before proceeding. CNAC and EI follow the instructions of the Institutional Review Board with regard to any additional disclosures or issues specifying the type of consent required from the research participant. Specifics regarding confidentiality of data for CNAC and EI are provided below.

The CNA Corporation: Confidentiality of Data

CNAC’s policy ensures strict confidentiality for data access and management. Our handling procedure for all projects that involve individually identifiable data is as follows:

• For projects that require no exchange of data with external personnel, all sensitive data will be stored and utilized on servers that are segregated from the corporate Windows 2000 domain and all other domains on the corporate network.

• Projects that require external exchange of sensitive data will use RADCON01 or other secure means of data exchange for this purpose. This server exists in a separate Windows 2000 domain (CNACCON) that was established to give non- CNAC employees the ability to exchange data securely with CNAC employees.

• Access to servers that have sensitive data is granted through access to the domain by a separate user account (separate from the corporate network account) on an as-needed basis. Access may be further restricted to a particular server if necessary.

• Servers that contain sensitive data must use warning banners to post security reminders/warnings.

• A separate user account (from overall corporate network account) and password are required to access all servers that contain sensitive data.

• Access control lists (ACLs) are used to restrict access to data. User access is restricted to the minimum necessary to perform the job. User rights are granted on an as-needed, need-to-know basis.

• Project personnel can move only sanitized data, which have been stripped of personal identifiers, to user folders.

• Any data that comes in on tape, floppy, CDRom, or zip disk will be returned to the client as soon as possible. Only necessary copies of these data will be made, and these sensitive data will not be copied from the server to any removable media.

• No sensitive data will be removed from the CNAC premises.

• Access to data from remote sites from an authorized computer is permitted as long as the data is not copied to any media.

• Typically, all data will be backed up using secure procedures. Individual projects can request that their data not be backed-up if it is deemed too sensitive in nature.

Any concerns regarding human subject research will be reported to the HSO, and he/she will determine the appropriate corrective actions. All complaints will be taken seriously and fully evaluated.

Education Innovations (EI): Confidentiality of Data

EI has strict confidentiality procedures for research data received in two formats: physical and electronic. Once collected research data is handled and stored according to the most appropriate of the two methods below:

Physical Data – Physical data such as questionnaires, observation materials, documents collected as part of a document review, interview tapes from focus groups and interviews or any other data collected for the purposes of completing the research in question is treated carefully and confidentially at all times. Upon receipt of the physical data, it is logged in and immediately stored in a file folder associated with the project in question located in a private office that is locked each evening and is only accessible to project staff during the day. Upon completion of the project all related files are sealed and moved to an archive office located on site and then to a secure offsite archive a year later where it is kept until the data is a total of seven years old before then being destroyed. The data is always in possession of organizational staff and is never at any time shared with anyone, including the project sponsor, in its original form; it’s presented as a summary where all names and other identifying information has been removed.

Electronic Data – Electronic data exists as a result of collecting data directly from research subjects through online questionnaires or student achievement databases, or it exists as a summary of physical data collected during the course of research. In either case it is treated carefully and confidentially at all times. Upon receipt of the data it is immediately stored on a central file server accessible only to project staff and located behind a secure firewall that has all unessential ports blocked. Accessing the data requires domain name authentication of both the user and the computer and can only occur from within the offices located behind the firewall except for the rare occasion that project staff may be given access through the use of a secure Virtual Private Network connection also requiring user authentication. Computer use and access to the data is limited to regular office hours and all computers are located in offices that are secured each evening.

11. Provide additional justification for any questions of a sensitive nature, such as sexual behavior and attitudes, religious beliefs, and other matters that are commonly considered private

No private or sensitive information such as sexual behavior and attitudes or religious beliefs is requested in this collection. In order to assess the impact of an early literacy program on different children, however, the collection does ask identifying information which includes gender, race/ethnic background, identified disabilities and socio-economic background (eligibility for free/reduced lunch). This information request is standard for enrollment in public school in Tennessee. All information is kept confidential, as described in Item #10. IRB approval for the study was obtained from The University of Memphis (see Appendix E).

12. Provide an estimate in hours of the burden of the collection of information

The estimated burden includes the reading of the instructions and the completion of the survey items. See Table 2 below.

Table 2. Estimated Respondent Burden

Data Collection Activity	Number of Respondents per Data Collection	Number of Data Collections	Total Number of Responses	Time per Response (in minutes)	Total Hour Burden	Hourly Rate	Total Monetary Burden
MCS OWL Program Teacher Questionnaire	26	1	26	12	5.2	$31.17	$162.08
MCS Pre-K Teacher Questionnaire	24	1	24	12	4.8	$31.17	$149.62
MCS OWL Program Paraprofessional Questionnaire	26	1	26	10	4.3	$31.17	$134.97
MCS Pre-K Paraprofessional Questionnaire	24	1	24	10	4.0	$31.17	$124.68
Total	100	4	100		18.3		$571.35

13. Provide an estimate of the total annual cost burden to the respondents or record-keepers resulting from the collection (excluding the value of the burden hours in #12 above)

No costs are imposed on respondents or record keepers. Data collectors are employed to obtain all necessary data.

14. Provide estimates of annualized cost to the Federal government

OMB Approved Data Collection Burden - $32,425.40

Total Project Cost - $2,575,874.00

15. Explain the reasons for any program changes or adjustments reported in Items 13 or 14 of the OMB 83-I

Not applicable since this is a new collection

16. For collections whose results will be published, outline the plans for tabulation and publication

Please see Table 3 for a schedule of the study activities.

Table 3. Schedule of Study Activities

Activity	Schedule
Memphis City Schools (MCS) created Pool for Site Selection/Recruitment	Spring 2006
MCS conducted School Recruitment	Spring- Summer 2006
Random Assignment	Summer 2006
Study IRB approval	Fall 2006
MCS (District) Informed Consent forms to parents and staff to participate in study	Fall 2006
MCS Start Intervention	Fall 2006
MCS start collection of administrative data, ELLCO, E-LOT and PPVT-III & PALS pre-test (Cohort 1). Technical assistance and support by EI as requested	Fall 2006
MCS conduct ELLCO, E-LOT and PPVT-III & PALS Post-test, survey administration (Cohort 1) Technical assistance and support by EI as requested	Spring 2007
Receipt by EI of district data for Cohort 1 (children, parents, teacher and paraprofessional)	Spring 2007
EI conduct DIBELS pre-test for Kindergarten students (Cohort 1).	Fall 2007
EI start collection of administrative data, ELLCO, E-LOT and PPVT-III & PALS pre-test (Cohort 2)	Fall 2007
EI consent forms to teachers and paraprofessionals	Spring 2008
EI conduct DIBELS and WJACH-III post-tests for Kindergarten students (Cohort 1)	Spring 2008
EI conduct ELLCO, E-LOT and administer PPVT-III & PALS Post-test (Cohort 2)	Spring 2008
EI conduct pre-K Teacher and Paraprofessional surveys	Spring 2008
Receipt by EI of district data (parent surveys for Cohort 2)	Spring 2008
EI conduct DIBELS pre-test for 1st grade students (Cohort 1)	Fall 2008
EI conduct DIBELS pre-test for Kindergarten students (Cohort 2)	Fall 2008
EI conduct DIBELS and WJACH-III post-tests for 1st grade students (Cohort 1)	Spring 2009
EI conduct DIBELS and WJACH-III post-tests for Kindergarten students (Cohort 2)	Spring 2009
EI conduct DIBELS pre-test for 2nd grade students (Cohort 1)	Fall 2009
EI conduct DIBELS pre-test for 1st grade students (Cohort 2)	Fall 2009
Receipt by EI of state assessment data post-test for 2nd grade students (Cohort 1)	Spring 2010
EI Final Report of Findings	Fall 2010

To test the stated hypotheses, the data will be analyzed using multi-level analysis (Hierarchical Linear Modeling). The results will be put into a final report and submitted for internal review within REL Appalachia and then sent to IES, where an external peer review process will be completed to obtain approval. Once approved, the report and study findings will be published to the IES website and disseminated.

Results of the study will be disseminated in strict accord with the policies and permissions of the Appalachian Regional Educational Laboratory and the Institute of Educational Sciences. Identifying the audiences to be targeted, determining the messages to be delivered, and researching prospective venues for presentations, workshops and other forms of distribution will develop the dissemination plans.

Targeted audiences will be categorized by local, state, regional and national level. Most appropriately, the plan will be first to disseminate the results to the Appalachian Regional Educational Laboratory and the local school district, whose children, staff and parents participated in the study. The Institute of Education Sciences will be provided the results in its prescribed format. These presentations will be scheduled soon after completion of the final research report. The nature of the study results will dictate other audiences and the extent to which additional presentations are made. Regional audiences to be considered for dissemination activities in the Appalachian region include state departments of education, state boards of education, state associations of superintendents, Appalachia ListServ, and early childhood education organizations (i.e. The Urban Child Institute of Memphis and Head Start agencies). At the national level, venues may include conferences of the American Education Research Association (AERA), the International Reading Association (IRA), web site links, and various journal publications.

The nature of the results will be a significant factor in the determination of the message(s) to be delivered. For example, if results indicate that the implementation of an early literacy curriculum is associated with significantly improved readiness for kindergarten, the message will be informational and will likely encourage support for consideration of early literacy instruction as a best practice in early childhood education. Similarly, strategies for professional development and program monitoring may be considered for wide dissemination if associated with effective curriculum implementation. The limited scope of this study does not suggest an actionable message other than perhaps to the local school district.

Dissemination activities will be the outcome indicators of the plan. Short-term indicators (within 12 months) will be reports, articles, and other written material. Indicators will be number of requests for REL Appalachia research findings and number of copies distributed in hard and soft copy. Presentations, briefings and conferences will be reported in terms of number of conferences attended, number of presentations made, number of attendees and type of audience. Internet materials (REL web site) will be reported by number of downloads, number of email distributions/recipients, number of subscription services, subscribers and notifications. Mid-term indicators (within two years) will be the number of citations appearing in publications of studies and papers.

17. If seeking approval to not display the expiration date for OMB approval of the information collection, explain the reasons why display would be inappropriate

Not applicable

18. Explain each exception to the certification statement identified in Item 19 of the

OMB 83-I

There are no certification exceptions identified with this information collection.

References

Dunn, L. M., & Dunn, L. M. (1997). Peabody Picture Vocabulary Test Third Edition. Circle Pines, MN: American Guidance Services, Inc.

Education Sciences Reform Act (2002). P. L. No. 107-279.

Good, R. H. & Kaminski, R. A. (2003). DIBELS Indicator of Basic Early Literacy Skills. Longmont: CO. Sopris West Educational Services.

Grehan, A. W. & Smith, L., J. & Ross, S. M. (2004). The Early Reading First Literacy Observation Tool. Memphis, TN: The University of Memphis, Center for Research in Educational Policy.

Lee, V., & D. Burkam. (2002). Inequality at the starting gate: Social background differences in achievement as children begin school. Washington, DC: Economic Policy Institute.

Makin, L. (2003). Creating positive literacy learning environments in early childhood. In Hall, N, Larson, J. and Marsh, J (Eds.) Handbook of Early Childhood Literacy. London: Sage

NAEP (National Assessment of Educational Progress). (2004). Percentage of students, by reading achievement level, grade 4: 1992-2003. Online: http://nces.ed.gov/nationsreportcard/reading/results2003/natachieve-g4.asp.

National Institute of Child Health and Human Development, NIH, DHHS. (2006). The NICHD Study of Early Child Care and Youth Development (SECCYD): Findings for Children up to Age 4 1/2 Years (05-4318). Washington, DC: U.S. Government Printing Office.

NICHD Early Child Care Research Network. (2002) Early Child Care and Children's Development Prior to School Entry: Results from the NICHD Study of Early Child Care. American Educational Research Journal, 39 (1). 133-164.

NICHD Early Child Care Research Network. (2001). A new guide for evaluating child care quality. Bulletin of Zero to Three: National Center for Infants, Toddlers, and Families, 21(5), 40-47.

NICHD Early Child Care Research Network. (2000). The relation of child care to cognitive and language development. Child Development, 71(4), 960-980.

NICHD Early Child Care Research Network, Duncan, GJ (2003). Modeling the impacts of child care quality on children’s preschool cognitive development. Child Development, 74, 1454-75.

No Child Left Behind Act 2001. Pub. L. No. 107-110, 107^th Congress

PALS-Pre K. Retrieved June 1, 2007 from http://pals.virginia.edu/PALS-Instruments/PALS-PreK.asp

Teale, W., & Yokota, J. (2000). Beginning reading and writing: Perspectives on instruction. In D. S. Strickland & L. M. Morrow (Eds.), Beginning reading and writing. Language and literacy series (pp. 3-21). Newark, DE: International Reading Association.

Todd, C. M. (2001). The NICHD Child Care Study Results: What do they mean for parents, child-care professionals, employers and decision makers? Washington, DC: USDA/CSREES: Extension CARES for America's Children and Youth Initiative.

Woodcock, R. W., & Johnson, M. B. Woodcock- Johnson Psycho-Educational Battery. Hingham, MA: Teaching Resources, 1978.

Appendices

Appendix A

Education Sciences Reform Act of 2002

For a copy of this legislation authorizing the study, please visit:

http://www.ed.gov/policy/rschstat/leg/PL107-279.pdf.

Appendix B

Early Literacy Study Consent Form Administered by Memphis City Schools

(Not administered by EI)

Early Literacy Study

Consent Form for Teachers and Paraprofessionals

As part of a preschool program research study conducted in Memphis City Schools by Education Innovations, LLC, you are being asked to participate in an evaluation of the program at your school this year. The focus of the study will be to determine the effectiveness of preschool programs in raising student achievement. We expect to have 50 classrooms and approximately 1,000 children participating in the study. Approximately half of the classrooms will use the Opening the World of Learning curriculum as part of their instructional program as a way of measuring the effectiveness of that curriculum.

If you agree to participate in the study, you will be involved in the following ways:

1. We will ask you to complete a brief survey about your perception of the professional development, resources, pedagogical change, outcomes, and support of the preschool program at your school.

2. We will send trained staff to individually assess children in your classroom on language skills to determine progress made during the year.

3. We will also send site researchers to observe your classroom twice during the year.

All survey responses and observation data will be considered confidential, within the limits allowed by law. When findings are described or quoted in technical reports or journal articles related to this study, neither the identity of your school nor the identity of any individual will be revealed. There are no perceived risks or costs to you associated with participating in the study. The benefits are the additional information available to you regarding each student’s current skill levels and progress made by the end of the year.

If you have any questions regarding this study or the use of data collected, you may contact Dr. Steven Ross, at (901) 678-2310 or Dr. Anna Grehan, at (901) 678-4222. Questions regarding the rights of research subjects may be directed to the Chair of the University of Memphis Institutional Review Board (901) 678-2533. (Note: The University of Memphis does not have any funds budgeted for compensation for injury, damages, or other related expenses.)

I have read the information in the consent form. Any questions that I may have had have been answered. I may withdraw my participation at any time by notifying Dr. Ross or Dr. Grehan. Refusal to participate will involve no penalty or loss of benefits to which I am otherwise entitled. Similarly, the researcher may choose to terminate my participation if the study requirements recommend participation changes. I have not waived any rights by agreeing to participate.

Signature		Date

Responses to this data collection will be used only for statistical purposes. The reports prepared for this study will summarize findings across the sample and will not associate responses with a specific district or individual. We will not provide information that identifies you or your district to anyone outside the study team, except as required by law.

2597 Avery Avenue Memphis, Tennessee 38112 (901) 416-5455

Appendix C

Draft of 60-Day Federal Register Notice

60-Day Register Notice -- Draft

DEPARTMENT OF EDUCATION

Notice of Proposed Information

Collection Requests

AGENCY: Department of Education.

SUMMARY: The Leader, Information

Management Case Services Team,

Regulatory Information Management

Services, Office of the Chief Information

Officer, invites comments on the

proposed information collection

requests as required by the Paperwork

Reduction Act of 1995.

DATES: Interested persons are invited to

submit comments on or before DATE

SUPPLEMENTARY INFORMATION: Section

3506 of the Paperwork Reduction Act of

1995 (44 U.S.C. Chapter 35) requires that the

Office of Management and Budget (OMB) provide interested Federal agencies and the public an early

opportunity to comment on information

collection requests. OMB may amend or

waive the requirement for public

consultation to the extent that public

participation in the approval process

would defeat the purpose of the

information collection, violate State or

Federal law, or substantially interfere

with any agency’s ability to perform its

statutory obligations. The Leader,

Information Management Case Services

Team, Regulatory Information

Management Services, Office of the

Chief Information Officer, publishes that

notice containing proposed information

collection requests prior to submission

of these requests to OMB. Each

proposed information collection,

grouped by office, contains the

following: (1) Type of review requested,

e.g. new, revision, extension, existing or

reinstatement; (2) Title; (3) Summary of

the collection; (4) Description of the

need for, and proposed use of, the

information; (5) Respondents and

frequency of collection; and (6)

Reporting and/or Recordkeeping

burden. OMB invites public comment.

The Department of Education is

especially interested in public comment

addressing the following issues: (1) Is

this collection necessary to the proper

functions of the Department; (2) will

this information be processed and used

in a timely manner; (3) is the estimate

of burden accurate; (4) how might the

Department enhance the quality, utility,

and clarity of the information to be

collected; and (5) how might the

Department minimize the burden of this

collection on the respondents, including

through the use of information technology.

Dated: XX/XX/XXXX

Angela C. Arrington,

Leader, Information Management Case

Services Team, Regulatory Information

Management Services, Office of the Chief

Information Officer.

Institute of Education Sciences

Type of Review: New.

Title: An Impact Evaluation of Early Literacy Programs

Frequency: One-time collection

Affected Public: Memphis City Schools; Preschool teachers and paraprofessionals.

Reporting and Recordkeeping Hour

Burden:

Responses: 100.

Burden Minutes: 1,100.

Abstract: Preparing children for success in kindergarten is a priority for preschool programs. This proposed randomized-control longitudinal study seeks to determine whether Early Literacy preschool programs have an impact on participating children and, if so, whether such effects vary among different types of children, families, schools, and configurations of children’s preschool and program experiences. The study will address the following questions. How does the Early Literacy preschool program affect the school readiness of participating children? Does the Early Literacy preschool program improve children’s cognitive development, and early literacy skills? Under what conditions does the Early Literacy program work best, and for which children?

Requests for copies of the proposed information collection request may be

accessed from http://edicsweb.ed.gov,

by selecting the ‘‘Browse Pending

Collections’’ link and by clicking on

link number XXXX. When you access the information collection, click on ‘‘Download Attachments’’ to view. Written requests for information should be addressed to U.S. Department of Education, 400 Maryland Avenue, SW., Potomac Center, 9th Floor, Washington, DC 20202–4700. Requests may also be

electronically mailed to the Internet address OCIO[email protected] or faxed to 202–245–6621. Please specify the complete title of the information

collection when making your request.

Comments regarding burden and/or

the collection activity requirements

should be directed to Kathy Axt at her

e-mail address [email protected].

Individuals who use a telecommunications

device for the deaf (TDD) may call the Federal Information Relay Service (FIRS) at 1–800–877–

8339.

[FR Doc.# Filed DATE; TIME]BILLING CODE

Appendix D

Draft of 30-Day Federal Register Notice

30-Day Register Notice -- Draft

DEPARTMENT OF EDUCATION

Submission for OMB Review;

Comment Request

AGENCY: Department of Education.

SUMMARY: The IC Clearance Official,

Regulatory Information Management

Services, Office of Management invites

comments on the submission for OMB

review as required by the Paperwork

Reduction Act of 1995.

DATES: Interested persons are invited to

submit comments on or before MONTH DAY, YEAR.

ADDRESSES: Written comments should

be addressed to the Office of

Information and Regulatory Affairs,

Attention: Rachel Potter, Desk Officer,

Department of Education, Office of

Management and Budget, 725 17th

Street, NW., Room 10222, New

Executive Office Building, Washington,

DC 20503 or faxed to (202) 395–6974.

SUPPLEMENTARY INFORMATION: Section

3506 of the Paperwork Reduction Act of

1995 (44 U.S.C. Chapter 35) requires

that the Office of Management and

Budget (OMB) provide interested

Federal agencies and the public an early

opportunity to comment on information

collection requests. OMB may amend or

waive the requirement for public

consultation to the extent that public

participation in the approval process

would defeat the purpose of the

information collection, violate State or

Federal law, or substantially interfere

with any agency’s ability to perform its

statutory obligations. The IC Clearance

Official, Regulatory Information

Management Services, Office of

Management, publishes that notice

containing proposed information

collection requests prior to submission

of these requests to OMB. Each

proposed information collection,

grouped by office, contains the

following: (1) Type of review requested,

e.g. new, revision, extension, existing or

reinstatement; (2) Title; (3) Summary of

the collection; (4) Description of the

need for, and proposed use of, the

information; (5) Respondents and

frequency of collection; and (6)

Reporting and/or Recordkeeping

burden. OMB invites public comment.

Dated: MONTH DAY, YEAR.

Angela C. Arrington,

IC Clearance Official, Regulatory Information

Management Services, Office of Management.

Institute of Education Sciences

Type of Review: Revision.

Title: An Impact Evaluation of Early Literacy Programs

Frequency: One-time collection.

Affected Public: Memphis City Schools; Preschool teachers and paraprofessionals.

Reporting and Recordkeeping Hour

Burden:

Responses: 100.

Burden Minutes: 1,100.

Abstract: Preparing children for success in kindergarten is a priority for preschool programs. This proposed randomized-control longitudinal study seeks to determine whether Early Literacy preschool programs have an impact on participating children and, if so, whether such effects vary among different types of children, families, schools, and configurations of children’s preschool and program experiences. The study will address the following questions. How does the Early Literacy preschool program affect the school readiness of participating children? Does the Early Literacy preschool program improve children’s cognitive development, and early literacy skills? Under what conditions does the Early Literacy program work best, and for which children?

Requests for copies of the information

collection submission for OMB review

may be accessed from http://

edicsweb.ed.gov, by selecting the

‘‘Browse Pending Collections’’ link and

by clicking on link number XXXX. When

you access the information collection,

click on ‘‘Download Attachments’’ to

view. Written requests for information

should be addressed to U.S. Department

of Education, 400 Maryland Avenue,

SW., Potomac Center, 9th Floor,

Washington, DC 20202–4700. Requests

may also be electronically mailed to

[email protected] or faxed to 202–

245–6623. Please specify the complete

title of the information collection when

making your request.

Comments regarding burden and/or

the collection activity requirements

should be electronically mailed to

[email protected]. Individuals who

use a telecommunications device for the

deaf (TDD) may call the Federal

Information Relay Service (FIRS) at 1–

800–877–8339.

[FR Doc. xx–XXXX Filed DATE; TIME]

BILLING CODE xxxxxxx

Appendix E

IRB Approval Letter for Study

Attachments

Exhibit A

Memphis City Schools OWL Program Teacher Questionnaire

Please see separate file for a copy of this measure.

Exhibit B

Memphis City Schools Pre-K Program Teacher Questionnaire

Please see separate file for a copy of this measure

Exhibit C

Memphis City Schools OWL Program Paraprofessional Questionnaire

Please see separate file for a copy of this measure.

Exhibit D

Memphis City Schools Pre-K Program Paraprofessional Questionnaire

Please see separate file for a copy of this measure.

21