Reading 1st 1810-New Supporting Statement

Supporting Statement for Paperwork Reduction Act Submissions

Overview

This document has been prepared to support clearance of procedures and data collection instruments to be used in the U.S. Department of Education’s study to obtain data on the usage of Reading First grant funds. This Study of Reading First Expenditures falls under the Reading First Best Practices contract # ED-04-CO-0041-0004. This overview briefly describes the purpose and methodology of this study.

Part A “Justification” and Part B “Collections of Information Employing Statistical Methods” respond to specific instructions in SF93. Appendices contain copies of the instruments and procedure descriptions for which clearance is sought.

Purpose

The U.S. Department of Education (ED or the Department) Reading First program awards grants to state agencies to implement comprehensive reading instruction programs based on scientifically based reading research to children in kindergarten through third grade. Initial program grants were awarded to State educational agencies (SEAs) starting in July, 2002. SEAs were expected to award at least 80% of their grant funds to eligible local educational agencies (LEAs). LEAs have used these subgrants to fund Reading First programs.

The U.S. Department of Education Reading First program requires from state grantees applications that describe the planned expenditure of Reading First funds, and annual performance reports that describe their program implementation and outcomes. However, there is no formal or systematic mechanism for grantees to report on specific uses of Reading First grant funds. Some local evaluations may be collecting such data but these data collections are not systematic or consistent across the entire Reading First program. The Reading First Implementation study collected some funds-use data, as well, but only from a sample of LEA subgrantees. No SEA grantees were included, and only minimal information about grant money was collected. No information was collected about allocations.

To remedy the lack of information, ED requires a study to obtain data on the use of Reading First grant funds. The proposed surveys will systematically collect data on the use and allocation of Reading First grants from current SEA grantees and their LEA subgrantees. ED has contracted with RMC Research, Inc. (RMC) to assist in the implementation of this study.

Methodology

RMC has designed two surveys to collect data on the use of Reading First funds, one for SEA grantees and one for LEA subgrantees. RMC expects Reading First state coordinators and district Reading First coordinators will respond to the surveys, although it is likely they will require assistance from others more familiar with their budgets, and pilot testing has confirmed this. RMC will suggest the type of information needed for contacting and administering the surveys, and the methods required for participants to complete and submit their responses.

Given the easy access to contact information about grantees at both the SEA and LEA levels, and the likelihood of a high level of cooperation by grantees in a survey conducted by their funder, RMC proposes to survey the universe of SEA grantees and LEA subgrantees rather than survey representative samples of each. The reason for the broad approach is threefold. First, given the high level of interest in the use of Reading First, ED wishes to account for all uses of those funds. Second, states varied in how prescriptive they were in their allocation to LEAs and oversight of funds. Only by surveying all recipients can we capture the full variance in the way that funds were spent. Third, in surveying the universe of LEA and SEA Reading First recipients, it will be possible to disaggregate data at a finer level than would be possible using a sample. Although a universe survey may differ from methods generally used for these types of surveys, ED wishes to understand the variance in use of funds by type of locality, grant size, demographic makeup (i.e., student subgroups), and funding year, among other sub-variables.

Data Collection

With the approval of the surveys by ED and OMB, and construction of the online forms and database, RMC will begin implementation of the two surveys. Respondents will complete the surveys online using a code and password. This process will include the following steps:

1. Announcements will be sent to SEAs via emails to Chief State School Officers (CSSO) and copied to state Reading First coordinators. The early announcement of the survey will give respondents time to begin pulling together the information they will need to complete the questionnaire.

2. SEA respondents will be provided with the website address, code, and password required for completing the surveys using e-mail.

3. RMC will obtain a list of LEA subgrantees that received Reading First funding at any time between July 1, 2004 and June 30, 2008.

4. While SEAs are completing the survey, they will also be asked to forward an email letter to the LEAs in their state, announcing the survey.

5. LEA respondents will be provided with the website address, code, and password required for completing the surveys using e-mail.

6. Electronic and telephone call reminders will be given to all grantees as required to ensure participation.

7. To minimize data entry errors, RMC will build verification checks into the programming of the online survey. For example, out-of-range values will be flagged and respondents prompted to verify the value. Missing data will also be treated similarly; respondents will be unable to submit a survey until data for key categories are provided. Data checks will also be built for percentage estimates and inconsistent values between categories.

8. Completed surveys will automatically be saved and logged into RMC’s secure database.

9. Any outlying or questionable survey data will be verified via email or phone to ensure data integrity. For example, most states provided explicit guidelines to LEAs regarding use of their funds. An LEA with a response pattern very different from other LEAs in their state will be asked to verify their data.

Additionally, RMC will collect information on the grantees that will be used during the data analysis task to supplement the data collected by the survey. Staff will search documents, such as those available through the National Center for Reading First Technical Assistance or the Reading First Annual Performance Report (APR) database to identify background data specific to the Reading First programs. Other data collected from sources such as the National Center for Educational Statistics (NCES) databases and Reading First websites will be searched for relevant information about grantees. For example, Title I status, type of school, urban/rural locality status; and characteristics of the student populations such as poverty rates. This information can be used to describe grantees and make comparisons between groups of grantees. Grantee annual and progress reports may also be requested from ED and/or SEAs to help identify data on activities implemented by Reading First grantees. Further, information from IES or other studies might be found that can be triangulated with the findings from this study.

Pilot Testing

RMC is conducting a pilot test of the electronic questionnaire with six state and local grantees. Pilot testing ensures the clarity and ease of completing the surveys. Also, pilot testing will allow RMC to minimize any technological problems when the survey is administered. The pilot test will be conducted once the web survey is designed. In addition to grantees completing the survey, RMC will conduct brief interviews with respondents to make sure all comments, validations, suggestions, and issues with completion are understood and resolved. Results will be used to potentially improve the survey design and alter the electronic submission process if required.

Timetable

Pending Departmental and OMB clearance, the timeline for conducting this study is shown in the table below.

Task Description	Oct ‘08	Nov	Dec	Jan ‘09	Feb	Mar	Apr	May	June
Develop Survey Instruments
Finalize instruments in response to pilot	X
Obtain cover letter from ED					X
OMB Review
OMB Review	X	X	X	X	X
Revise and submit final to ED					X
Collect Grantee Contact & Grant Information
Contact sources for data about grantees & grants			X	X
Enter grantee and grant data into database			X	X
Develop Online Version of Survey & Database
Draft preliminary online survey forms and database	X	X
Contact 6 respondents & pilot test database			X
Finalize forms and database					X
Administer Survey
Prepare and e-mail online survey announcement					X
E-mail online passwords and link to web survey					X
Send reminders to complete online by e-mail						X
Collect & verify survey data					X	X	X
Analyze Survey Results
Analyze state and district survey data							X	X
Write Report
Write draft report								X
Submit draft report to ED for review								X
Revise report									X
Submit final report to ED									X

Part A. Justification

A.1 Circumstance Requiring Collection of Information

Need

The U.S. Department of Education (ED or the Department) Reading First program awards grants to state educational agencies to implement comprehensive reading instruction programs based on scientifically based reading research to children in kindergarten through third grade. Initial program grants were awarded to State educational agencies (SEAs) starting in July, 2002. SEAs were expected to award at least 80% of their grant funds to eligible local educational agencies (LEAs). LEAs have used these subgrants to fund Reading First programs.

Grantees at both levels are expected to spend the funds for professional development and training of teachers, technical assistance to schools, and program planning, administration, data collection, and evaluation. LEA subgrantees also use the money to develop the school reading programs, buying core reading program curricula, supplementary instructional materials and technology, assessments, and other programming.

The U.S. Department of Education Reading First program requires from state grantees applications, and annual performance reports that describe their program implementation. However, there is no formal or systematic mechanism for grantees to report on specific uses of Reading First grant funds.

In addition to filling a gap in their documentation of the Reading First program, data on expenditures may help the Department with its future grant-making and other states in the process of building upon their Reading First model. For example, learning where the states and LEAs have spent their funds can help identify the aspects of the program which require the most and least need for outside grant support. Expenditure data may also help identify which aspects of support are most likely to be utilized and by whom. For example, some types of districts or schools may require more professional development resources or more supplementary materials than others. Also, information from the study can help identify patterns in costs, particularly over time, which any scientifically-based reading program might encounter. Data from this survey would also enable the Department to better respond to requests for information from congressional and media sources.

Reading First is authorized by Title I, Part B, Subpart 1 of the Elementary and Secondary Education Act, as amended by the No Child Left Behind Act of 2001 (20 U.SC. 6361). The appropriate legal authority for collecting the data requested in this survey is Section 443 of the General Education Provisions Act, 20 USC 1232f (Records kept by recipient; full disclosure; maintenance period).

Authority

A.2 Indicate How, By Whom, and for What Purpose the Information is to be Used

The primary users of the information collected from this study will be the Department of Education. Collecting such information will help inform future grant-making efforts, as well as satisfy the informational needs of key stakeholders.

A.3 Use of Information Technology to Reduce Burden

The data collection relies on two sources of data. Initially, RMC will use electronic databases to collect data on grants and grantees that are already available from other sources and can be used in the analysis with the survey data. This approach minimizes asking questions of grantees that they have already answered or where the data are available elsewhere. For example, from this review RMC Research expects to locate information about the types and amounts of awards, and characteristics of the states, districts, and schools that can be used to describe grantees and make comparisons between groups of grantees. The review may also surface information from other studies that can be triangulated with the findings from this study.

For soliciting data directly from SEAs and LEAs, RMC will use employ internet-based methods. Through an e-mail notification, respondents will be encouraged to respond online using a code and password. A pilot test will be conducted of the electronic questionnaire with six or seven state and local grantees to ensure minimal technological problems when the survey is administered. This approach minimizes data entry error, eliminates the cost and reduces the time required of data entry required by paper forms. The stored data will be converted to an SPSS database for analysis.

RMC Research Corporation’s IT Director performs a security self-assessment review patterned on the NIST guidelines for low-baseline security assessments (Special Publication 800-53). Security requirements include: the updating of all applicable security controls; continuous monitoring; 3^rd party vendor security considerations; authorizing connections; physical security considerations; the creation of off-site backups; and the purging of retired equipment. An independent security review consists of a vulnerability scan of the internal network servers, routers, and selected workstations, as well as an assessment of perimeter security, including firewall configuration and administration.

A.4 Avoidance of Duplication

There is currently no systematic process in place for Reading First grantees to formally report the specific uses of the Reading First funds they received. SEA grantees and LEA subgrantees must describe the planned expenditure of their Reading First funds to the Department, and provide annual performance reports and three-year progress reports that describe their program implementation. Local evaluations and monitoring activities may be collecting such data but these data collections would not be systematic or consistent across the entire Reading First program. In 2007, the Department added a fiscal component to its Reading First monitoring activities; however, this component only looks at whether proper financial procedures are in place, including management, controls, and compliance with approved plans and statute. ED monitoring does not collect any information regarding the amount or allocation of SEA or LEA expenditures of Reading First funds. Furthermore, the Department has monitored only a small proportion of LEAs. Thus, there should be no duplication of information collected.

A.5 Methods to Minimize Burden on Small Entities

All entities participating in this data collection effort are either SEAs or LEAs receiving Reading First funds. None are small businesses or organizations.

A.6 Consequences of Not Collecting Information

Reading First has been the subject of intense scrutiny from the ED Inspector General, Congress, the media and various stakeholders. The Department needs to know and be able to provide information on how the bulk of the program’s funds were used by grantees and subgrantees.

A.7 Explain Any Special Circumstances

None of the special circumstances apply to these data collections.

A.8 Consultation Outside the Agency

RMC consulted with Abt Associates about any possible connection between the Study of Reading First Expenditures and the Reading First Impact Study, which Abt is currently conducting.

A.9 Payments of Gifts

No payments or gifts will be made to state or district personnel who cooperate with RMC in the collection of materials for this study. Their participation is strictly voluntary and uncompensated. SEA grantees and LEA subgrantees agreed to participate in any studies of Reading First in their grant/subgrant applications.

A.10 Assurances of Confidentiality

There are no confidentiality issues related to this collection.

A.11 Justification of Sensitive Questions

No questions of a sensitive nature will be asked of or about individuals. Reading First grantees are already required to report the total amount of their funds spent. The data collected for this study will simply ask respondents to provide a more specific breakdown of funds obligated by expenditure category.

A.12 Estimates of Respondent Hour Burden

Based on initial pilot testing, it is expected that most respondents will require about three hours to gather the data and complete the survey. Actual time required will be highly dependent on the quality of the system respondents used for record-keeping and archiving. The total number of annual respondents to the survey is 4,420 with a total of 13,260 response burden hours.

A.13 Estimates of Annual Cost Burden to Respondents

The collection of program information is a one-time data collection activity and is of SEA- or LEA-collected information known to exist and be readily available. There are no direct monetary costs to respondents for this activity other than the time required to locate, compile, and submit the requested information electronically. If respondents have trouble with submitting electronically, there could be a small cost for either faxing or mailing the information. At an estimated three hours to complete, $30 per labor hour, an estimated overall 80% response rate resulting in 4720 submitted surveys, the overall cost burden for information collection would be a total of $424,800.

A.14 Estimates of Annual Cost Burden to Federal Government

The estimated cost of the first data collection and analysis to the Federal government is $70,489. This is the total of already invoiced plus budgeted future costs charged to the government by RMC to prepare for, implement, and analyze the data collection. At this time, no further collections are expected.

A.15 Program Changes in Burden/Cost Estimates

This request is for a new information collection so no changes apply.

A.16 Plans/Schedules for Tabulation and Publication

Data will be presented in aggregated descriptive tables for each type of grantee (SEA and LEA). The data will be analyzed across time, comparing usage, activities, and obligated funds by year of program implementation to identify patterns and trends. An analysis plan is in preparation, but it is expected that specific types of comparison groups can be examined. For instance, comparisons will be made for size of grant, locality, poverty rates, or state/district size.

A.17 Expiration Date Omission Approval

Not applicable. All data collection instruments will include the OMB data control number and data collection expiration date.

A.18 Exceptions

Not applicable. No exceptions are requested.

8