AN EXPERIMENTAL STUDY OF
THE PROJECT CRISS READING PROGRAM ON
NINTH GRADE READING ACHIEVEMENT
IN RURAL HIGH SCHOOLS
PAPERWORK REDUCTION ACT
CLEARANCE REQUEST
SECTION A
Prepared For:
Institute of Education Science
United Stated Department of Education
Contract No. ED-06-CO-0016
Prepared By:
Northwest Regional Educational Laboratory
Center for School and District Improvement
Third Revised submission
October 19, 2007
Table of Contents
List of Exhibits ……………………………………………………………………………. ii |
||||
|
||||
SUPPORTING STATEMENT FOR PAPERWORK REDUCTION ACT SUBMISSION |
||||
|
|
|
|
|
A. |
JUSTIFICATION |
|
||
|
1. |
Circumstances Making Collection of Information Necessary …………………. |
1 |
|
|
2. |
Purposes and Uses of the Data …………………………………………………. |
3 |
|
|
3. |
Uses of Technology to Reduce Burden ………………………………………... |
4 |
|
|
4. |
Efforts to Identify Duplication …………………………………………………. |
4 |
|
|
5. |
Methods to Minimize Burden on Small Entities ………………………………. |
4 |
|
|
6. |
Consequences of Not Collecting the Data ……………………………………... |
5 |
|
|
7. |
Special Circumstances …………………………………………………………. |
5 |
|
|
8. |
Federal Register Comments and Persons Consulted Outside the Agency ……... |
5 |
|
|
9. |
Payments or Gifts ………………………………………………………………. |
6 |
|
|
10. |
Assurances of Confidentiality ………………………………………………….. |
6 |
|
|
11. |
Justification on Sensitive Questions …………………………………………… |
7 |
|
|
12. |
Estimates of Hour Burden ……………………………………………………… |
7 |
|
|
13. |
Estimate of Cost Burden to Respondents ……………………………………… |
8 |
|
|
14. |
Estimate of Annual Cost to the Federal Government ………………………….. |
8 |
|
|
15. |
Program Changes or Adjustments ……………………………………………... |
9 |
|
|
16. |
Plans for Tabulation and Publication of Results ………………………….......... |
9 |
|
|
17. |
Approval to Not Display OMB Expiration Date ………………………………. |
12 |
|
|
18. |
Explanation of Exceptions ……………………………………………………... |
12 |
|
|
|
|
|
|
REFERENCES ……………………………………………………………………………. |
13 |
|||
|
|
|
|
|
LIST OF APPENDICES ………………………………………………………………….. |
14 |
|||
List of Exhibits
1. Technical Working Group Members …………………………………………………... |
6 |
2. Hour Burden for Respondents …………………………………………………………. |
8 |
3. Project CRISS Study Timetable ……..………………………………………………… |
10 |
SUPPORTING STATEMENT FOR PAPERWORK REDUCTION ACT SUBMISSION
A. JUSTIFICATION
Circumstances Making Collection of Information Necessary
The Regional Educational Laboratories were authorized by Congress in the Education Sciences Reform Act of 2002, Part D, Section 174, (20 U.S.C. 9564). The central mission for the regional laboratories is to “support applied research, development, wide dissemination, and technical assistance activities by … developing and widely disseminating, including through Internet-based means, scientifically valid research, information, reports, and publications that are usable for improving academic achievement, closing achievement gaps, and encouraging and sustaining school improvement ….” This section goes on to include as part of this mission: “in the event such quality applied research does not exist as determined by the regional educational laboratory or the Department, carrying out applied research projects that are designed to serve the particular educational needs (in pre-kindergarten through grade 16) of the region ….” The act further specifies that one of the key activities undertaken by regional educational laboratories should be to “Identify successful educational programs that have either been developed by such laboratory in carrying out such laboratory’s functions or that have been developed or used by others within the region served by the laboratory….” The current research study addresses this authorization by conducting an experimental evaluation of a widely used and promising approach to improving adolescent reading comprehension.
To guide the work of the Regional Educational Labs, annual needs assessments are administered by each laboratory, pursuant to the legislative directive to conduct “a continuing survey of the educational needs, strengths, and weaknesses within the region ....” The most recent needs assessment conducted in 2004 included a survey of district superintendents and school principals in the Northwest Region and found that 80 percent of both groups rated “improving junior and senior high reading comprehension” as needing more or much more effort.
This identified concern is not surprising given that results of the 2003 National Assessment of Educational Progress (NAEP)—the Nation’s education “Report card”—indicate that only 30 percent of eighth graders achieved reading scores that were “proficient” or above,1 a percentage that has been more or less steady since 1992 (National Center for Education Statistics, 2004). At the twelfth-grade level, the latest NAEP results show only 40 percent of students at proficiency or above (National Center for Education Statistics, 2000). The results for states in the Northwest region mirror the national picture: the percentages of eighth graders scoring at or above reading proficiency on the 2003 NAEP were 27 for Alaska, 32 for Idaho, 33 for both Oregon and Washington, and 37 for Montana.
It is, therefore, clear that achieving reading proficiency in the early high school years is a high priority need in the Northwest, especially among poor and minority students. While this reflects a national trend, the Northwest is somewhat unique as a large geographic region that is sparsely populated with small rural districts, including those serving American Indian and Alaska Native students, and increasing numbers serving immigrant and migrant students of Hispanic origin. Poverty has also become an endemic problem in many Northwest rural communities since the downturn of the timber industry in the 1980s and repeated economic recessions. As a consequence, this study will focus on relatively small high schools in rural areas to represent high-need schools that often have fewer available programs and resources to help students with reading comprehension difficulties. Results will be generalizable not only to Northwest rural schools and students but to other similar rural areas throughout the country.
This study is designed to test the effectiveness of a program designed to improve reading comprehension called Project CRISS (which stands for Creating Independence Through Student Owned Strategies), a long-standing Montana-based educational program that uses extended teacher professional development and follow-up support to help teachers and students apply research-based learning principles to improve reading comprehension. The project developers estimate that CRISS training and technical assistance has been provided to teachers in more than 5,000 elementary and secondary schools in 41 states and several foreign countries.
Despite the identified need for improved instruction for secondary students, there is a serious lack of effectiveness studies of promising reading intervention programs for middle and high school students. While there is a preponderance of funded research on early reading acquisition or experimental studies in the elementary grades, there is a paucity of such studies at the high school level. Still, the problem of struggling high school readers has been well documented in national statistics and reports, and in our own regional needs assessment in the Northwest. Studying the effectiveness of a research-based adolescent reading comprehension intervention will address a gap in the research and at the same time address an important regional need.
There is preliminary evidence of the effectiveness of Project CRISS in studies using quasi-experimental research designs in both middle and high schools. Despite these positive indications, the existing research base on CRISS does not meet the U.S. Department of Education’s criteria to support a recommendation for widespread adoption of the instructional model. The studies conducted to date are based on quasi-experimental designs, and most have been conducted on middle class or lower middle class students who are primarily white. Under the priorities for No Child Left Behind (NCLB) and the Institute of Education Sciences (IES), there is a new commitment to promote the use of scientific methods in evaluating the effectiveness of educational interventions, which in turn will help schools select and adopt scientifically-based practices. Consequently, the proposed study will use a randomized control trial (RCT), the most rigorous scientific method, to evaluate the effectiveness of Project CRISS.
The Project CRISS Study is a randomized control trial (RCT) with the CRISS intervention systematically implemented to all core content teachers (reading/language arts, science, social science, and mathematics) in a sample of rural Northwest high schools randomly assigned to the treatment group. Project CRISS will be implemented in 33 treatment schools beginning in the 2007-08 school year for two successive years. During the second year (2008-09) of implementation—by which time teachers in the treatment schools will have received training and coaching in Project CRISS and will have had opportunities to integrate the learning principles and strategies into their classrooms—student outcome data will be collected via a fall and spring reading comprehension test administered to ninth-grade students in both treatment and control schools. Control schools (n=33) will also receive two years of Project CRISS after all student outcome and project implementation data have been colleted; that is, beginning in the 2009-10 school year. Data will be analyzed and a final report produced by 1/31/2011.
The overall goal of this study is to determine if Project CRISS can improve reading comprehension of early high school students thereby increasing the likelihood of success in course work, school engagement and school retention, and ultimately high school graduation and success in postsecondary education or work. It is beyond the scope of this study to track students longitudinally to assess long term effects. Instead, the goal is to first establish whether or not CRISS is an effective research-based intervention in a population of high-need Northwest schools, and to estimate the overall effect size on reading comprehension for ninth-grade students.
This study has a specific focus on Northwest high schools in rural areas with relatively fewer services and where poverty can be pervasive due to economic downturns in many rural communities. Project CRISS is a promising but unproven intervention with this population based on previous developer studies. The developer stresses that Project CRISS works to build the reading comprehension skills of all students when teachers across a school implement the strategies in all core content classes in English language arts, mathematics, science, and social science. Project CRISS is not designed as a special program or "pull out" program for high-need students or only for teachers who work with high-need populations. Therefore, the study is designed to show the impact of the Project CRISS intervention when it is provided to all core subject teachers in rural high schools. We have worked with the developer to define the intervention as it occurs in high schools they typically work with.
Specifically, the study is designed to answer the following research question:
What impact does Project CRISS have on the reading comprehension of ninth grade students in rural high schools?
We will also address two process questions on the implementation of CRISS in the treatment schools and the differences in classroom practices across the treatment and control schools:
(1) To what extent is Project CRISS implemented with fidelity in the treatment schools? What adjustments or adaptations are made from the program developer’s prescribed intervention in the actual implementation?
(2) How do classroom instructional practices in the treatment schools compare to instructional practices in the control schools?
Data to answer these questions will come from several sources:
Student outcome data will be collected through the administration of a standardized test, the Reading Comprehension subtest from the Stanford Diagnostic Reading Test, 4th Edition (SDRT-4), to estimate the effect of the intervention on student achievement.
Program and teacher implementation data will be collected through structured classroom observations, a CRISS local facilitator log, principal and teacher questionnaires, and trainer documents. These data will be used to document whether full or partial implementation of CRISS occurs across the treatment schools to help interpret the student outcome results. These data will also be used to describe the nature of instruction in the control schools—any differences in instruction between the treatment and control schools will determine student impacts—and to determine if some of the elements of Project CRISS intervention are present to a varying degree in control schools. The instruments and data collection are more fully discussed in Part B of the Supporting Statement (see Appendices A–H for background information and the instruments).
Purposes and Uses of the Data
The data described above will be used in a rigorous analysis (described later in this report) to estimate the impact of Project CRISS on student reading comprehension. The results from this study will be used by educators within NWREL’s region and throughout the country—most importantly, those who are considering using Project CRISS as an intervention to improve adolescent literacy in the early high school years. Improving adolescent literacy is a high-need area in the Northwest region as documented earlier. The results will provide scientific evidence and the effect size, if any, of Project CRISS in rural high school settings. Because this is a randomized study, the results will also be used by policy makers and others to identify promising national programs to help struggling high schools improve their reading/language arts test scores as they work to meet the demands of No Child Left Behind. Finally, the study will be used by other researchers and IES and add to the growing body of evidence about randomized control trials in education and “what works.”
Use of Technology to Reduce Burden
In order to reduce burden, available electronic information on the characteristics of the selected study schools will be obtained from existing data sources—including Web sites and national databases available to NWREL—rather than requesting this information directly from districts or schools. School names and basic characteristics will be downloaded from State Education Agency websites from the four participating states: Montana, Idaho, Oregon, and Washington. School demographic data will be downloaded from the National Center for Education Statistics, Common Core of Data: Public Elementary/Secondary School Universe Survey. This data set includes definitions for rural areas that will be used to classify schools as rural. For the demographic variable of poverty, district-level poverty rates for school-age children will be downloaded from the U.S. Census Bureau, Small Area Income and Poverty Estimates Program.
The SDRT-4 is currently available as a paper-and-pencil test (a computerized version is not currently available). Classroom observations will utilize the Vermont Classroom Observation Tool (VCOT) that has been adapted for this study by the instrument’s developer. Such classroom observations are best conducted by trained human observers physically present at the classroom. We currently plan to have the observers record their observations on written documents for later use by study analysts. The development of technology for this purpose was too expensive for this single study and probably more intrusive than the use of the written documents.
For the teacher and principal questionnaires, we will ask schools to complete their questionnaires online through a password protected Web site. This is more efficient for both the respondents and research team. In the event that small rural schools are unable to do this because of technology limitations, we will offer these schools a paper-and-pencil option. We will also collect monthly log data from Project CRISS Local Facilitators through an online form posted on the Internet. We will provide telephone assistance to help Local Facilitators complete their data reports, including filling out their form via telephone interview in the event the respondent does not have adequate access to a computer and the Internet at their school.
Efforts to Identify Duplication
Data on student achievement in reading are available from study schools in the form of the state standard test score. However, the study is being conducted in the four states in the Northwest region, and data from schools in different states cannot be combined in a meaningful way as each state’s established standards and student assessments are different. For this reason, the administration of a uniform norm-referenced test such as the SDRT-4 cannot be avoided. The remaining data collection instruments are intended to collect information that are not available through any other source.
Methods to Minimize Burden on Small Entities
This study focuses on relatively small high schools and districts, and steps will be taken to minimize data collection burden on these entities. School test coordinators who collect student data will be provided compensation for their time, and briefings on how to administer the test will be conducted by NWREL staff. Research and Project CRISS staff will be available to school personnel for questions or problems via toll-free telephone and email. Researchers will also make regular monthly contact with the in-school CRISS facilitator to gather information and answer questions, and assist them with completing the monthly log.
The principal and teacher questionnaires are short, each requiring about 10 minutes to complete per year over two years. We also limited the teacher questionnaire to ninth-grade teachers only since impact will be measured on ninth-grade students. For the SDRT-4, we are only including the Comprehension subtest rather than the entire battery to reduce student and test administration time to approximately one hour, once in the fall and once in the spring, for ninth-grade students only.
Consequences of Not Collecting the Data
As noted above, Project CRISS is currently being used in a relatively large number of schools yet its effectiveness has not been established using a scientifically rigorous method. The current study will enable a scientific test of the model that meets the current criteria of rigor set by IES. The study will be a cluster-randomized trial (CRT), which enables a test of causality. Without the study, schools and districts will continue to use this model based only on preliminary, non-experimental evidence of its effectiveness. That would hinder the current effort of NCLB and IES in promoting the use of interventions that have been tested for effectiveness in experimental studies, such as the CRT design used in this study.
Not collecting the data would prevent NWREL from carrying out its responsibility as a regional educational laboratory to identify successful educational programs through rigorous scientific research that meet identified regional needs. We have calculated sample sizes in order to have adequate statistical power to detect a minimum effect size of about 0.20, as discussed in Part B of the Supporting Statement. Collecting data on fewer cases would not allow us to detect a minimum detectable size on a standardized test that is typical of educational interventions like Project CRISS. The current study address the authorizing legislation, the Education Sciences Reform Act of 2002 as noted earlier, by conducting a rigorous experimental evaluation (with sufficient sample size) of a widely-used and promising educational intervention that lacks experimental evidence.
Special Circumstances
No special circumstances will exist in connection to the data collection for this study.
Federal Register Comments and Persons Consulted Outside the Agency
The notice for the data collection was announced on [insert page number] in the [insert date issue of the Federal Register]. Appendix J contains the Federal Register Notice.
[Insert the summary of public comments received in response to the notice, and our actions taken.]
The study team has drawn on the methodological expertise of two outside consultants: Michael Puma and David Connell from Chesapeake Research Associates, LLC in planning the current study, including the design, instrumentation, and plans for analysis. Additional review of the study design has been obtained from members of our internal technical working group (TWG). The names and institutional affiliations of the TWG members appear in Exhibit 1.
In addition, the study design has gone through extensive review by staff of the Analytical and Technical Support (ATS) contract established by IES to review all of the research being conducted by the 10 Regional Educational Laboratories.
Exhibit 1. Technical Working Group Members
Technical Working Group |
|
|
|
|
|
|
|
|
|
|
|
|
Payment or Gifts
Participating schools assigned to the treatment group will receive the Project CRISS intervention, including the training of all participating teachers and follow-up support by a certified Project CRISS national trainer. At the conclusion of the study, CRISS will be provided to the control schools.
There are no specific funds allocated for teacher and principal time to complete the questionnaires because they are very short, requiring about 10 minutes of time per respondent per year. The 10 minutes per month for the Local Facilitator log is viewed as part of the Project CRISS LF function and does not require significant additional time demands for data collection. The SDRT-4 Reading Comprehension test to ninth-grade students will be administered by project staff or consultants to ensure complete and high quality testing.
Assurances of Confidentiality
The Education Sciences Reform Act of 2002, Title I, Part E, Section 183, requires all collection of data to conform to the Privacy Act (5 USC 552), the confidentiality standards of subsection (c) of this section, and Sections 444 and 445 of the General Education Provision Act (20 USC 1232g, 1232h). These citations refer to the Privacy Act, the Family Educational Rights and Privacy Act, and the Protection of Pupil Rights Amendment. NWREL will establish procedures and safeguards consistent with these requirements for the collection, maintenance, and disclosure of information from or about identifiable individuals.
Student data collected from schools (a pre and post administration of the SDRT-4 Comprehension subtest) will not include individual student identifiers. The district test coordinator will be instructed to assign a unique subject number to students in order to identify them for pre- and post-testing. This number will be used to transmit the student data to NWREL. Only school personnel will have the list of subject numbers by student name and school ID. This procedure will allow the researchers to determine pre- and post-test scores of individual students, and determine the number of students who were present for only one of the two testings, without disclosure of student names or other identifying information to researchers.
Information obtained from the SDRT-4, classroom observations, the teacher and principal questionnaire, the Local Facilitator Log, and through various training-related documents to document CRISS implementation will be strictly confidential. As such, any information that NWREL obtains for the purpose of the study will be used solely for evaluating the effectiveness of the Project CRISS intervention. Results will only be reported in the aggregate such that no individual student, teacher, school, or district will be identified in any reports that are prepared and distributed. Statements to this effect will be included in the study questionnaires (Appendices D–H) and in the parent information and consent form (Appendix K).
NWREL uses a number of procedures to ensure appropriate access and storage of data in order to ensure confidentiality. Only staff members directly involved with the study will have access to study data. All research staff who collect data or have access to data will sign a confidentiality pledge (Appendix L). Computer data files are protected with passwords and access is limited to specific users. Where there is identifying information, a sample identification number is used to enter the data onto a database, and only a small number of individuals have the information linking subject numbers to respondents’ identification. This information and any other hard copies, data CDs, or materials with identifiable information is kept in locked files and cabinets. Discarded material is shredded and electronic files are deleted or destroyed upon study completion.
Justification on Sensitive Questions
No sensitive information will be collected through the instruments for which clearance is sought.
Estimates of Hour Burden
Exhibit 2 presents estimates of annual hour burden and costs for the classroom observation and questionnaire instruments. The observation instrument and questionnaires were pilot tested in three Project CRISS pilot schools in order to generate the average hour burden estimates in Exhibit 2.
Exhibit 2. Hour Burden for Respondents
Measure |
Expected Number of Respondents |
Average Hour Burden Per Administration |
Frequency of Administration Per Year |
Total Hour Burden Per Year |
Average Respondent Wage Rate |
Total Cost Per Year |
Classroom Observations (VCOT-CRISS) |
300 |
0.132 |
1 |
39 |
$42/hr3 |
$1,638 |
Teacher Questionnaire |
3964 |
0.17 |
1 |
67 |
$42/hr |
$2,814 |
Principal Questionnaire |
66 |
0.17 |
1 |
11 |
$69/hr5 |
$759 |
Local Facilitator Log |
33 |
0.17 |
9 |
51 |
$42/hr |
$2,142 |
Estimate of Cost Burden to Respondents
Aside from the hour burden detailed in Item #12, no additional cost burden to respondents is expected.
Estimate of Annual Cost to the Federal Government
The estimated cost to the federal government is $3,693,072 over five years—including designing the study, developing and pilot testing data collection instruments, recruiting schools, implementing the random assignment, providing Project CRISS to treatment schools and control schools on a delayed basis, processing and analyzing data, and producing interim and final reports. This total figure includes $1,481,800 in estimated payments to the developer to provide Project CRISS to treatment and control schools over a four-year period. The remaining $2,211,272 represents all research and evaluation costs. Therefore, the total average annual cost to the federal government is $738,614—$442,254 per year for research activities and $296,360 per year for the developer to provide Project CRISS to the study schools. These estimates are based on NWREL’s previous experience in conducting and managing similar research, evaluation, and school improvement projects.
Program Changes or Adjustments
The information collection activities described in this request are all new and therefore do not represent a change or modification to a previously submitted Clearance request.
Plans for Tabulation and Publication of Results
Data analyses will take place during the Year 4 of the study (i.e., during 2009-10), and the dissemination of a report on the study results is expected to be released in 2011.
The analysis of student outcome will have two components. First, outcome data will be subjected to a series of descriptive analyses to examine their central tendencies and dispersions. The student data will then be used to estimate the impact of CRISS on student achievement by comparing average outcomes for schools in the treatment (CRISS) group to those of schools in the control or non-CRISS group. In addition to simple mean differences, statistical models will be used to both improve the precision of the estimated impacts and to take into account the clustering of students within schools. As described in Part B of this Supporting Statement, such modeling will entail the use of hierarchical linear modeling (HLM) which is best suited to the nested structure of the study sample. Estimates of program impact will be obtained for the overall treatment group.
In addition to the primary analysis of impacts on student achievement, analysis of implementation data will be conducted to understand how well CRISS was implemented in the treatment schools, to describe the nature of instruction in both the treatment and control schools, and to look for any CRISS “contamination” in the control schools.
With regard to CRISS implementation, an index will be developed to assess how faithfully CRISS training and technical assistance, and Local Facilitator assistance, were implemented in the treatment schools. The descriptive data will be recorded in a database for each school and will include the number and hours of trainings, the number and percent of teachers attending trainings, the frequency and type of Local Facilitator activities, and any qualitative notes related to implementation. Descriptive statistics will be used to describe the extent and variation in Project CRISS implementation across the treatment schools.
The VCOT-CRISS observation instrument consists of separate rating scales for four dimensions of classroom instruction: planning/organization of the lesson, implementation of the lesson, content of the lesson, and classroom culture. A separate score will be developed for each of the four dimensions for each classroom observation and the average results will be compared across the treatment and control groups schools selected for this aspect of the study.
The teacher and principal questionnaires will be analyzed to determine any differences in teacher/principal qualifications and experience in treatment vs. control schools (ANOVA). Descriptive data on professional development occurring in control schools around adolescent literacy will be summarized by reporting frequency and types of these programs. Finally, the principal self-report from the questionnaire on Project CRISS walk-throughs for the treatment schools will be included in the implementation database described above.
The results of these data analyses will be summarized in a series of preliminary and final reports to be submitted to IES. Those reports will be produced for the purpose of summative evaluation. These reports, when approved by IES, will be placed on the National Laboratory Network Web site for public dissemination. The final report will only include aggregate statistical findings and will not disclose names of school districts, schools, or individuals within schools as explained previously in Part A under Assurances of Confidentiality.
Exhibit 3 contains: (1) the overall schedule of Project CRISS Study, (2) the detailed timetable for each major activity, and (3) the timetable for the reporting of results.
Exhibit 3. Project CRISS Study Timetable
(1) Overall Schedule of Project CRISS Study
(2) Detailed Timetable for Each Major Activity
Where, is an ongoing activity, X = treatment, O = measurement/observation, P = preliminary report, D = Draft Final Report, F = Final Report
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
(3) Timetable for the Reporting of Results
Annual Preliminary Report of Progress Annual Preliminary Report of Progress Annual Preliminary Report of Progress Draft Final Report of Findings Final Report of Findings |
October 2007 October 2008 October 2009 October 2010 January 2011 |
Approval to Not Display OMB Expiration Date
Exemption from the required display of the OMB expiration date is not requested. All the data collection instruments will include the OMB expiration date.
Explanation of Exceptions
No exceptions are requested.
REFERENCES
National Center for Education Statistics (2004). National Assessment of Educational Progress—The Nation’s Report Card: 2004 Long Term Assessment Results. Washington, DC: U.S. Department of Education, National Center for Education Statistics. (NCES Website: http://nces.ed.gov/nationsreportcard/)
National Center for Education Statistics (2000). Learner Outcomes. Indicator 13: Reading performance of students in grades 4,8, and 12. Washington, DC: U.S. Department of Education, National Center for Education Statistics.
LIST OF APPENDICES
NOTE: EACH APPENDIX IS SUBMITTED AS A SEPARATE ELECTRONIC DOCUMENT.
Appendix A: Schedule of Project CRISS Services
Appendix B: Vermont Classroom Observation Tool as Applied to Project CRISS
Appendix C: VCOT-CRISS Observational Protocol and Scoring Rubric
Appendix D: CRISS Local Facilitator Log of Activities
Appendix E: Teacher Questionnaire (Treatment Schools)
Appendix F: Teacher Questionnaire (Control Schools)
Appendix G: Principal Questionnaire (Treatment Schools)
Appendix H: Principal Questionnaire (Control Schools)
Appendix I: Justification for Items in the Teacher, Local Facilitator, and Principal Questionnaires
Appendix J: Federal Register Notice
Appendix K: Parent Information and Consent Form
Appendix L: NWREL Confidentiality Pledge
1 NAEP defines the proficient level as “solid academic performance for each grade assessed with demonstrated competency over challenging subject-matter” and “application of such knowledge to real world situations.”
2 Teachers are observed as they naturally teach their classrooms requiring no teacher burden, but there is sometimes a need for follow-up questions to clarify the observational context, which is estimated to range from 0 minutes to 15 minutes, with an average of about 8 minutes (0.13 hrs.) based on pilot testing.
3 The teacher hourly rate of $42 is computed based on an annual average teacher salary of $50,000 plus 35% additional benefits = $67,500 in annual salary and benefits. For a 200 day school year and 8 hour day, the hourly rate computes as (67,500/200)/8 = $42.19.
4 We estimate about 6 ninth grade teachers per school based on demographic information; 6 teachers*66 schools results in about 396 questionnaires from the treatment and control schools combined.
5 The principal hourly rate of $69 is computed based on an annual average principal salary of $90,000 plus 35% additional benefits = $121,500 in annual salary and benefits. For a 220 day school year and 8 hour day, the hourly rate computes as (121,500/220)/8 = $69.03.
| File Type | application/msword |
| File Title | An Investigation of the Impact of a |
| Author | NWREL User |
| Last Modified By | Sheila.Carey |
| File Modified | 2007-10-24 |
| File Created | 2007-10-24 |