2008-2009 Original and Replication Studies Teacher Class

OMB Number: xxxx-xxxx

Expiration Date: xx/xx/07

Appendix E

AMSTI Study Teacher Classroom Observation Protocol

AMSTI Study Teacher Classroom Observation Protocol

A.Overview and Guidelines

1.Purpose of the observation

The major goal for these observations is to describe the mathematics or science instruction in the teachers’ classrooms, to document students’ participation in instructional activities, and to determine the extent to which the teachers’ instructional practices are related to the AMSTI model and materials. For purposes of this instrument, the AMSTI model is defined as a set of reform-oriented mathematics practices endorsed by the National Council of Teachers of Mathematics and reform-oriented science practices endorsed by the National Academy of Sciences. In the case of the AMSTI classroom observations, the purpose is to determine the extent to which the school is implementing AMSTI as it was designed to be implemented. In the case of the control school observations, the purpose is to determine the extent to which the control schools are similar to AMSTI schools in their instructional approach and practices. This observation form is adapted from a synthesis of the Authentic Instructional Practices Classroom Observation form (Borman, G., Rachuba, L., Datnow, A., Alberg, M. Stringfield, S., & Ross, S.; 2000.) and the Reformed Teaching Observation Protocol (MacIsaac, Sawad, Daiyo & Falconer, 2001).The protocol was developed in a way that facilitates quantitative ratings and development of constructs, with a goal being that it will inform the development of an AMSTI implementation index, in conjunction with other data (e.g. teacher and principal surveys). Additionally, the researcher is asked to describe the lessons and activities and justify some ratings qualitatively.

2.Procedures

Schedule the observation for a full class period, lasting at least 45 minutes. Please enter the room at least five minutes before the class is scheduled. Introduce yourself to the teacher and indicate that you will be observing the class as part of the AMSTI Evaluation Study. Indicate to the teacher that (1) he/she has the option at that time to elect not to be observed or to end the observation at any time; (2) he/she will have the opportunity after the class to explain/elaborate on points made during class (3) all notes taken during the class will be confidential, and not disseminated in any way that would make the teacher individually identifiable and (4) after class, the researcher will debrief the teacher further on the study.

The collection of information in this study is authorized by Public Law 107-279 Education Sciences Reform Act of 2002, Title I, Part C, Sec. 151(b) and Sec. 153(a). Participation is voluntary. You may skip questions you do not wish to answer; however, we hope that you will answer as many questions as you can. Your responses are protected from disclosure by federal statute (PL 107-279 Title I, Part C, Sec. 183). All responses that relate to or describe identifiable characteristics of individuals may be used only for

statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose, unless otherwise compelled by law. Data will be combined to produce statistical reports. No individual data that links your name, school name, address, telephone number, or identification number with your responses will be included in the statistical reports.

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB control number.  The valid OMB control number for this information collection is XXXX-XXXX (expiration date: XX/XX/07).  The time required to complete this information collection is estimated to average 60 minutes, including the time to review instructions, search existing data resources, gather the data needed, and complete the information collection.  If you have any comments concerning the accuracy of the time estimate or suggestions for improving this form, please contact: the Department of Education 50 North Ripley Street PO Box 302101 Montgomery, AL 36104.  If you have comments or concerns regarding the status of your individual submission, e-mail directly to: [email protected].

3.Note taking during the observation

When taking notes during the observation, keep in mind the overall purpose of the observation: to describe the mathematics or science instruction in the teachers’ classroom, to document students’ participation in instructional activities, and to determine the extent to which the teachers’ instructional practices are related to the AMSTI model and materials (or in the case of control schools, the extent to which practices and approaches are AMSTI-like). Collect copies of any handouts or materials that are relevant to lesson. The following provides some guidelines on how to focus your note taking during various activities.

Teacher-directed activities

During teacher-directed activities, in either whole class or small group settings, focus note taking on that activity. Try to provide as complete a description as possible of the activity. Your notes should address:

• The type of activity

• the way the teacher sets up the activity

• materials used, including technology and instructional

• questions the teacher asks the students

• responses the teacher gives to students’ questions

• students’ comments and responses

• grouping of students and how long they were grouped

Small-group work

When students work in small groups you will have to decide if you should follow the teacher to see what she is doing with each of the small groups or stay with a group long enough to determine what they are doing and how they are understanding the activity. A balanced approach will help you get the most out of the observation. To get a sense of the teacher’s instructional approach and rapport with students, shadow the teacher and record as much of the conversations and interactions he/she has with individuals and groups as possible. Then spend some time with individual students or small groups. Notice the way students approach the task/activity, what they are doing and how much they seem to understand about the task and knowledge/concepts taught through the task.

Completing the Observation Forms

The following forms should be completed as soon after the observation as possible. The sooner you complete the form, the easier it will be to remember the observation and expand on your hand-written notes. The form includes five sections: Part I: Descriptive information, Part II: Narrative; Part III: Objectives emphasized in the classroom; Part IV: Authentic Instruction Principles; and Part V: Classroom context.

AMSTI Study Teacher Classroom Observation Form

Part I. Descriptive information

Date:

Observer:

Teachers’ name and school:

Time of observation: From: to:

Grade level:

Subject area:

Number of children in the room:

Race/ethnic breakdown of children:

African American: White: Latino: Asian: Not sure:

Gender breakdown of children: Boys: Girls:

Brief description of the classroom setting: How are the seats arranged (you may draw a map or describe)? How is the room decorated? What materials are available?

Part II: Narrative

Please describe in narrative form the activities that took place in the class. This should be a sequential account of the session. In your description, address the following:

What was the nature and objective of the activity(ies)? What were the instructions for the activity(ies)? Please attach the written instructions that students received for any aspect of the assignment. Explain any additional instructions (i.e those that were given orally, put on the blackboard or displayed visually in some other manner). Describe the materials and resources that were provided for the students for this activity (e.g. text books, encyclopedias, trade books, calculators, teacher made materials, etc.)

1. List the curriculum/textbook used during the observed class:

2. Was the lesson from the AMSTI materials provided in the summer training (if it is not clear based on your observation, ask the teacher during the follow-up interview)?

Yes No Partially

If partially, describe what part was AMSTI and what part was not AMSTI.

Part III. Objectives Emphasized in Classroom

Rate the classroom on the degree of emphasis the teacher placed on having students learn the following objectives. Emphasis here does not necessarily refer to the frequency or rate that students were engaged in related tasks. Rate level of emphasis based on the priority or value the teacher placed on developing students' skills within each area. Ask the question to yourself, "How central was this objective to the substance of the lesson?" Explain and provide examples of each.

1. Knowledge (e.g., recalling information or memorizing facts or principles)

Level of Emphasis: 0 = None 1 = Low 2 = Moderate 3 = High

Explain and provide examples for your rating:

2. Comprehension (e.g., interpreting or showing understandings of facts, such as interpreting the meaning of a graph, inferring the principle underlying a science experiment, or estimating)

Level of Emphasis: 0 = None 1 = Low 2 = Moderate 3 = High

Explain and provide examples for your rating:

3. Application (e.g., using principles or knowledge to solve novel or real-life problems, such as applying knowledge of the relationship between temperature and pressure to understand why a balloon is larger on a hot day than on a cold day, or applying an algorithm to a math problem)

Level of Emphasis: 0 = None 1 = Low 2 = Moderate 3 = High

Explain and provide examples for your rating:

4. Analysis (e.g., breaking down complex information or ideas to understand how the parts are related or organized; such as describing the strategy for solving a multi-step problem)

Level of Emphasis: 0 = None 1 = Low 2 = Moderate 3 = High

Explain and provide examples for your rating:

5. Synthesis (e.g., using knowledge and skills to create completely new products, such as designing a science experiment, or deriving a mathematical rule).

Level of Emphasis: 0 = None 1 = Low 2 = Moderate 3 = High

Explain and provide examples for your rating:

6. Science only: Evaluation (e.g., making value judgments against some criterion or standard, such as evaluating the adequacy of science experiment to answer a particular research question)

Level of Emphasis: 0 = None 1 = Low 2 = Moderate 3 = High

Explain and provide examples for your rating:

Part IV. Authentic Instruction Principles

Instructions: For each of the five dimensions below, indicate the rating that most accurately describes the observed lesson. Provide examples that support your rating.

1. Coherence of Material

0 = Material is presented in superficial fragments with very little connection between parts.

1 = Some over-arching concepts or ideas are covered, but they are not well connected to the whole lesson content.

2 = Some over-arching concepts or ideas are covered in moderate depth. There are periods of sustained focus on these significant topics that are key to the whole lesson content. Coverage is uneven though; other key concepts or ideas are superficially covered.

3 = Key concepts/ideas are covered in depth. The lesson content is presented as a whole, and is structured in a way that allows for the sequencing and structuring of a complex topic. Each topic appears to build on another in an effort to foster deeper student understanding.

Provide examples that support your rating:

2. Connection to Students' Out-of-School Experiences

0 = Lesson topic and activities have no clear connections to out-of-school-experiences.

1 = Students encounter a sub-topic of the lesson and the teacher tries to connect it to students' experiences (e.g., the topic of "beach" is encountered and teacher says, "how many of you have been to a beach..."). The purpose of activating prior knowledge is to aid in describing the topic or making concrete something abstract.

2 = Students study a topic or issue that is directly connected to their experiences. For instance, they may do a word problem related to their community environment. Students do not, however, explore these connections in ways that create personal meaning or significance for their lives.

3 = Students see the connection between lesson material and their lives. The lesson allows them to enhance their understanding of their cultural and self values, and their aspirations. For instance, students may have to solve a mathematics problem that involves estimating the size of their apartment building based upon the length of the shadow the sun produces at midday.

Provide examples that support your rating:

3. Connections with other content disciplines.

0 = Lesson topic and activities have no clear connections to other content disciplines.

1 = Students find that topics on one content discipline are related to topics in other content disciplines, but only on a factual level. For instance, students use data collected on temperatures to explore the differences among means, modes and medians, or math is used as part of an experiment to investigate properties of buoyancy. Students do not, however, explore these connections in ways that create a deeper understanding of the connections between subjects.

2= Students find that topics on one content discipline are related to topics in other content disciplines, but on a conceptual level (e.g. using math to examine personal and societal issues, such as conducting analysis of demographic and economic data to determine the best location for a new pharmacy). The lesson allows students to enhance their understanding of the connections between subjects.

Provide examples that support your rating:

4. Substantive Conversation

0 = There is no probing of student answers and no discussion of the lesson material

1 = Teacher occasionally probes for student answers. Students are not solicited by the teacher for their opinions, but they're not discouraged from providing them either.

2 = Students are encouraged to state their ideas/opinions, but there is little conversation occurring among students --most dialogue is directed through the teacher. The lesson is not completely scripted by teacher or lesson material. Teacher tends to probe answers.

3 = Students are encouraged to converse among themselves about the lesson material. The teacher may or may not engage in the conversations. There is evidence that the purpose of sharing ideas is to arrive at a deeper understanding.

Provide examples that support your rating:

5. Teacher Support for Students

0 = The rapport between teacher and students is not good. The working relationship between teacher and students is rarely constructive. Many students do not appear to like the setting.

1 = Teacher support for students is mixed. Teacher praises students occasionally. At other times student effort goes unnoticed. Students are not encouraged to support one another.

2 = Teacher support is usually positive. Teacher-student rapport is good. There is some evidence of high expectations for learning and trying hard. Teacher focuses on student successes and does not dwell on failures.

3 = A strong friendship and mutual trust develops between teacher and students. The atmosphere clearly supports student effort. Lowest achieving students receive support from all.

Provide examples that support your rating:

6. Student Engagement

0 = Most students appear to be inattentive. They may look as though they are bored and preoccupied with thoughts unrelated to the task at hand. One or more students may be disruptive.

1 = Most students appear to be occasionally on-task. For those that are on-task, however, they seem to be rather lethargic and/or not trying very hard.

2 = Most students for most of the time are on-task pursuing the substance of the lesson. Students have, however, occasional lapses in concentration. A few students are only occasionally off-task.

3 = All but one or two student(s) are deeply engaged in the lesson (paying attention, clearly interested in learning the material, concentrating) for all but a few short instances of the lesson.

Provide examples that support your rating:

Part V. Classroom Context

1. Technology

Technology includes: overhead projector, calculator, graphing calculator, headphones, computers, TV, VCR, LCD displays, etc.

0= no technology was used during the lesson.

1= Technology was used by the teacher, but not the students.

2= Students used technology during instruction and/or to work on an assignment. However, the students used the technology procedurally (in a step-by-step method).

3= Students used technology to investigate and arrive at a deeper understanding, rather than procedurally.

Describe the technology used by the teacher and students:

Teacher:

Students:

List the technology present in the classroom that was not used by either the teacher or the students during the observed lessons:

2. Manipulatives (e.g. counting cubes, Cuisenaire rods, blocks, geoboards, pegboards, tiles, fraction circles, geometric shapes, etc.)

0= no manipulatives were used during the lesson.

1= manipulatives were used by the teacher, but not the students.

2= Students used manipulatives during instruction and or to work on an assignment. However, the students used the manipulatives procedurally (in a step-by-step method).

3= Students used manipulatives to investigate and arrive at a deeper understanding, rather than procedurally.

List the manipulatives used during the observed lessons:

3. Student grouping

For each type of student grouping, describe the activity students worked on in that grouping and approximately how long they were grouped in that configuration.

A) students worked individually

Yes No (skip to B)

If yes, what was the activity they were working on?

How long were they grouped like this?

less than 25% of the time

More than 25% but less than half of the time

Between 50% and 75% of the time

More than 75% of the time

B) Students worked in pairs

Yes No (skip to C)

If yes, what was the activity they were working on?

How long were they grouped like this?

less than 25% of the time

More than 25% but less than half of the time

Between 50% and 75% of the time

More than 75% of the time

C) Students worked in small groups

Yes No (skip to D)

If yes, what was the activity they were working on?

How long were they grouped like this?

less than 25% of the time

More than 25% but less than half of the time

Between 50% and 75% of the time

More than 75% of the time

D) Students were taught as a whole class

Yes No (skip to next question)

If yes, what was the activity they were working on?

How long were they grouped like this?

less than 25% of the time

More than 25% but less than half of the time

Between 50% and 75% of the time

More than 75% of the time

4. Assessments: formal and informal

List specific student assessment instruments used during the class and where they came from (teacher made, from the text/curriculum, district etc.). Formal assessment instruments might include: quiz, test, portfolio, student presentation, homework, etc. Informal might include individual, small group and whole group questioning.

Formal assessments used in class:

Type of Assessment	Source of assessment (e.g. teacher made, text/curriculum, district, etc.)
□ quiz
□ test
□ student presentation
□ homework
□ portfolio
□ other (specify):

Informal assessments used in class:

Type of Assessment	Source of assessment (e.g. teacher made, text/curriculum, district, etc.)
□ individual questioning
□ small group questioning
□ whole group questioning
□ worksheets/workbooks
□ board problems/to do problems
□ other (specify):

5. Teacher comfort level with lesson

How comfortable did the teacher appear to be with delivering the lesson and materials she/he was presenting during your observation?

Not at all comfortable

A little comfortable

Fairly comfortable

Very comfortable

Please explain your rating.

6. Teacher knowledge of subject matter

To what extent did the teacher show a solid grasp of the subject matter content inherent in the lesson?

____Not at all

____A small extent

____A moderate extent

____A great extent

Please explain your rating.

7. Any additional comments about the class?

10

R

egional Educational Laboratory