Classroom Video Analysis (CVA) Assessment

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless such collection displays a valid OMB control number. The valid OMB control number for this information collection is 1850-xxxx. Public reporting burden for this collection of information is estimated to average 30 minutes per response, including time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. The obligation to respond to this collection is voluntary. If you have any comments concerning the accuracy of the time estimate, suggestions for improving this individual collection, or if you have comments or concerns regarding the status of your individual form, application or survey, please contact Janelle Sands at the Institute of Education Sciences (IES) at [email protected] directly.

Thank you for participating in the Regional Educational Laboratory Northeast and Islands’ evaluation of the Visual Access to Mathematics Professional Development (“VAM PD”) program. The following set of questions is expected to take 30 minutes to complete.

VAM Classroom Video Analysis (CVA) measure to assess teacher ability to analyze student thinking related to proportional reasoning

Instructions for each video clip

There are 6 video clips in this exercise. Once you start the task, the arrows in the upper right corner will help you to navigate through the video clips. The arrow pointing up will take you back to this page.

(1) Start by reading the short clip description above the video player before you watch each video.

(2) Use the play icon to start the video clip. The pause and play icons and the slider under the video clip allow you to pause, watch, or rewatch the video clips.

(3) Answer the prompt. Save your response by clicking Save Draft. You may edit saved responses. Please check the box Mark as Final once you are ready to submit your response. Once you have submitted your response as final, you are not able to go back and change it at a later time.

(4) Use the red forward arrow in the upper right corner of the page to navigate to the next video. Please note that some clips might take a few minutes to load. You may click on the Start tab and see your progress in the red circle diagram on the upper right corner of the page.

(5) Once you have marked all of your responses as final, you have completed the video analysis task and you will receive a completion email.

To start, click on Clip 1 in the list on the right.

We really appreciate your participation and look forward to reading your responses. Thank you!

Prompt for each video clip:

Please comment on what mathematical ideas the student(s) in the clip might or might not understand. Be specific.

Clips:

Clip 1

Description:

The students are working in small groups and were asked to solve the following problem: “A pelican can hold 3 gallons of water in its mouth and a gallon of water in its stomach. If you were like a pelican and you could fit half a gallon of liquid in your stomach, how much liquid could you fit in your mouth?” The clip shows the teacher helping one group that is working on this problem.

Clip 2

Description:

In this lesson, students compare two lemonade recipes (the 1st recipe has 3 lemons to 2 cups water, and the 2nd recipe has 6 lemons to 3 cups water). The students have already written several conjectures about which recipe is “more lemony:; the clip picks up during the discussion about whether one conjecture is correct or incorrect. After some students claim one conjecture is not correct, the teacher provides another two recipes to add to the conversation.

Clip 3

Description:

In this lesson students learn how to solve a proportion. They are working on the following problem: “How many students need to hold hands to cover the distance from LA to Long Beach (99,845 feet)?” To help solve the problem, the teacher had them determine that it takes five students to cover a 20-foot rope. The clip begins with the teacher checking on the progress of one of the students.

Clip 4

Description:

In this screencast clip, a sixth-grade student is working with a teacher on the following task: “Maureen ordered 4 bags of soil for her flower gardens. Each garden needs 3/4 of a bag of soil. She wants to know how many gardens she can fill completely and how much soil she has left over.” The clip begins as the student draws a visual representation to help her solve the problem. There are two rows of subtitles in this clip; the first row of subtitles is in Spanish, and the second row of subtitles is an English translation.

Clip 5

Description:

After a conversation about how much sugar is needed to bake a cake, and the more ingredients you use the more people you can serve, the teacher poses the following problem: “If 2 cups of sugar are needed for 4 people and 4 cups of sugar for 8 people, how many cups of sugar are needed for 12 people and how many cups are needed for 15 people?” One student explains her answer.

Clip 6

Description:

The lesson is to consider two different lemonade recipes that use different amounts of lemon and water, conjecture which recipe makes the more lemony lemonade, and then prove or disprove these conjectures. The students have already worked in small groups to provide conjectures. The clip picks up during the whole class discussion while the students are analyzing the correctness of these conjectures. Two students share how they are now thinking about the problem.

Scoring Guide for the VAM Classroom Video Analysis measure

Overview of measure. The VAM Classroom Video Analysis (CVA) measure includes six clips showing students and a teacher working on tasks on ratio or proportional reasoning. Teachers are asked to view the clips and respond to the following prompt: “Please comment on what mathematical ideas the student(s) in the clip may or may not understand. Be specific.”

Overview of scoring.

The VAM CVA scoring rubric was developed in consultation with the author of the CVA and is based on the author’s Student Thinking scoring rubric (Kersting, 2008; Kersting et al., 2010; Kersting et al., 2012). We code teachers’ responses to each item for the presence of (a) a claim about students’ understandings of specific mathematical concepts or ideas, and (b) evidence to support the claim. Teachers’ responses can receive a score of 0, 1, or 2 for each item. A teacher’s average score for all items represents the teacher’s ability to analyze students’ understandings of ratio and proportional reasoning content as it is displayed in video clips of classroom learning activities. Rubric scores for each item represent the following:

• A score of zero: The response does not include a claim about what a student understands or does not understand about a specific mathematical idea or concept related to ratio and proportional reasoning in the clip. A score of zero may include unclear or incomplete claims, a blank response, or a description of students’ verbal and/or written output that does not connect to any claim. Without such a claim, it is not possible for the response to have evidence to support the claim.

• A score of 1: The response includes a claim about what a student understands or does not understand about a key mathematical idea or concept related to ratio and proportional reasoning in the clip but does not include evidence to support the claim.

• A score of 2: The response includes a claim about what a student understands or does not understand about a key mathematical idea or concept related to ratio and proportional reasoning in the clip and includes evidence to support the claim. The evidence must come from a description of students’ verbal and/or written output that makes a clear and logical connection to the claim.

Table I summarizes the conditions under which raters assign a score of 0, 1, or 2 to a teacher’s response. Table II provides additional details about how the creators of the measure define claims and evidence in this exercise.

Table I. Scoring rubric for the VAM CVA exercise

Score to assign	When participant response includes the following:
	a. A claim about what students understand or do not understand about a specific mathematical idea or concept related to ratio and proportional reasoning content in the clip.	b. Specific and relevant evidence for the claim (from a) with a description of students’ verbal and/or written output linked to the claim.
0	No	No
1	Yes	No
2	Yes	Yes

Note: Rubric adapted from Kersting et al., 2012, and used in a prior study of VAM PD (DePiper et al., 2021b)

Table II. Scoring rubric details on claims and evidence in teacher responses to the VAM CVA exercise

The response includes:	Coding notes
A claim about what students understand or do not understand about a specific mathematical idea or concept related to ratio and proportional reasoning in the clip	Definition of a “claim” in this exercise: A claim is a conclusion or assertion about what a student understands or does not understand about the ratio and proportional reasoning content in the video clip. Given that the clips show student work or students talking about a task, a claim can be what the respondent thinks the student is thinking or understands. A respondent can suggest tentativeness or uncertainly about a claim; that is, a response can say, “I think” or “I’m not entirely sure, but…” The claim must be about ratio/proportional reasoning content and the work that the student is doing on the task and cannot be general, such as “he understands ratios” or “he understands proportions.” The claim must be about what the student(s) understand or do not understand and not only about what the teacher is doing. Notes about possible claims: Possible qualifying and non-qualifying claims differ by clip, with examples listed below. Possible qualifying claims must refer to whether the student understands a specific mathematical idea or concept in the clip. The wording in the response should be consistent with, and as detailed as, what is below; but it does not need to match the samples below. The claim can refer to a student’s understanding (or lack thereof) regardless of whether the response includes the words “student understands” or “student does not understand.” A respondent can make a qualifying claim even if it is not a claim that others would agree with. Possible nonqualifying claims include references to students in general, not just to students in the class depicted in the video. Statements without details about the task or context do not count as a qualifying claim. The part of the response that specifies the claim can also contain the evidence (see section b below for notes about evidence). Because the evidence can help to clarify the claim, a sentence or statement can present both claim and evidence. Possible qualifying and non-qualifying claims by clip (Examples for three clips shown below) Clip 1 (pelican mouth/stomach water-holding ratio) Qualifying claims include: Statements about students’ understanding (or lack thereof) of proportional relationships, relationships, or ratios. Must include details about the task or context. Example: Student may not understand that the relationship between volume in stomach and volume in mouth sack can be written as a ratio. It is not clear that the student understands when the teacher asks, “What is the factor?” or whether he understands rates/rate of change in this context. Statements about students’ understanding (or lack thereof) of additive or multiplicative reasoning or ratios. Examples: The student understands how to subtract or add to find the difference between 3 and 1, and he is trying to apply that process to find the volume. Student may not understand multiplicative reasoning or how multiplicative reasoning can be used in this context (or in proportions). Statements about student understanding (or lack thereof) of setting up and/or solving proportional relationships. Examples: Student does not understand how to set up the proportional relationship (or which quantities to compare in the setup of the relationship). The student may not understand how to decontextualize the amounts into a proportion. Nonqualifying claims include: Statements that omit details about the task or context. Example: He doesn’t understand proportional relationships. Statements that do not connect to proportional reasoning. Example: Student doesn’t understand inverse relationships of multiplication and division. Clip 3 (lemon/water – conjecture about more lemons = more lemony) Qualifying claims include: Statements about student understanding (or lack thereof) of proportional relationships, ratios, part-to-part (or part-to-whole, and so on), equivalent fractions. Must include details about the task or context. Examples: Marticia understands that it is important to consider how the amount of water or how lemon to water ratio (not just one of the parts) influences the flavor. Jose understands how one ingredient (lemons) influences the flavor and concentration but may not understand how the lemon to water ratio (not just one of the parts) influences the flavor. Statements about student understanding (or lack thereof) of a relationship, accompanied by details that qualify/explain the nature of the relationship and articulate that relationship. Example: Jose does not understand the relationship of lemons to water. He thinks that it would be more lemony if there are more lemons but is not taking into account the amount of water. Nonqualifying claims include: Statements that refer to students in general, not the students in the class. Statements that lack details about the task or context. Statements that simply mention “lemony” or “lemoniness” without connection to a mathematical idea. Clip 5 (3/4 bag of soil for each flower garden, 4 bags of soil) Qualifying claims include: Statements about student understanding (or lack thereof) of the gardens that can be filled and the amount of soil left over. Examples: The student understands that she can fill 5 gardens and there is ¼ of a bag of soil leftover. She understands that from the ¼ left from each bag, she can form another garden. She may not understand the fraction of a garden that can be filled with the ¼ of a bag of leftover soil. Statements about how students recognize the ratio or proportional relationship in the task. Examples: She understands the ratio of one garden for every ¾ bag of soil. She understands the proportional relationship that for every ¾ bag of soil, there is one garden. Statements about student understanding (or lack thereof) of fraction division: She understands division of a whole number by a fraction. The student can show division by a fraction with a diagram. The student knows how to use a diagram to show division by a fraction. Nonqualifying claims include: Statements that refer to students in general, not the students in the class. Statements that lack details about the task or context. Examples: Student understands fractions. Student understands division (or concept of division). Student understands division of fractions. Student understands ratios or unit rates. Students understand the relationship between the number of bags of soil and the number of gardens. [Note: Here, the relationship is not defined.]
Provide specific and relevant evidence for the claim (in a) with a description of students’ verbal and/or written output linked to the claim	Definition of “evidence” in this exercise: A description of student (or students’) work that relates to and supports the claim. To receive a “Yes” for this criterion, the response must: Include a claim as defined above. Include a description of student (or students’) work. This includes direct or paraphrased student talk or action from the clip—for example, detailing what a student produced verbally, visually, or physically (something they said, drew, gestured, and so forth). Provide an accurate description of student (or students’) work. Inaccurate descriptions do not count as evidence for a claim. Provide a description of student (or students’ work) that discusses the same topic and relates to and supports the claim provided.

Note: Rubric adapted from Kersting et al. (2012), and used in a prior study of VAM PD (DePiper et al., 2021b)