Appendices

OMB math study appendices 9-10-06.pdf

Evaluation of Math Curricula (KI) v.3

Appendices

OMB: 1850-0813

Document [pdf]

Download: pdf | pdf

APPENDIX A
Letter to School Principals Requesting Class Lists
Letter to Teachers and Fall (Baseline) Survey

MATHEMATICA
Policy Research, Inc.
P.O. Box 2393
Princeton, NJ 08543-2393
Telephone (609) 799-3535
Fax (609) 799-0005
www.mathematica-mpr.com

Dear Principal,
Thank you for agreeing to participate in the Evaluation of Mathematics Curricula. Mathematica
Policy Research, Inc. (MPR), along with RG Research Group and SRI International, is conducting the
study for the U.S. Department of Education’s Institute of Education Sciences. As part of the study, we
develop our student sample from class rosters containing names of all children enrolled in each
sampled teacher’s mathematics class. Your district has indicated that the most accurate class listings
should be obtained from the school.
Your school will be implementing the __________ curriculum in all first grade mathematics
classrooms. As you know, each classroom will be observed once during the school year and teachers
will be asked to complete questionnaires in the fall and spring. Additionally, student achievement will
be measured using the ________ mathematics assessment in the fall and again in the spring.
Enclosed is a list of all teachers who have enrolled in the study. Please check this list to be sure it
includes all the teachers who teach first grade mathematics at your school and provide updates as
appropriate.
Also, please provide a list of all students in each teacher’s math classroom. You may use existing class
rosters. For each teacher, please provide a complete list of all students enrolled in his/her math class
including:
First Name
Last Name
Gender
Date of Birth
IEP Status or any disability status that should be considered for student testing
NonEnglish Speaker status that should be considered for student testing
All student information will be kept confidential and used for research purposes only. While your
school’s participation in this study is voluntary, collecting accurate class lists is essential for building
our student sample. As one of about 100 schools in the study, every response is critical. Class lists
and updates to our teacher list can be returned to Mathematica Policy Research in the envelop
provided.

If you have any questions, please contact the study’s survey director Sheila Heaviside at 1-866-8693187, or by email at or [email protected].
Thank you for your help with this important study.
Sincerely,

Roberto Agodini
Study Director

MATHEMATICA
Policy Research, Inc.
P.O. Box 2393
Princeton, NJ 08543-2393
Telephone (609) 799-3535
Fax (609) 799-0005
www.mathematica-mpr.com

Dear Teacher,
You and your school are participating in the Evaluation of Mathematics Curricula. Mathematica
Policy Research, Inc. (MPR), along with RG Research Group and SRI International, is conducting the
study for the U.S. Department of Education’s Institute of Education Sciences. As part of the study, we
are gathering information about your experiences with the curricula you are piloting and training and
support you have received, along with some background information about you.
We greatly appreciate your participation in this study, and as part of that participation, ask you to
complete the enclosed questionnaire. All of the information that you provide will be kept confidential
and used for research purposes only. No one at your school will see your responses, and you will not
be identified in any report or presentation. While your participation in this study is voluntary, it is
very important that you complete the questionnaire. As one of only a few hundred teachers in the
study this year, every response is critical in understanding teachers’ classroom experiences. We have
enclosed a check for you in appreciation of your participation.
If you have any questions, please contact the study’s survey director Sheila Heaviside at 1-866-8693187, or by email at [email protected].
Thank you for your help with this important study.
Sincerely,

Roberto Agodini
Study Director

TEACHER SURVEY (Fall 2006)
NATIONAL EVALUATION OF ELEMENTARY MATHEMATICS CURRICULA
U.S. DEPARTMENT OF EDUCATION

ATTACH LABEL HERE
Teacher ID Teacher Name
School ID School Name
IF ABOVE INFORMATION IS INCORRECT,
PLEASE MAKE CORRECTIONS DIRECTLY ON LABEL.

This survey is part of the Evaluation of Elementary Mathematics Curricula, a national evaluation
being conducted for the U.S. Department of Education. All information you provide will be kept
confidential. While you are not required to respond, your cooperation is needed to make the
results of this survey comprehensive, accurate, and timely. Thank you.

Please return the completed form to:

If you have questions, please contact:

This survey is authorized by the U.S. Department of Education (P. L. 20 U.S.C. 1221e.1) and the
Confidential Information Protection and Statistical Efficiency Act of 2002. These laws require that
the survey sponsor treat all information you provide as confidential. The information you provide
will be used only for research and statistical purposes by the survey sponsor, their contractors,
and collaborating researchers for the purpose of analyzing data and preparing scientific reports
and articles. Any information publicly released (such as statistical summaries) will be in a form
that does not personally identify you. Your response is voluntary. According to the Paperwork
Reduction Act of 1995, no persons are required to respond to a collection of information unless
such collection displays a valid OMB number. The OMB control number for this survey is 18500813. The time required to complete this survey is estimated to average 30 minutes per
response. If you have any comments concerning the accuracy of the time estimate or
suggestions for improving this form, please write to: U.S. Department of Education, Washington,
DC 20202-4651. If you have comments or concerns about the content of this questionnaire,
contact Sheila Heaviside by e-mail at: [email protected] or at 1-866-869-3187.
OMB NO.: 1850-0813
EXPIRATION DATE: ##/##/200#

TRAINING, RESOURCES, AND SUPPORT FOR TEACHING MATH
F1a. Did you participate in training on how to use the assigned curriculum prior to the start of the
school year?
! Yes
! No ! SKIP to Question 3
F1b. How many days prior to the start of the school year did you participate in training on how to
use the assigned curriculum? Mark (X) only one box.
! Less than a day
! 1day
! 2 days
! 3 days
! 4 days or more
F2.

Overall, how well did the training and/or support you received from the publisher prepare
you to use the assigned curriculum with your students? Mark (X) only one box.
! Very Well
! Somewhat
! Not at all

F3.

To what extent are the following materials from the assigned curriculum available for your
use in teaching math? Mark (X) one box for each row.

Not at all

Materials are
shared with
other teachers
at my school

Materials are
dedicated for
use with my
students

a. Teaching guide or teacher’s manual

b. Student workbooks or textbooks

c. Manipulatives

d. Supplemental materials recommended by the publisher

EMC Teacher Survey – FALL Items

F4.

How well prepared are you to do the following during math instruction?
Mark (X) one box for each row.
Not
prepared

Somewhat Adequately
prepared
prepared

Very well
prepared

Demonstrate mathematical procedures and
concepts to students.

Respond to students’ mathematical errors.

Ask students to explore a concept or procedure
prior to it first being modeled.

Ask students to demonstrate a procedure or
explain a concept to other students.

Teach a class in which students use
manipulatives.

Teach a class in which small groups of students
work on collaborative activities.

Differentiate instruction for individual students or
small groups.

Allow students to practice math facts using
manipulatives, pictures, or diagrams.

F5.

In this item, we are interested in the types of discussions you are prepared to facilitate during
math instruction. Please indicate how well prepared you are to do the following.
Mark (X) one box for each row.
Not
prepared

Somewhat Adequately
prepared
prepared

Very well
prepared

Facilitate discussions that allow students to
explain their answers.

Facilitate discussions that enable students to
offer or share multiple approaches to solving a
problem.

Facilitate discussions that enable students to
raise mathematical questions and/or discuss
mathematical concepts.

Facilitate discussions that enable students to
reference other students’ ideas in their
comments.

EMC Teacher Survey – FALL Items

F6.

During the 12 months prior to the start of this school year, have you participated in any
professional development activities on the following math topics that were NOT specific to
the assigned curriculum (COLUMN A)? If yes, how many hours did you spend on these
activities (COLUMN B)? Include courses you have taken for recertification or advanced
certification, workshops sponsored by your school or district, conferences, or other training
that is relevant to your teaching of math.
For each row, mark (X) one box in Column A. If you answer “Yes,” mark (X) one box in
Column B for that row.
COLUMN A:
Participation

COLUMN B: Number of hours of participation
8 or less

9-16

17-32

33-40

More
than 40

Mathematics instruction

Yes !

No !

Content and performance standards
Yes !
in mathematics education

No !

F7.

During the 12 months prior to the start of this school year, what have been the sources of your
professional development in math? Mark (X) all that apply.
! Workshops or training provided by your school or district
! Coursework taken towards a credential for teaching
! University coursework in math or math instruction, not including coursework for a credential
! Activities such as conferences or working groups about math
! Other ! Please specify: __________________________________________________
! Did not participate in professional development in math

F8.

(Item was deleted. Subsequent item numbers and skip patterns will be revised accordingly.)

MATH INSTRUCTION IN YOUR TARGET CLASS
F9.

How many students are currently enrolled in your class that uses the assigned curriculum
(hereafter, referred to as your “target” class)?

!! students
F10. Approximately how many students in your target class are:
High math achievers

Average math achievers

Low math achievers

EMC Teacher Survey – FALL Items

F11. Approximately how many students in your target class are:
English proficient

Limited English proficient

F12. Approximately how many students in your target class have an Individualized Education Plan
(IEP)?
Students with an IEP

F13. On average, how many minutes per week do you spend preparing to teach math to your
target class using the assigned curriculum (including lesson planning, grading student work,
etc.)?

!!! minutes per week
F14. On average how many days per week do you teach math to your target class?

! days per week
F15. For approximately how many minutes each day do you teach math to your target class?

!!! minutes per day
F16. Are you using the assigned curriculum as your core math curriculum?
! Yes
! No ! Please specify the name/publisher of your core curriculum:___________________
F17a. Do you use other math curricula in addition to the assigned curriculum?
! No !

SKIP to Question 18a

! Yes ! Please specify the name(s) and publisher(s) of the curricula:
_______________________________________________________________________________
_______________________________________________________________________________
F17b. For what purpose(s) are the curricula specified in Question 17a being used?
Mark (X) all that apply.
! Remediation with a small group of students
! Remediation with the entire class
! Enrichment with a small group of students
! Enrichment with the entire class
! As a replacement for selected units or lessons in the assigned curriculum
! As a supplement to units or lessons in the assigned curriculum
! Other ! Please specify: __________________________________________________
EMC Teacher Survey – FALL Items

F18a. Have you ever used the assigned curriculum in a primary grade (K-3) prior to this school
year?
! Yes
! No
F18b. If you taught math in a primary grade (K-3) last year (in 2005-06), please indicate the math
curriculum you used. Mark (X) only one box.
! I did NOT teach math in a primary grade (K-3) last year. ! SKIP to Question 19
! I used the following math curriculum last year (please specify the name and publisher of
the curriculum): ______________________________________________________________________
F18c. Please indicate the total number of years that you used the curriculum specified in Question
18a.

!! years

APPROACHES TO TEACHING
F19. Please indicate how strongly you agree or disagree with the following general statements
about teaching math. Mark (X) one box for each row.
Strongly
disagree

Disagree

Agree

Strongly
Agree

Whenever students ask how to solve a math
problem, teachers should provide a thorough
explanation.

Formative and multiple assessments are an
important means of documenting students’ learning.

It is important that students not have mathematical
misconceptions at the end of class.

Students learn from one another when they work
together on math problems.

Students should demonstrate mastery of a particular
math concept before proceeding to the next
concept.

Students learn math best when they share their
reasoning about a math problem with other
students.

It is important that teachers observe and listen to
how students think about math.

A math concept is best learned if a concrete
example is presented when it is first introduced.

EMC Teacher Survey – FALL Items

Teachers should emphasize computational skills.

Teachers should clearly model to students how to
solve a type of problem they have never seen
before.

Students should primarily work individually in math to
ensure that they master skills and are able to work on
their own.

YOUR SCHOOL’S INSTRUCTIONAL CLIMATE
F20a. Is there a math coach or district specialist to assist you in teaching math?
Mark (X) only one box.
! Yes
! No ! SKIP to Question 21
! Don’t know ! SKIP to Question 21
F20b. How accessible to you is a math coach or district specialist? Mark (X) only one box.
! Not at all
! Rarely
! Sometimes
! Almost always
! Don’t know
F20c. Is the math coach or district specialist knowledgeable about the assigned curriculum you
are using? Mark (X) only one box.
! Yes
! No
! Don’t know
F21. Is there another adult (such as an aide, assistant, volunteer) who assists you when you are
teaching math to your target class?
! Yes
! No

EMC Teacher Survey – FALL Items

F22. This question concerns how teachers interact in your school. Please indicate about how
many teachers in your school do each of the following. Mark (X) one box for each row.
No
teachers

Some
teachers

Most
teachers

All
teachers

Don’t
know

Work together to develop curriculum
and instructional materials

Observe each other teaching

Offer advice or help to each other

Share ideas on teaching

Promote new or innovative teaching
practices

F23. Consider the conditions for teaching math in your school. How well does each of the
following describe conditions in your school? Mark (X) one box for each row.
Strongly
disagree

Disagree

Agree

Strongly
Agree

I feel supported by other teachers to try out new
ideas in teaching math.

Administrators at this school promote innovations
in math education.

Teachers in this school regularly share ideas
about math instruction.

There is a lot of disagreement among teachers
about how to teach math.

I regularly work with other teacher(s) on math
curriculum and instruction.

A specialist in math education regularly works
with teachers in this school.

Most curriculum changes introduced at this
school gain little support among teachers.

YOUR BACKGROUND
F24. Including the 2006-07 academic year, how many years have you taught full-time in a
regular teaching position?
Total years

Years in primary grades (K-3)

Years at your present school

EMC Teacher Survey – FALL Items

F25a. Do you have a bachelor’s degree?
! Yes
! No ! SKIP to Question 31a
F25b. In what year did you receive your bachelor’s degree?

!!!!

F25c. What was your major field of study? Record the major code and name from Table 1.
Code

!!!

Major field _____________________________________________

F26a. Did you have a second major field of study?
! Yes
! No ! SKIP to Question 27a
F26b. What was your second major field of study? Record the major code and name from Table 1.
Code

!!!

Major field _____________________________________________

F27a. Do you have a master’s degree?
! Yes
! No ! SKIP to Question 28a
F27b. In what year did you receive your master’s degree?

!!!!

F27c. What was your major field of study? Record the major code and name from Table 1.
Code

!!!

Major field _____________________________________________

EMC Teacher Survey – FALL Items

Table 1
Major Field of Study Codes
Elementary Education
101 Early childhood/Pre-K, general
102 Elementary grades, general
Secondary Education
103 Middle grades, general
104 Secondary grades, general
Special Education
104 Special education, any
Other Education
131 Administration
132 Counseling and guidance
133 Educational psychology
134 Policy studies
135 School psychology
136 Other non-subject matter specific education
Arts & Music
141 Visual art
142 Dance
143 Drama/Theater
144 Music
English and Language Arts
151 Communications
152 Composition
153 English
154 Journalism
155 Language arts
156 Reading
157 Speech
English as a Second Language
160 ESL/Bilingual education: General
161 ESL/Bilingual education: Spanish
162 ESL/Bilingual education: Other languages
Foreign Languages
171 French
172 German
173 Latin
174 Spanish
175 Other foreign language
Health Education
181 Health education
182 Physical education
Mathematics and Computer Science
190 Mathematics
191 Computer Science

Natural Sciences
211 Biology/Life sciences
212 Chemistry
213 Earth sciences
214 Engineering
215 Physics
216 Other
Social Sciences
221 Anthropology
222 Area/Ethnic studies (excluding Native American studies)
223 Criminal justice
224 Cultural studies
225 Economics
226 Geography
227 Government/Civics
228 History
229 International studies
230 Law
231 Native American studies
232 Political science
233 Psychology
234 Sociology
235 Other social science
Vocational/Technical Education
241 Agriculture and natural resources
242 Business/Office
243 Keyboarding
244 Marketing and distribution
245 Health occupations
246 Construction trades
247 Mechanics and repair
248 Drafting/Graphics/Printing
249 Metals/Woods/Plastics, and other precision production
(electronics, leatherwork, meatcutting, etc.)
250 Communications and other technologies (not including
computer science)
251 Culinary arts/Hospitality
252 Child care and education
253 Personal and other services (including cosmetology,
custodial services, clothing and textiles, and interior
design)
254 Family and consumer sciences education
255 Industrial arts/Technology education
256 Other vocational/Technical education
Miscellaneous
261 Architecture
262 Humanities/Liberal studies
263 Library/Information science
264 Military science/ROTC
265 Philosophy
266 Religious studies/Theology/Divinity
Other
268 Other

EMC Teacher Survey – FALL Items

F28a. Have you earned any of the degrees listed below in 28b?
! Yes
! No ! SKIP to Question 29
F28b. Please indicate your major field(s) of study and the year in which your degree was received:

Degree

Code for major
field of study

Major field of study

Year
received

a. Second bachelor’s degree

Code

!!!

_____________________

!!!!

b. Second master’s degree

Code

!!!

_____________________

!!!!

c. Educational specialist or
professional diploma (at least one
year beyond a master’s degree)

Code

!!!

_____________________

!!!!

d. Certificate of Advanced
Graduate Studies

Code

!!!

_____________________

!!!!

e. Doctorate (Ph.D. or Ed.D.)

Code

!!!

_____________________

!!!!

F29. As part of either your undergraduate or graduate coursework, how many advanced math
courses did you take (such as trigonometry, calculus, or statistics)? Mark (X) only one box.
! None
! 1 or 2 courses
! 3 or 4 courses
! 5 or more courses
F30. As part of either your undergraduate or graduate coursework, how many math education
courses did you take? Mark (X) only one box.
! None
! 1 or 2 courses
! 3 or 4 courses
! 5 or more courses

EMC Teacher Survey – FALL Items

F31a. Which of the following describes the teaching certificate you currently hold in this state?
Mark (X) only one box.
! Regular or standard state certificate or advanced professional certificate
! Probationary certificate (the initial certificate issued after satisfying all requirements
except the completion of a probationary period)
! Provisional or other type given to persons who are still participating in an “alternative
certification program”
! Temporary certificate (requires some additional college coursework and/or student
teaching before regular certification can be obtained)
! Emergency certificate or waiver (issued to teachers who do not have regular
certification who need to complete a regular certification program in order to continue
teaching)
! I do not have any of the above certifications in THIS state ! SKIP to Question 32
F31b. In what content area does the teaching certificate specified in Question 31a allow you to
teach in this state? For some teachers, the content area may be the grade level (e.g.,
elementary general, secondary general, etc.).
Record the code and content area from Table 2.
Code

!!!

Content Area

_____________________________________________

F31c. To which of the following grade ranges does the teaching certificate specified in Question
31a apply? Mark (X) all that apply.
! Elementary grades (including early childhood, preschool and kindergarten)
! Secondary grades (including middle school)
! Ungraded
F32. Please indicate whether you are male or female.
! Male
! Female
F33. Are you Hispanic or Latino?
! Yes
! No

EMC Teacher Survey – FALL Items

Table 2
Certification Content Area Codes
Elementary Education
101 Early childhood/Pre-K, general
102 Elementary grades, general

Mathematics and Computer Science
190 Mathematics
197 Computer Science

Secondary Education
103 Middle grades, general
104 Secondary grades, general

Natural Sciences
210 Science, general
211 Biology/Life sciences
212 Chemistry
213 Earth sciences
216 Physical science
217 Physics
218 Other natural sciences

Special Education
111 Special education, general
112 Autism
113 Deaf and hard-of-hearing
114 Developmentally delayed
115 Early childhood special education
116 Emotionally disturbed or behavior disorders
117 Learning disabilities
118 Mentally retarded
119 Mildly/Moderately disabled
120 Orthopedically impaired
121 Severely/Profoundly disabled
122 Speech/Language impaired
123 Traumatically brain-injured
124 Visually impaired
125 Other special education
Arts & Music
141 Art/Arts or crafts
143 Dance
144 Drama/Theater
145 Music
English and Language Arts
151 Communications
152 Composition
153 English
154 Journalism
155 Language arts
156 Reading
157 Speech
English as a Second Language
160 ESL/Bilingual education: General
161 ESL/Bilingual education: Spanish
162 ESL/Bilingual education: Other languages
Foreign Languages
171 French
172 German
173 Latin
174 Spanish
175 Other foreign language
Health Education
181 Health education
182 Physical education

EMC Teacher Survey – FALL Items

Social Sciences
220 Social studies, general
221 Anthropology
225 Economics
226 Geography
227 Government/Civics
228 History
231 Native American studies
233 Psychology
234 Sociology
235 Other social sciences
Vocational/Technical Education
241 Agriculture and natural resources
242 Business/Office
243 Keyboarding
244 Marketing and distribution
245 Health occupations
246 Construction trades
247 Mechanics and repair
248 Drafting/Graphics/Printing
249 Metals/Woods/Plastics, and other precision production
(electronics, leatherwork, meatcutting, etc.)
250 Communications and other technologies (not including
computer science)
251 Culinary arts/Hospitality
252 Child care and education
253 Personal and other services (including cosmetology,
custodial services, clothing and textiles, and interior design)
254 Family and consumer sciences education
255 Industrial arts/Technology education
256 Other vocational/Technical education
Miscellaneous
262 Driver education
263 Humanities/Liberal studies
264 Library/Information science
265 Military science/ROTC
266 Philosophy
267 Religious studies/Theology/Divinity
Other
268 Other

F34. What is your racial background? Mark (X) one or more.
! American Indian or Alaska Native
! Native Hawaiian or other Pacific Islander
! Asian
! Black or African American
! White
F35. In what year were you born?

EMC Teacher Survey – FALL Items

!!!!

APPENDIX B
Letter to Teachers and Spring (Implementation) Survey

MATHEMATICA
Policy Research, Inc.
P.O. Box 2393
Princeton, NJ 08543-2393
Telephone (609) 799-3535
Fax (609) 799-0005
www.mathematica-mpr.com

Dear Teacher,
You and your school are participating in the Evaluation of Mathematics Curricula. Mathematica
Policy Research, Inc. (MPR), along with RG Research Group and SRI International, is conducting the
study for the U.S. Department of Education’s Institute of Education Sciences. As part of the study, we
are gathering information about your experiences with the curricula you are piloting now that you
have had several months of experience with the instructional materials.
Please complete the enclosed survey. This survey includes items that update information collected in
the fall survey. All of the information that you provide will be kept confidential and used for research
purposes only. No one at your school will see your responses, and you will not be identified in any
report or presentation. While your participation in this study is voluntary, it is very important that you
complete the questionnaire. As one of only a few hundred teachers in the study this year, every
response is critical in understanding teachers’ classroom experiences. We have enclosed a check for
you in appreciation for your participation.
If you have any questions, please contact the study’s survey director Sheila Heaviside at 1-866-8693187, or by email at [email protected].
Thank you for your help with this important study.
Sincerely,

Roberto Agodini
Study Director

TEACHER SURVEY (Spring 2007)
NATIONAL EVALUATION OF ELEMENTARY MATHEMATICS CURRICULA
U.S. DEPARTMENT OF EDUCATION

ATTACH LABEL HERE
Teacher ID Teacher Name
School ID School Name
IF ABOVE INFORMATION IS INCORRECT,
PLEASE MAKE CORRECTIONS DIRECTLY ON LABEL.

Please return the completed form to:

If you have questions, please contact:

This survey is authorized by the U.S. Department of Education (P. L. 20 U.S.C. 1221e.1) and the
Confidential Information Protection and Statistical Efficiency Act of 2002. These laws require that
the survey sponsor treat all information you provide as confidential. The information you provide
will be used only for research and statistical purposes by the survey sponsor, their contractors,
and collaborating researchers for the purpose of analyzing data and preparing scientific reports
and articles. Any information publicly released (such as statistical summaries) will be in a form
that does not personally identify you. Your response is voluntary. According to the Paperwork
Reduction Act of 1995, no persons are required to respond to a collection of information unless
such collection displays a valid OMB number. The OMB control number for this survey is 18500813. The time required to complete this survey is estimated to average 20 minutes per
response. If you have any comments concerning the accuracy of the time estimate or
suggestions for improving this form, please write to: U.S. Department of Education, Washington,
DC 20202-4651. If you have comments or concerns about the content of this questionnaire,
contact Sheila Heaviside by e-mail at: [email protected] or at 1-866-869-3187.
OMB NO.: 1850-0813
EXPIRATION DATE: ##/##/200#

TRAINING, RESOURCES, AND SUPPORT FOR TEACHING MATH
S1a. Since the start of the school year, has any follow-up training or on-site support from the
publisher of the assigned curriculum been available to assist you in teaching math?
! Yes
! No ! SKIP to Question 2a
! Don’t know ! SKIP to Question 2a
S1b. Since the start of the school year, have you participated in follow-up training or on-site
support from the publisher of the assigned curriculum?
! Yes
! No ! SKIP to Question 2a
S1c. Since the start of the school year, how many hours have you spent participating in follow-up
training or on-site support from the publisher of the assigned curriculum?

!!! hours
S2a. Are the following types of support from the publisher of the assigned curriculum available to
assist you in teaching math? Mark (X) one box for each row.
Don’t
Know

Yes

a. Technical phone support

b. Technical online support

c. CD or DVD-based or printed reference materials

S2b. How often have you used each type of support from the publisher of the assigned
curriculum? Mark (X) one box for each row.
Never

Less than
once a Once a 2-3 times Weekly
month
month a month or more

Technical phone support

Technical online support

CD or DVD-based or printed reference materials

EMC Teacher Survey – SPRING Items

S3.

Since the start of the school year, have you participated in any professional development
activities on the following math topics that were NOT specific to the assigned curriculum
(COLUMN A)? If yes, how many hours did you spend on these activities (COLUMN B)?
Include courses you have taken for recertification or advanced certification, workshops
sponsored by your school or district, conferences, or other training that is relevant to the
teaching of math.
For each row, mark (X) one box in Column A. If you answer “Yes,” mark (X) one box in
Column B for that row.
COLUMN A:
Participation

COLUMN B: Number of hours of participation
8 or less

9-16

17-32

33-40

More
than 40

Mathematics instruction

Yes !

No !

Content and performance standards
Yes !
in mathematics education

No !

S4.

Since the start of the school year, what have been the sources of your professional
development in math? Mark (X) all that apply.
! Workshops or training provided by your school or district
! Coursework taken towards a credential for teaching
! University coursework in math or math instruction, not including coursework for a credential
! Activities such as conferences or working groups about math
! Other ! Please specify: __________________________________________________
! Did not participate in professional development in math

S5.

Are you currently participating in any professional or staff development activities focused on
math instruction or content and performance standards in mathematics education that are
NOT specific to the assigned curriculum?
! Yes
! No

MATH INSTRUCTION IN YOUR TARGET CLASS
S6a. How many students are currently enrolled in your class that uses the assigned curriculum
(hereafter, referred to as your “target” class)?

!! students

EMC Teacher Survey – SPRING Items

S6b. Approximately how many students in your target class are:
High math achievers

Average math achievers

Low math achievers

S6c. Approximately how many students in your target class are:
English proficient

Limited English proficient

S6d. Approximately how many students in your target class have an Individualized Education Plan
(IEP)?
Students with an IEP

S7a. On average, how many minutes per week do you spend preparing to teach math to your
target class using the assigned curriculum (including lesson planning, grading student work,
etc.)?

!!! minutes per week
S7b. On average how many days per week do you teach math to your class that uses the
assigned curriculum (hereafter, referred to as your “target” class)?

! days per week
S7c. For approximately how many minutes each day do you teach math to your target class?

!!! minutes per day
S8.

Are you using the assigned curriculum as your core math curriculum?
! Yes
! No ! Please specify the name/publisher of your core curriculum: _______________________

S9.

Which of the following topics have you covered with your target class using the assigned
curriculum? Mark (X) all that apply.

! Counting, with whole numbers
! Place value, with whole numbers
! Adding and addition facts, with whole numbers
! Subtracting and subtraction facts, with whole numbers
! Multiplying and multiplication facts, with whole numbers
! Dividing and division facts, with whole numbers
! Fractions
EMC Teacher Survey – SPRING Items

! Decimals
! Percents
! Geometric shapes
! Understanding or predicting with patterns
! Collecting or analyzing data
! Graphs
! Probability
! Measurement using rulers
! Other ways of measuring
! Time
! Money
S10a. Approximately what percentage of all the lessons from the assigned curriculum have you
used so far this year with your target class? Mark (X) only one box.
! Less than 20%
! 20-49%
! 50-79%
! 80-100%
S10b.

Please indicate today’s date (DD/MM/YYYY):

!!/!!/!!!!

S11. How often do you use materials from the assigned curriculum with your target class?
Mark (X) one box for each row.
Never

Less than Once or
once a
twice a
month
month

Once a
week

2-4
times a
week

Daily

a. Student worksheets

b. Student workbooks or textbooks

c. Student assessments

d. Manipulatives

e. Lesson plans or lesson scripts

Recommended supplemental materials

EMC Teacher Survey – SPRING Items

S12. Please indicate the extent to which you agree or disagree with the following statements
about the assigned curriculum. Mark (X) one box for each row.
Strongly
disagree

Disagree

Agree

Strongly
Agree

I have had adequate opportunities to learn about
the curriculum.

I can explain to other teachers how to use the
curriculum.

The curriculum corresponds well with the math
understandings I want my students to demonstrate.

The curriculum is aligned well with our state
curriculum standards.

The curriculum conflicts with my preferred approach
to math instruction.

The curriculum assumes major changes in the way I
teach math.

The curriculum has prompted me to change some
of my teaching practices in math.

The curriculum is more trouble than it is worth.

All teachers in my school are committed to the
implementation of the curriculum.

Administrators at my school are committed to the
implementation of the curriculum.

I believe my students will score better on required
accountability tests because of their experience
with the curriculum.

S13. About how often do students in your target class take part in the following activities?
(Include only activities that take place in the class during math instruction.)
Mark (X) one box for each row.
How often do students…

Never

Less than
once a
month

Once or
twice a
month

Once or
twice a
week

Almost
daily

Work in small groups or with a partner.

Practice or take tests on computational skills.

Work individually on math problems from the
textbook/workbook.

Work on investigations or problems that
extend for several days.

Write about how to solve a problem.

EMC Teacher Survey – SPRING Items

Do problems that have more than one
correct solution.

Discuss different ways of solving a problem.

Reference other students’ ideas in their
classroom contributions.

Explain a math concept or procedure to the
other students.

Ask mathematical questions of other students

Use manipulatives, pictures, or diagrams to
solve problems.

Use manipulatives, pictures, or diagrams to
support explanations.

m. Work on activities that integrate math with
other subjects.

Write in math journals (e.g., explain their
mathematical reasoning or create their own
math problems).

Take part in activities designed to develop
rapid recall of math facts.

Practice math facts using manipulatives,
pictures, or diagrams for support.

S14. How often do you do the following with your target class? Mark (X) one box for each row.
Three
times or
more a
day

Once or
twice a
day

Once or
twice a
week

Once or
twice a
month

Less
than
once a
month

Never

a. Prompt students to explain their
answers.

b. Invite students to use multiple
strategies or solutions to a problem.

c. Ask students to demonstrate a
procedure or concept to other
students.

d. Ask students to explore a concept or
procedure before it is modeled.

e. Differentiate math instruction for
students at different ability levels.

EMC Teacher Survey – SPRING Items

S15a. What strategies do you use to respond to students’ errors during math instruction?
Mark (X) all that apply.
! Correct the student’s mistake as soon as possible.
! Ask the student questions that guide him/her to the correct answer.
! Ask another student for the correct answer.
! Use the incorrect response as a basis for an exercise or class discussion about the
misconception.
! Re-teach the procedure and/or concept.
! Other ! Please specify: __________________________________________________
S15b. Of the strategies selected in Question 15a (above), which one do you use the most often?
Mark (X) only one box.
! Correct the student’s mistake as soon as possible.
! Ask the student questions that guide him/her to the correct answer.
! Ask another student for the correct answer.
! Use the incorrect response as a basis for an exercise or class discussion about the
misconception.
! Re-teach the procedure and/or concept.
! Other ! Please specify: __________________________________________________
S16. What percent of math instructional time do students in your target class spend practicing
math procedures and the recall of math facts?

!! % of math instructional time
S17a. Do you use other math curricula in addition to the assigned curriculum?
! No !

SKIP to Question 18

! Yes ! Please specify the name(s) and publisher(s) of the curricula:
_______________________________________________________________________________
_______________________________________________________________________________
S17b. For what purpose(s) are the curricula specified in Question 17a being used?
Mark (X) all that apply.
! Remediation with a small group of students
! Remediation with the entire class
! Enrichment with a small group of students
! Enrichment with the entire class
! As a replacement for selected units or lessons in the assigned curriculum
EMC Teacher Survey – SPRING Items

! As a supplement to units or lessons in the assigned curriculum
! Other ! Please specify: __________________________________________________
S17c. How often do you supplement the assigned curriculum with materials or math problems from
other sources? Mark (X) only one box.
! Almost daily
! Once or twice a week
! Once or twice a month
! Less than once a month
! Never
S18. If given your choice of math curriculum to use next year, how likely are you to choose the
assigned curriculum over other curricula available in your district? Mark (X) only one box.
! Very likely
! Likely
! Not at all likely

CURRICULUM IMPLEMENTATION (example item)
NOTE: This type of item will be included as a separate insert with the Spring survey, and will be
curriculum-specific (i.e., each teacher will receive an insert that has items specific to the
curriculum to which they are assigned). The following item is an illustrative example for one of
the curricula, Scott-Foresman/Addison Wesley, and is based on critical activities in the
implementation of this curriculum.
S19. How often do you do the following with your target class? Mark (X) one box for each row.

Never

Less than
once a
month

Once or
twice a
month

Once
or twice
a week

Almost
daily

Use Spiral Review

Use Investigating the Concept activity

Use manipulatives during the
Investigating the Concept activity when
appropriate

Groups students into small groups for
collaborative activities (e.g., Reaching
All Learners)

Identify the important math concept or
key idea before the lesson begins

Use Warm Up activity

EMC Teacher Survey – SPRING Items

Use Focus Questions from the curriculum
materials

Use Check activity

Provide step-by-step instructions and
guidance to students on how to
complete the practice page

Provide practice opportunities for
students at varying levels (below, on
level, above)

Provide reading assistance to students as
they complete the practice page

Use Instant Check Mat

m. Provide opportunities for students to use
online materials
n.

Provide additional activities for “early
finishers”

Conduct closure activity (e.g., journal,
“curriculum connection,” test taking
page, etc.)

EMC Teacher Survey – SPRING Items

APPENDIX C
Classroom Observation Protocol for Math Expressions Curriculum

1
Math Expressions Fidelity of Implementation Observation Measure

Before the Main Lesson
1. Did the class complete all parts of the daily Routine?

2. Did the class complete the Quick Practice activity?
If yes, was at least one aspect lead by a student leader.

N
N

Y
Y

Teaching the Lesson
3. Teacher conducts activities in the lesson.
1 = None
2 = Some
3 = All
4. Teacher uses the materials required for the lesson.
1 = None
2 = Some
3 = All
5. Math Talk Learning Community: Students ask questions and make comments relating to
math.
1 = No Dialogue about math
2 = Multiple students participate but there is no student to student interaction
3 = Evidence of student-to-student interactions (e.g., students add to, comment on,
correct etc. previously made comments, student ask questions of another student, etc.)
Consider denoting the degree to which math talk is occurring for each 15-minute segment of
instruction. (for whole class Math talk)
6. Multiple Approaches to a Problem:
1 = No or minimal elicitation of student problem solving strategies
2 = Two student problem solving strategies are elicited
3 = More than two student problem solving strategies are elicited
7. Explaining math thinking:
1 = No or minimal teacher elicitation of student thinking strategies, or explanations,
Teacher expects answer-focused responses. Teacher may give the answer
2 = Teacher probes students’ thinking somewhat
3 = Teacher probes more deeply to learn about student thinking

8. Teacher conducts On Going Assessment activities?

9. Teacher administers Quick Quiz

10. Homework is regularly assigned.

N/A

11. Error Correction:
1 = Teacher ignores incorrect response
2 = Teacher provides the correct answer
3 = Teacher’s first response is to ask another student
4 = Teacher asks the student guiding questions.
12. How frequently does the teacher use visual or concrete representation of mathematical
concepts:
1 = None
2 = Fewer than recommended in the Teacher’s Guide
3 = The same as or more than recommended in the Teacher’s Guide
Consider denoting the degree to which visuals or concrete representation of mathematical
representations are used for each 15-minute segment of instruction.

Extending the Lesson
13. The teacher implements activities designed to differentiate instruction.

APPENDIX D
School District Recruiting Materials

Superintendent First & Last Name
Superintendent
District Name
Street Address
City, State Zip
Dear :
The Institute of Education Sciences (IES) in the U.S. Department of Education is sponsoring a
study of early elementary math curricula, with the goal of identifying programs that are more
effective at improving student achievement. Currently, there is little information educators can use
to help them choose a math curriculum that will work best at improving student achievement in their
schools. Identifying strategies that improve math achievement is particularly important for
elementary school students because a large proportion of them show mastery of only rudimentary
mathematics.
The study is being conducted for the Department by Mathematica Policy Research, Inc., a
nationally-recognized research institute that conducts research on key public policy issues. This
large-scale, national study will select, implement, and compare the effectiveness of several early
elementary math curricula that use varying approaches to develop student math skills.
The study provides districts with the opportunity to use several early elementary math curricula
at no cost for up to two years. This includes free teacher training and follow-up support on the
curricula, and free curriculum materials.
I strongly encourage you to read the attached letter that provides more details about the study,
and hope you will consider participating. Mathematica will be contacting you shortly to discuss
your district’s interest in, and suitability for, participating in this important new study.
In the meantime, if you have questions about the study, please contact the study’s director, Dr.
Roberto Agodini from Mathematica, at 609-936-2712 or [email protected]. You
also are welcome to contact Audrey Pendleton, the IES project officer, at 202-208-7078 or
[email protected].
Sincerely,

Roberto Agodini
Study Director

P.O. Box 2393
Princeton, NJ 08543-2393
Telephone (609) 799-3535
Fax (609) 799-0005
www.mathematica-mpr.com

Superintendent First & Last Name
Superintendent
District Name
Street Address
City, State Zip
Dear Superintendent:
As part of a study sponsored by the Institute of Education Sciences (IES) in the U.S.
Department of Education, I am contacting you about an opportunity to implement (at no cost)
several early elementary math curricula in the first grade during the 2006-2007 school year. The
curricula use varying approaches to develop student math skills. The study will include about 20
districts and 100 schools national wide. The goal is to implement the curricula in a geographically
diverse group of districts with Title I eligible schools. The study plan includes extending the
implementation of the curricula in the second grade during the 2007-2008 school year. My
organization—Mathematica Policy Research, a non-partisan research institute—was selected by IES
to conduct this large-scale, national study.
The study includes several benefits:
• Free teacher training and follow-up support on the curricula
• Free curriculum materials for teachers and students
• Research evidence based on your own district.
Participating districts will be asked to nominate schools for the study in spring 2006, and
researchers will use a lottery to randomly assign the study’s curricula to the schools. The study will
collect data from first grade classrooms in each school during the 2006-07 school year. In the
selected classrooms, the study will administer math tests to students at the beginning and end of the
school year, and study team members will visit classrooms to observe curricula implementation.
I or someone from the study team will be contacting you soon to discuss your interest in
participating. We need to move quickly because the curricula will be implemented this fall, so I
hope we can connect soon.

An Affirmative Action/Equal Opportunity Employer

LETTER TO:
FROM:
DATE:
PAGE:

Superintendent First & Last Name
Roberto Agodini

In the meantime, if you have any questions, please feel free to contact me at (609) 936-2712 or
[email protected]. If you would like more information about the study before we
have a chance to talk, please browse the study’s website www.mathcurriculastudy.com.
We hope that you will consider participating in this important study, and we look forward to
speaking with you soon.

Sincerely,

You are invited
to participate in an
important new study of

How can I get more
information?
Please contact Mathematica Policy Research
toll-free at (866) 869-3187 and mention the
math curricula study, or visit the study website
at www.mathcurriculastudy.com.

early elementary math
Who is conducting the study?

curricula, sponsored
by the U.S. Department
of Education.

The study is sponsored by the U.S. Department
of Education’s Institute for Education Sciences.
Three highly regarded independent research
firms, Mathematica Policy Research, RG
Research Group, and SRI International, are
conducting the study.

7-2=?
P.O. Box 2393
Princeton, NJ 08543-2393
(609) 799-3535
(609) 799-0005 (fax)
www.mathematica-mpr.com

Which math
curriculum
should I use?

3+3=6
What curricula will be included
in the study?
The study will include several curricula that
use varying approaches to develop students’
math skills. A group of experts in math
and math instruction worked with the U.S.
Department of Education to select the
curricula included in the study.

What are the benefits
of participating?
The study is an important opportunity for
districts to try several early elementary math
curricula at no cost and to obtain research
evidence based on their own district. Teachers
will receive free training and support on the
curricula.

How will the study work?
The study includes two years of implementation
and data collection during the 2006-2007 and
2007-2008 school years. Participating districts
will nominate schools for the study in spring
2006, and researchers will use a lottery to
randomly assign the curricula to participating
schools. The study will collect data from
only a few classrooms in each school. In
selected classrooms, the study will administer
math tests to students at the beginning
and end of the school year, and study team
members will visit classrooms to observe
curricula implementation.

How many districts and
schools will participate?
The study will include about 20 districts and
100 schools nationwide. The goal is to implement the curricula in a geographically diverse
group of districts with Title I eligible schools.

Will information be
kept private?
All information collected by the study is
strictly confidential and carefully guarded to
the full extent allowed by law. The study’s
results will be reported only for groups of
students, such as “70 percent of first graders
achieved mastery of rudimentary mathematics.”

2+2=4

ABSTRACTS OF CURRICULA INCLUDED IN THE STUDY
Investigations in Number, Data, and Space
Investigations in Number, Data, and Space® is a K-5 mathematics curriculum developed by TERC under a grant
from the National Science Foundation. Its four major goals are:
• to offer students meaningful mathematical problems
• to emphasize depth in mathematical thinking rather than superficial exposure to a series of fragmented
topics
• to communicate mathematics content and pedagogy to teachers
• to substantially expand the pool of mathematically literate students
The Investigations curriculum offers in-depth experiences in number, data, geometry, and the mathematics of
change. The following aspects of the curriculum ensure that all students are included in significant mathematical
learning by:
• Spending time exploring problems in depth
• Finding more than one solution to many problems
• Developing their own strategies and approaches, based on their knowledge and understanding of
mathematical relationships
• Choosing from a variety of concrete materials and appropriate technology, including calculators, as a
natural part of their everyday mathematical work
• Expressing their mathematical thinking through drawing, writing, and talking
Each grade level is organized into units that involve students in the exploration of major mathematical ideas, and
may revolve around two or three related areas-for example, addition and subtraction or geometry and fractions.
The curriculum is presented through a series of teacher books. Each book provides lesson plans, materials lists,
reproducible student sheets for activities and games, a family letter, homework suggestions, opportunities for skill and
practice, assessment activities, notes to the teacher about the mathematics students are encountering, and examples of
classroom dialogues. Some units include software to extend students' experience with the mathematics being explored.
In addition to the curriculum units, Student Activity Books, and Investigations at Home Booklets, and End of Unit
Assessment Sourcebooks are also available for each unit in grades 1-5.

Math Expressions
Math Expressions is a complete Kindergarten through Grade 5 curriculum based on the research results of the
Children’s Math Worlds (CMW) project. The CMW project was conducted by Dr. Karen C. Fuson, now professor
emerita of learning sciences at Northwestern University, Evanston, Illinois, and funded over a ten-year period by the
National Science Foundation. Both the program and the research combine a focus on conceptual understanding with
opportunities to develop fluency with problem solving and computation. Math Expressions incorporates approaches
from both reform and traditional mathematics programs while contributing new and effective teaching strategies to
mathematics instruction. Key aspects of this curriculum include application of accessible algorithms that can be more
easily understood and used by students; use of student math drawings and research-based visual representations to
support student understanding and class discussion of mathematical thinking; an emphasis on in-depth sustained
learning of core grade-level concepts (rather than a spiral curriculum) to support students’ conceptual understanding
and fluency; and a “learn by teaching” design to support teachers new to the curriculum. Embedded in the program are
five core classroom structures—Building Concepts, Math Talk, Student Leaders, Quick Practice, and Helping
Community—that support children from all backgrounds in developing mathematical understanding, competence, and
confidence.

Saxon Math
For almost 20 years, Saxon Math has been providing elementary math curriculum that uses a multisensory
approach designed to enable all children to develop a solid foundation in the language and basic concepts of
mathematics. The program is intended to align with how young children learn and build fluency with math skills.
This is accomplished through hands-on activities and mathematical conversations that actively engage students in the
learning process. Concepts are developed reviewed and practiced over time supported by a philosophy that believes
that understanding follows doing and discussing; mastery follows learning over time, and fluency follows practicing
over time. Saxon is an imprint of Harcourt Achieve, Inc. Harcourt Achieve produces learning solutions and content
that fundamentally and positively change the lives of young and adult learners. Published under the Rigby, Saxon and
Steck-Vaughn imprints, its products are based on a developmental philosophy that assesses learners’ skills, matches
them to appropriate content and accelerates them to meet and exceed expectations. The Rigby imprint offers
progressive learning solutions for core reading and English language learner instruction that provide differentiated
instruction to match each student’s instructional level. The Saxon imprint offers the nation’s best selling and most
thoroughly researched skills-based mathematics program for grades K-12, as well as popular phonics, K-3 spelling,
and early learning programs. The Steck-Vaughn imprint offers easy-to-use, innovative learning solutions that
accelerate content-area knowledge, reading skills, and preparation for standards-based tests, allowing learners to meet
and exceed expectations. For more information, please visit www.HarcourtAchieve.com.

Scott Foresman-Addison Wesley Mathematics
Scott Foresman-Addison Wesley Mathematics promotes mathematical proficiency by focusing on the
development of both mathematics skills and essential understandings. This is accomplished through:
• An articulation of essential outcomes and conceptual understandings for both the teacher and the student
• Questioning strategies that develop higher order-thinking skills embedded into the student and teacher
materials
• Development of mathematical communication as a means of building a deep understanding of important
mathematics
A hallmark of Scott Foresman-Addison Wesley Mathematics is explicit instruction of essential mathematics skills
and concepts, using concrete manipulatives and pictorial and abstract representations. This approach helps to move all
students forward in the development of mathematical proficiency. Ongoing assessment and diagnosis are coupled with
strategic intervention to meet the individual needs of students, including frequent and timely student assessments
integrated throughout the program to demonstrate student understanding and guide and monitor instruction. The
authors of Scott Foresman-Addison Wesley Mathematics also recognize the importance of quality, ongoing
professional development and teacher support. Thus, professional development is provided daily within the teaching
materials and is ongoing in multiple formats, including various uses of technology, to support the continued
development of highly qualified teachers.

APPENDIX E
Confidentiality Pledge

CONFIDENTIALITY PLEDGE

I understand that the names, and any other identifying facts or information, of individuals, businesses,
organizations, and families participating in projects conducted by Mathematica, Inc. or its subsidiaries are
confidential information. I agree that I will not reveal such confidential information, regardless of how or where I
acquired it, to any person unless such person has been authorized by the cognizant Mathematica Project Director or
the Mathematica Project Manager to have access to the information.
I further understand that the unauthorized access to, use, or disclosure of any confidential information is a
breach of the terms of my employment, or my consultant agreement with Mathematica and may subject me to court
action by any interested party or to other sanctions by Mathematica. I acknowledge that this agreement shall
continue to bind me even after the project(s) is (are) completed and/or even though my employment or my
consultant agreement with Mathematica has terminated.
In addition, in the course of my employment I may have access to personal information, electronic and
otherwise, about fellow employees. I agree that I will treat that information as having the highest confidentiality,
and not communicate it to fellow employees or others outside Mathematica. Final determination of whether or not
there is a business purpose requiring that I access a fellow employees’ records will be made in consultation with
the Director of Human Resources. Failure to uphold this standard is a breach of trust and may subject me to
disciplinary action, including termination of employment.
Other than in the course of my authorized employment or my consultant agreement, I further agree that I will
not use, nor facilitate the use by any third party, in any way any information deemed confidential by the terms of
any contract or other written agreement between Mathematica and any other organization, except by written
authorization by both parties. It is my understanding that Mathematica and the contracting organization(s) have the
exclusive right to all information acquired or developed under such a contract or other written agreement. I
acknowledge that I acquire no right, title, or interest in and to any data or information to which I have access by
reason of my employment or my consultant agreement and that I may not remove such data from my assigned
work location without prior authorization.
I agree to promptly notify the cognizant Mathematica Project Director or Project Manager, the Survey
Operations Center Manager or Supervisor for survey work, and the Mathematica Security Officer of any
unauthorized disclosure, use, or alteration of confidential information that I observe.
Nothing herein shall be construed to prevent divulgence of information to any court or governmental agency,
provided such divulgence is required by law. However, if I am subpoenaed, or if I have reason to believe that I
may be called upon to make such divulgence, I agree to notify the President of Mathematica promptly in writing
and, upon his request, to cooperate in all lawful efforts to resist such divulgence.

Name:

Signature:
Date:

Revised December 2003

APPENDIX F
Teacher Survey Pretest Memo

MEMORANDUM

Date:

June 23, 2006

To:

Sheila Heaviside, Mathematica Policy Research

From:

Alejandra Lopez-Torkos

Re:

Pilot of Teacher Survey for the Evaluation of Elementary Mathematics Curricula

A preliminary pilot of the teacher survey for the Evaluation of the Elementary Mathematics
Curricula was conducted with two teachers at an elementary school in the San Francisco Bay
Area in March 2006. One pilot participant was a novice teacher, in her second year of teaching;
the other was a veteran teacher, in her 23rd year of teaching. This memo describes the
overarching issues raised during the pilot; and summarizes teachers’ comments, noting the
revisions made to the survey as well as possible additional revisions.
Overall, both teachers responded favorably to the survey, and commented that the language was
appropriate and that it was easy to complete. The novice teacher reported that it took her 20
minutes to fill out the survey; and the veteran said it took her almost 30 minutes, noting that she
spent more time thinking about each question as part of the pilot than she would have if she were
simply completing the survey as a regular respondent. This experienced teacher also found it
challenging to answer the question about approaches to teaching because she was not sure
whether her answers should reference the students she is teaching math to this year or students in
general. This was especially an issue since the pilot school groups students for math instruction
according to student ability, so the teachers work with a different group of children for math than
they necessarily have for the rest of the school day. As noted as a possible revision (below), it
may make sense to move the item about approaches to teaching to the top of the survey before
the items that ask teachers to focus on the target class for the study. For some items, which ask
teachers the extent to which they agree or disagree with statements, pilot participants expressed
an interest in having “neutral” be a response option. The novice teacher also reported that she
found it difficult to answer items that ask for the average frequency of particular activities over
the course of a school year. The other comments primarily were suggestions about wording and
about providing clarification in question stems.
Teachers’ comments are organized below by section of the survey. Keep in mind that there are
two parts to the survey – one proposed to be administered in the Fall, one in the Spring. Also,
note that some items have not yet been piloted since they were more recently developed,
including: F4-F6, F8, F17b, F20c, S1, S3, S5, S9, S14-17, S19 and parts of F19, S13. These will
be piloted with teachers prior to finalizing the survey.
Training, Resources, and Support for Teaching Math (Fall)
F3

The novice teacher was not sure whether the last response option, which used to read, “I
have my own materials for use with my class," meant that she owns these items
personally or that they are simply housed in her classroom for use with her students and
that she does not have to share them with other teachers.
Revision: Response option was reworded to read, “Materials are dedicated for use with
my students.”

Math Instruction in Your Target Class (Fall)
F10

Novice teacher requested that “approximately” be added to the beginning of this item
wording. She found the item easy to answer since students are grouped by ability for
math in her school, but she wondered if it would be more difficult to answer with a more
heterogeneous class.
Revision: “Approximately” was added to the beginning of the prompt. (Items F11 and
F12 were also revised in this way.)

F13

The veteran teacher questioned whether to include nightly grading in this count of
minutes spent preparing to teach math during the week.
Revision: Added to the end of the prompt: “(including lesson planning, grading student
work, etc.).”

F17a The veteran teacher requested more space to list curricula used.
Revision: Added another line for teachers to list curricula.
Approaches to Teaching (Fall)
F19

The veteran teacher was challenged in answering questions about teaching approaches
because she was not sure whether to answer only with regard to the students she teaches
math to (this year), or students more generally.
Revision: Refer to statements as “general statements” in the prompts to try to get teachers
to answer more broadly about teaching students in general.
Additional recommendation: Could move the question about attitudes and beliefs about
teaching to the front of the survey before respondents are zoomed in to focus on their
target classes.

F19

The novice teacher desired a “neutral” option, between “disagree” and “agree.”
Possible revision: Could add a middle, “neutral,” response option.

F19c Both teachers found the wording of this prompt confusing. Also, the veteran teacher
noted that she tends to correct students’ misconceptions the next day.
No revision recommended at this time.
Possible revision: Could reword to read: “All students’ questions about math are
resolved by the end of the lesson.”
F19f

The novice teacher recommended adding “or with a partner” to the end of the prompt that
used to read, “Students learn math best when they share their reasoning about a math
problem with the class.”
Revision: Prompt reworded: “Students learn math best when they share their reasoning
about a math problem with other students.”

Your School’s Instructional Climate (Fall)
F23

The veteran teacher desired a “neutral” option, between “disagree” and “agree.”
Possible revision: Could add a middle, “neutral,” response option.
2

F23

In responding to this question, the veteran teacher focused on the 3rd grade teachers at
her school. She questioned whether she was supposed to instead think about all teachers
at her school.
No revision recommended at this time. The item, as currently worded, allows
respondents to answer the question in the way that is most applicable or meaningful to
them. We will ask teachers in the next round of piloting the survey whether this is an
issue for them.

F23g The veteran teacher suggested specifying changes as curricular ones if that is what we
intend to learn about with this item.
Revision: Reword prompt: “Most curriculum changes introduced at this school gain little
support among teachers.” “Curriculum” was added to the prompt.
Training, Resources, and Support for Teaching Math (Spring)
S2

The novice teacher desired an unsure/don’t know option in Column A, which asks about
support from the curriculum publisher. The question had only Yes/No response options.
Revision: Split item S2 into two parts (S2a, S2b) and added a “Don’t know” option to S2a.

The novice teacher recommended that it be clarified how a teacher who is taking course
work to “clear” her teaching credential should respond to this item. A response option
used to read, “Coursework taken towards a teaching credential.”
Revision: This response option was reworded: “Coursework taken towards a credential
for teaching.”
Additional recommendation: Could revise the second and third response options as
follows: (ii) “Coursework taken towards a preliminary teaching credential or professional
clear credential”; (iii) “University coursework in math or math instruction, not including
coursework for a preliminary teaching credential or a professional clear teaching
credential.”

Math Instruction in Your Target Class (Spring)
S10a The novice teacher found this question confusing; she was not sure how to estimate how
much she used the curriculum. When she completed the survey, she estimated based on
the number of days in a week she works from the textbook.
Revision: Reword prompt: “Approximately what percentage of all the lessons from the
assigned curriculum have you used so far this year with your target class?” “All” was
added to the prompt.
S11e

The novice teacher found the word “scripts” confusing in the prompt, “Lesson plans or
scripts.”
Revision: Clarified as “lesson scripts” in the prompt.

S12

The veteran teacher desired a “neutral” option, between “disagree” and “agree.”
Possible revision: Could add a middle, “neutral,” response option.

APPENDIX G
Teacher Training Observation Protocol

EVALUATION OF MATHEMATICS CURRICULA
PROFESSIONAL DEVELOPMENT OBSERVATION PROTOCOL
GENERAL INSTRUCTIONS
Reminders:
•

Please collect an agenda and all handouts

•

Submit agenda, handouts, field notes and completed protocol to Bowyee Gong at:
Bowyee Gong
SRI International, BN 353
Menlo Park, CA 94025
[email protected]

Section A: Field Notes
This section is used to keep field notes on each topic area covered in the training, such as the
research supporting the curriculum, curricular structure, instructional techniques, mathematical
content, lesson protocol, pacing, and teachers’ questions/concerns. Remember to record the
format of the training (e.g., lecture, demonstration, discussion) as well.
Note: Please take time to attend to and describe any content of the training related to the items
under Section B Content of Training. These are activities that you will be asked to code for from
your field notes following the trainings.
Field notes can be kept in a format unique to the observer; however, an example format is
included. You may use this form or construct your own. Handwritten or electronic versions are
acceptable. If submitting handwritten notes, please write legibly. Please take notes on all days of
the training. Indicate the topic and the amount time for each topic.
[Note for laptop users: You can insert your notes directly into the tables included under
Section A. Use one table row per note. Rows will expand in height as additional lines of
text are inserted. Add rows to the table for each day, as required.]
Section B: Content of Training
This section is to get a snapshot of important instructional activities covered in the training. Use
your field notes to help complete this section accurately. For each instructional activity, indicate
how much time (not covered, less than 30 minutes, 30 minutes of more) if any. For trainings that
last more than 1 day, only one summary form for the entire training is needed (not one for each
day).
Any item checked as being covered during the training should be supported in your field notes
with a description of the activity.
1

Note on items 6, 7, 8, and 9: Only check these items as being covered if the trainer actually
spends time teaching the teachers on how to differentiate instruction, respond to student errors or
use the assessments. For example, if a facilitator does not instruct participants on how to
implement differentiated instruction for English Language Learners, but instead only shows
teacher where to find the curriculum resource material for English Language Learners or
suggests that teachers review the material, then the observer must mark “Not Covered” for item
6, regardless of how much time is spent on this activity.
Section C: Participant Concerns and Overall Appraisal
In this section you are asked to:
1. Record concerns that participants raise related to either the general approach of the
curriculum or implementation of the curriculum. Having a record of participants’
concerns will support the development of other protocols and inform the research team
about potential issues that might impact implementation fidelity during the study.
2. Provide your overall appraisal of the training. It might be helpful to review your field
notes (section A) and the content of the training (section B) to provide a basis for your
overall appraisal of this training. We would like you to comment on how well you think
the training went, how effective the facilitators were, and how engaged the teachers
were. You may comment on any overall observations you had about the training as well.

Training Information
Curriculum:________________________________________________________
Location of Training:_________________________________________________
Date(s) of Training:_________________________________________________
Observer:__________________________________________________________
Start Time: ____________________ End Time: ___________________________
Length of Training (hours/minutes): _______________________________________

SECTION A: FIELD NOTES
[Note for laptop users: Use one table row per note. Rows will expand in height as
additional lines of text are inserted. Add rows to the table for each day, as required.]
OBSERVATION – DAY 1

Topic

Starting/
Ending
Times

Notes

OBSERVATION – DAY 2

Topic

Starting/
Ending
Times

Notes

SECTION B: CONTENT OF TRAINING
* Please read note on items 6-9 under General Instructions.
Coverage

Not
Covered
1.

Facilitator models or role plays
instructional activities

Participants practice key instructional
activities in the role of the teacher

Participants practice key instructional
activities in the role of the students

Participants watch a video of key
instructional activities

Participants plan actual lessons

Facilitator instructs participants on how to
differentiate instruction for English
Language Learners

Facilitator instructs participants on how to
differentiate instruction for students with
different abilities, including special
education

Facilitator instructs participants on how to
address incorrect student responses

Facilitator instructs participants on the use
of curriculum assessments

10. Facilitator reviews actual math content
covered in the curriculum (e.g., What is an
ordinal number?; Does NOT include a
listing of math topics covered i.e., scope
and sequence).
11. Facilitator provides overview of research
supporting the curriculum

Extended
Covered
Coverage
Briefly
(30
(Less than Minutes or
30 Minutes)
More)

SECTION C: PARTICIPANT CONCERNS AND OVERALL APPRAISAL
During the training, what concerns and issues did participants raise related to the general
approach of the curriculum or implementing the curriculum?

Please provide your overall appraisal of the training below. It might be helpful to review
your field notes (section B) and the content of the training (section A) to provide a basis for
your overall appraisal of this training. We would like you to comment on how well you
think the training went, how effective the facilitators were, and how engaged the teachers
were.

File Type	application/pdf
File Modified	2006-09-15
File Created	2006-03-13