Appendices A-H

Appendix A: Baseline Teacher Survey

Welcome! This survey is being conducted for the National Aeronautics and Space Administration (NASA) as part of its efforts to learn about the teachers who participate in the Summer of Innovation program. Teachers who participate in the Summer of Innovation program during summer 2010 are being asked to complete this survey. We estimate that it will take approximately 20 – 25 minutes to complete the questionnaire. Thank you very much for your cooperation!

The Summer of Innovation is a 3-year initiative that provides middle school students, who underperform, are underrepresented, and underserved in science, technology, engineering and mathematics (STEM) fields, with intensive, stimulating math and science based learning experiences using NASA’s STEM assets.

All information that would permit identification of individual respondents will be held in strict confidence, will be used only by persons engaged in and for the purposes of the survey, and will not be disclosed or released to others for any purpose except as required by law. For more information about this data collection, including OMB clearance and burden estimates, please contact Lori Parker, NASA PRA Clearance Officer ([email protected], 202-358-4616).

Contact Information

Please print your name, address, telephone numbers, and e-mail address.

Name:
	Last name First name MI
Address (include number, street, apartment number, P. O. box, etc.)
	City State Zip Code
Home Telephone:	( ________ ) _________ - ___________	 I do not have a telephone
Work Telephone:	( ________ ) _________ - ___________	 I do not have a telephone
Cell Telephone:	( ________ ) _________ - ___________	 I do not have a telephone
E-mail address:	______________________________	 I do not have an email address

1. For items a–y, please consider your practice within your own subject area of science or math and indicate the degree to which you agree or disagree with each statement below by circling the appropriate number to the right of each statement. Please circle only ONE value per statement.

1= Strongly Disagree

2 = Disagree

3 = Uncertain

4 = Agree

5 = Strongly Agree

Statement	Strongly Disagree	Disagree	Uncertain	Agree	Strongly Agree
When a student does better than usual in science/math class, it is often because the teacher exerted a little extra effort.	1	2	3	4	5
I am continually finding better ways to teach.	1	2	3	4	5
Even when I try very hard, I don't teach science/math well.	1	2	3	4	5
When the grades of students improve, it is most often due to their teacher having found a more effective teaching approach.	1	2	3	4	5
I know the steps necessary to teach science/math concepts effectively.	1	2	3	4	5
I am not very effective in monitoring science/math hands-on activities or investigations.	1	2	3	4	5
If students are underachieving in science/math classes, it is most likely due to ineffective teaching.	1	2	3	4	5
I generally teach science/math ineffectively.	1	2	3	4	5
The inadequacy of a student's science/math background can be overcome by good teaching.	1	2	3	4	5
The low science/math achievement of some students cannot generally be blamed on their teachers.	1	2	3	4	5
When a low achieving child progresses, it is usually due to extra attention given by the teacher.	1	2	3	4	5
I understand science/math concepts well enough to be an effective teacher.	1	2	3	4	5

Statement	Strongly Disagree	Disagree	Uncertain	Agree	Strongly Agree
Increased effort in teaching produces little change in some students' achievement.	1	2	3	4	5
The science/math teacher is generally responsible for the achievement of students in science/math.	1	2	3	4	5
Students' achievement in science/math is directly related to their teacher's effectiveness in science/math teaching.	1	2	3	4	5
If parents comment that their child is showing more interest in science/math at school, it is probably due to the performance of the child's teacher.	1	2	3	4	5
I find it difficult to explain to students why science experiments or math problems work.	1	2	3	4	5
I am typically able to answer students' science/math questions.	1	2	3	4	5
I wonder if I have the necessary skills to teach science/math.	1	2	3	4	5
Effectiveness in science/math teaching has little influence on the achievement of students with low motivation.	1	2	3	4	5
Given a choice, I would not invite the principal to evaluate my science/math teaching.	1	2	3	4	5
When a student has difficulty understanding a concept, I am usually at a loss as to how to help the student understand it better.	1	2	3	4	5
When teaching science/math, I usually welcome student questions.	1	2	3	4	5
I don't know what to do to turn students on to science/math.	1	2	3	4	5
Even teachers with good science/math teaching abilities cannot help some kids learn.	1	2	3	4	5

For items a-p, please indicate the degree to which each described behavior occurs by circling the appropriate number to the right of each statement. Please circle only ONE value per statement.

1= Never

2 = Rarely (e.g., a few times a year)

3 = Sometimes (e.g., once or twice a month)

4 = Often (e.g., once or twice a week)

5 = All or almost all science / math lessons

2. About how often do you do each of the following in your instruction?

Statement	Never	Rarely (e.g., a few times a year)	Sometimes (e.g., once or twice a month)	Often (e.g., once or twice a week)	All or almost all science / math lessons
Introduce content through formal presentations	1	2	3	4	5
Pose open-ended questions	1	2	3	4	5
Engage the whole class in discussions	1	2	3	4	5
Require students to supply evidence to support their claims	1	2	3	4	5
Ask students to explain concepts to one another	1	2	3	4	5
Ask students to consider alternative explanations	1	2	3	4	5
Help students see connections between science/math and other disciplines	1	2	3	4	5
Assign homework	1	2	3	4	5

3. About how often do students in your class take part in the following types of activities?

Statement	Never	Rarely (e.g., a few times a year)	Sometimes (e.g., once or twice a month)	Often (e.g., once or twice a week)	All or almost all science / math lessons
Listen and take notes during presentation by teacher	1	2	3	4	5
Work in groups	1	2	3	4	5
Read from a science/math textbook in class	1	2	3	4	5
Do hands-on activities or investigations	1	2	3	4	5
Answer textbook or worksheet questions	1	2	3	4	5
Record, represent, and/or analyze data or numbers	1	2	3	4	5
Follow specific instructions in a hands-on activity or investigation	1	2	3	4	5

4. How often do you assess student progress in each of the following ways?

Statement	Never	Rarely (e.g., a few times a year)	Sometimes (e.g., once or twice a month)	Often (e.g., once or twice a week)	All or almost all science / math lessons
Review student homework	1	2	3	4	5
Give predominantly short-answer tests (e.g., multiple choice, true/false, fill in the blank)	1	2	3	4	5

5. Do you hold a teaching license or certificate?

• Yes

• No (skip to question 8)

5. What type of teaching license or certificate do you hold? Please check ALL that apply.

• National board certificate

• State certificate (not national board certificate)

• Provisional certificate

• Emergency certificate

• Other, please specify:_______________________________

5. What subject area is your license or certificate? Please check ALL that apply.

• Elementary education (K-5)

• Secondary education (6-12)

• Science education

• Mathematics education

• Special education

• Other, please specify:_______________________________

5. Do you have each of the following degrees? Please circle only ONE value per row.

1= Yes

2 = No

	Yes	No
Bachelor’s	1	2
Master’s	1	2
Doctorate	1	2

5. What grade(s) are you teaching in the upcoming 2010-2011 school year? Please check ALL that apply.

• Elementary grades (K-5)

• Middle grades (6-8)

• High school grades (9-12)

• I will not be teaching in a K-12 classroom in the 2010-2011 school year (skip to question 11)

5. Which of the following subject area(s) are you teaching in the upcoming 2010-2011 school year? Please check ALL that apply.

• Mathematics

• Science

• Computer Science

• Technology

• Engineering

• Other, please specify: _________________________

5. Counting this past school year, how many years have you taught at the elementary and secondary level? Please also note the number of years in total. Enter the number of years in each row EXCLUDING any student teaching that you’ve done. If zero, enter 0.

Elementary (K-5) _____ yrs

Secondary (6-12) _____ yrs

Total (K-12) _____yrs

5. To what extent have you used NASA content, materials or experts for instructional purposes in a K-12 classroom? Please check only ONE value per statement.

• Never

• Rarely (e.g., a few times a year)

• Sometimes (e.g., once or twice a month)

• Often (e.g., once or twice a week)

• Always (e.g., all or almost all science / math lessons)

5. How would you rate your current level of need for professional development in each of these areas? Please circle only ONE value per statement.

1= None needed

2 = Slightly needed

3 = Needed

4 = Very needed

5 = Critically needed

	None needed	Slightly needed	Needed	Very needed	Critically needed
Deepening my own STEM content knowledge	1	2	3	4	5
Understanding student thinking in STEM	1	2	3	4	5
Learning how to use inquiry/investigation-oriented teaching strategies	1	2	3	4	5
Learning how to use technology in STEM instruction	1	2	3	4	5
Learning how to assess student learning in STEM	1	2	3	4	5
Learning how to teach STEM in a class that includes students with special needs	1	2	3	4	5

5. Do you consider yourself to be Hispanic or Latino? Please check only one.

• Yes

• No

5. What is your race? Select one or more.

• American Indian or Alaska Native

• Asian

• Black or African American

• Native Hawaiian or Other Pacific Islander

• White

5. What is your gender? Please check only one.

• Male

• Female

Appendix B: Post Teacher Survey

Welcome! This survey is being conducted for the National Aeronautics and Space Administration (NASA) as part of its efforts to learn about the teachers who participate in the Summer of Innovation program. Teachers who participate in the Summer of Innovation program during summer 2010 are being asked to complete this survey. We estimate that it will take approximately 10 minutes to complete the questionnaire. Thank you very much for your cooperation!

The Summer of Innovation is a 3-year initiative that provides middle school students, who underperform, are underrepresented, and underserved in science, technology, engineering and mathematics (STEM) fields, with intensive, stimulating math and science based learning experiences using NASA’s STEM assets.

All information that would permit identification of individual respondents will be held in strict confidence, will be used only by persons engaged in and for the purposes of the survey, and will not be disclosed or released to others for any purpose except as required by law. For more information about this data collection, including OMB clearance and burden estimates, please contact Lori Parker, NASA PRA Clearance Officer ([email protected], 202-358-4616).

Contact Information

1. Please print your name and e-mail address.

Last Name: ______________________________ First Name: ____________________ M.I.______
Email address:

2. Will you be teaching at least one course in the fall? Yes No (Please skip to question 7.)

For the following items, please indicate the degree to which each described behavior occurs by circling the appropriate number to the right of each statement. Please circle only ONE value per statement.

1= Never, 2 = Rarely (e.g., a few times a year). 3 = Sometimes (e.g., once or twice a month),
4 = Often (e.g., once or twice a week), 5 = All or almost all science / math lessons

3. In the upcoming school year, how often are you planning on doing each of the following in your instruction?

Statement	Never	Rarely (e.g., a few times a year)	Sometimes (e.g., once or twice a month)	Often (e.g., once or twice a week)	All or almost all science / math lessons
Introduce content through formal presentations	1	2	3	4	5
Pose open-ended questions	1	2	3	4	5
Engage the whole class in discussions	1	2	3	4	5
Require students to supply evidence to support their claims	1	2	3	4	5
Ask students to explain concepts to one another	1	2	3	4	5
Ask students to consider alternative explanations	1	2	3	4	5
Help students see connections between science/math and other disciplines	1	2	3	4	5
Assign homework	1	2	3	4	5

4. In the upcoming school year, how often are you planning on having students in your class take part in the following types of activities? (Please complete table, following page)

Statement	Never	Rarely (e.g., a few times a year)	Sometimes (e.g., once or twice a month)	Often (e.g., once or twice a week)	All or almost all science / math lessons
Listen and take notes during presentation by teacher	1	2	3	4	5
Work in groups	1	2	3	4	5
Read from a science/math textbook in class	1	2	3	4	5
Do hands-on activities or investigations	1	2	3	4	5
Answer textbook or worksheet questions	1	2	3	4	5
Record, represent, and/or analyze data or numbers	1	2	3	4	5
Follow specific instructions in a hands-on activity or investigation	1	2	3	4	5

5. In the upcoming school year, how often are you planning on assessing student progress next year in each of the following ways?

Statement	Never	Rarely (e.g., a few times a year)	Sometimes (e.g., once or twice a month)	Often (e.g., once or twice a week)	All or almost all science / math lessons
Review student homework	1	2	3	4	5
Give predominantly short-answer tests (e.g., multiple choice, true/false, fill in the blank)	1	2	3	4	5

6. In the upcoming school year, to what extent are you planning on using NASA content, materials or experts for instructional purposes in a K-12 classroom? Please check only ONE value per statement.

 Never  Rarely (e.g., a few times a year)  Sometimes (e.g., once or twice a month)
 Often (e.g., once or twice a week)  Always (e.g., all or almost all science / math lessons)

7. How would you rate your current level of need for professional development in each of these areas? Please circle only ONE value per statement.

STEM = Science, Technology, Engineering, and/or Mathematics	None needed	Slightly needed	Needed	Very needed	Critically needed
Deepening my own STEM content knowledge	1	2	3	4	5
Understanding student thinking in STEM	1	2	3	4	5
Learning how to use inquiry/investigation-oriented teaching strategies	1	2	3	4	5
Learning how to use technology in STEM instruction	1	2	3	4	5
Learning how to assess student learning in STEM	1	2	3	4	5
Learning how to teach STEM in a class that includes students with special needs	1	2	3	4	5

Appendix C: Student Baseline Survey

Welcome! Congratulations on being part of NASA’s Summer of Innovation. Students who attend the Summer of Innovation program during summer 2010 are being asked to complete this survey. There are no “right” or “wrong” answers to any of the questions. Your opinion is what is wanted.

We estimate that it will take about 15 minutes to complete the questions. Thank you very much for your help!

All information that would permit identification of individual respondents will be held in strict confidence, will be used only by persons engaged in and for the purposes of the survey, and will not be disclosed or released to others for any purpose except as required by law. For more information about this data collection, including OMB clearance and burden estimates, please contact Lori Parker, NASA PRA Clearance officer ([email protected], 202-358-4616).

Contact Information

1. Please print your name, address, home telephone number, and e-mail address.

Last name _______________________________First name ________________________________MI__
Address (include number, street, apartment number, P. O. box, etc.)
	City State Zip Code
Telephone:	( ________ ) _________ - _________	 I do not have a telephone
E-mail address:	___________________________	 I do not have an e-mail address

2. Please provide the name of one parent or guardian with whom you live most of the time.

Last name __________________________________First name _________________________________

3. What is your parent or guardian’s work phone number?

Work telephone:

( _______ ) ________ - ________ ext. ____

 Does not have a work telephone

4. Is your parent or guardian’s address and telephone number the same as yours? Please check one.

No Yes  Skip to Question 6

5. Please fill in your parent or guardian’s address and telephone number in the space below. If you don't know the complete address, fill in as much as you know.

Address (include number, street, apartment number, P. O. box, etc.
	City State Zip Code
Home telephone:	( ________ ) _________ - _________	 He/She does not have a telephone

6. Please write in the name and telephone number of a relative or close friend who does not live with you and who will always know how to contact you.

Last name _______________________________First name _________________________________MI __
Telephone:	( ________ ) _________ - ___________

7. What is this person's relationship to you? Please check one.

A parent A grandparent An aunt or uncle A brother or sister A friend

Other ________________________

8. Please write in the name and telephone number of another relative or close friend who does not live with you and who will always know how to contact you.

Last name _______________________________First name _________________________________MI __
Telephone:	( ________ ) _________ - ___________

9. What is this person's relationship to you? Please check one.

A parent A grandparent An aunt or uncle A brother or sister A friend

Other ________________________

10. Grade level completed Spring 2010. Please check one only.

5^th 6^th 7^th 8^th 9^th 10^th Name of school____________________________

11. Name of school you will be attending in the upcoming year____________________________

12. This next question contains a number of statements about math/science. You will be asked what you think about these statements. There are no “right” or “wrong” answers. Your opinion is what is wanted.

1= Strongly Disagree, 2 = Disagree, 3 = Uncertain, 4 = Agree, 5 = Strongly Agree

For each statement, draw a circle around the specific value corresponding to how you feel about each statement. Please circle only ONE value per statement.

Statement	Strongly Disagree	Disagree	Uncertain	Agree	Strongly Agree
I would like to belong to a science or math club.	1	2	3	4	5
I do not do very well in science.	1	2	3	4	5
I would like to be a scientist when I leave school.	1	2	3	4	5
I get bored when watching science or math related programs on TV at home.	1	2	3	4	5
Math is easy for me.	1	2	3	4	5
I would dislike becoming a scientist because it needs too much education.	1	2	3	4	5
I would like to be given a science or math book or a piece of scientific equipment as a present.	1	2	3	4	5
I usually understand what we are talking about in science.	1	2	3	4	5
A job as a scientist would be interesting.	1	2	3	4	5
I dislike reading books about science or math during my free time.	1	2	3	4	5
No matter how hard I try, I cannot understand math.	1	2	3	4	5
A job as a scientist would be boring.	1	2	3	4	5
I would like to do science experiments or math problems at home.	1	2	3	4	5
I often think, “I cannot do this,” when a science assignment seems hard.	1	2	3	4	5
I would like to teach science when I leave school.	1	2	3	4	5
I would like to teach math when I leave school.	1	2	3	4	5
Talking to friends about science or math after school would be boring.	1	2	3	4	5
I do not do very well in math.	1	2	3	4	5
A career in science would be boring.	1	2	3	4	5
A career in math would be boring.	1	2	3	4	5
I would enjoy having a job in a science laboratory during my summer vacation.	1	2	3	4	5
Science is easy for me.	1	2	3	4	5
Working in a science laboratory would be an interesting way to earn a living.	1	2	3	4	5
Watching a video about science or math would be boring.	1	2	3	4	5
I usually understand what we are talking about in math.	1	2	3	4	5
I would dislike a job in a science laboratory after I leave school.	1	2	3	4	5
I would enjoy visiting a science museum on the weekend.	1	2	3	4	5
No matter how hard I try, I cannot understand science.	1	2	3	4	5
When I leave school, I would like to work with people who make discoveries in science or math.	1	2	3	4	5
I dislike looking at websites about science or math.	1	2	3	4	5
I often think, “I cannot do this,” when a math problem seems hard.	1	2	3	4	5
I would dislike being a scientist after I leave school.	1	2	3	4	5

13. For the next set of questions, please indicate how often each described activity occurs by circling the appropriate number to the right of each statement.

1= Never, 2 = Rarely (e.g., a few times a year), 3 = Sometimes (e.g., once or twice a month), 4 = Often (e.g., once or twice a week), 5 = Always or almost always (e.g., everyday)

How often did one or both of your parents or guardians do the following during the past school year (the school year that just ended)? Please circle only ONE value per statement.

	Never	Rarely (e.g., a few times a year)	Sometimes (e.g., once or twice a month)	Often (e.g., once or twice a week)	Always or almost always (e.g., everyday)
Helped you with your homework or a project for school	1	2	3	4	5
Checked on whether you had done your homework	1	2	3	4	5
Went with you to an event (e.g., movie, play, museum, concert, sports event)	1	2	3	4	5
Got upset or angry about your behavior	1	2	3	4	5
Got upset or angry about your grades	1	2	3	4	5
Rewarded you for your grades	1	2	3	4	5

14. How often did you talk about the following with one or both of your parents or guardians during the past school year (the school year that just ended)? Please circle only ONE value per statement.

	Never	Rarely (e.g., a few times a year)		Sometimes (e.g., once or twice a month)	Often (e.g., once or twice a week)	Always or almost always (e.g., everyday)
Selecting courses or programs at school	1		2	3	4	5
School activities or events of particular interest to you	1		2	3	4	5
Things you’ve studied in class	1		2	3	4	5
Your school work or grades	1		2	3	4	5
Switching to a different school	1		2	3	4	5
Going to college	1		2	3	4	5
A personal problem you were having	1		2	3	4	5
Getting in trouble at school	1		2	3	4	5
Getting rewarded at school	1		2	3	4	5

15. How often did your friend or friends do the following during the past school year (the school year that just ended)? Please circle only ONE value per statement.

	Never	Rarely (e.g., a few times a year)	Sometimes (e.g., once or twice a month)	Often (e.g., once or twice a week)	Always or almost always (e.g., everyday)
Encourage you to disobey your parents or teachers	1	2	3	4	5
Encourage you to do what your parents or teachers want you to do	1	2	3	4	5
Get in trouble at school	1	2	3	4	5
Get rewarded at school	1	2	3	4	5

16. How often did you talk about the following with a friend or friends during the past school year (the school year that just ended)? Please circle only ONE value per statement.

	Never	Rarely (e.g., a few times a year)	Sometimes (e.g., once or twice a month)	Often (e.g., once or twice a week)	Always or almost always (e.g., everyday)
Selecting courses or programs at school	1	2	3	4	5
School activities or events of particular interest to you	1	2	3	4	5
Things you’ve studied in class	1	2	3	4	5
Your school work or grades	1	2	3	4	5
Switching to a different school	1	2	3	4	5
Going to college	1	2	3	4	5
A personal problem you were having	1	2	3	4	5
Getting in trouble at school	1	2	3	4	5
Getting rewarded at school	1	2	3	4	5

17. Do you consider yourself to be Hispanic or Latino? Please check one only.

Yes No

18. What is your race? Select one or more.

American Indian or Alaska Native

Asian

Black or African American

Native Hawaiian or Other Pacific Islander

White

19. What is your gender? Please check one only.

Male

Female

Appendix D: Student Post Survey

Welcome! Congratulations on being part of NASA’s Summer of Innovation. Students who attend the Summer of Innovation program during summer 2010 are being asked to complete this survey. There are no “right” or “wrong” answers to any of the questions. Your opinion is what is wanted. We estimate that it will take about 15 minutes to complete the questions. Thank you very much for your help!

All information that would permit identification of individual respondents will be held in strict confidence, will be used only by persons engaged in and for the purposes of the survey, and will not be disclosed or released to others for any purpose except as required by law. For more information about this data collection, including OMB clearance and burden estimates, please contact Lori Parker, NASA PRA Clearance officer ([email protected], 202-358-4616).

Contact Information

Please print your name.

Name:
	Last name First name MI

This question contains a number of statements about math/science. You will be asked what you think about these statements. There are no “right” or “wrong” answers. Your opinion is what is wanted.

1= Strongly Disagree

2 = Disagree

3 = Uncertain

4 = Agree

5 = Strongly Agree

1. For each statement, draw a circle around the specific value corresponding to how you feel about each statement. Please circle only ONE value per statement.

Statement	Strongly Disagree	Disagree	Uncertain	Agree	Strongly Agree
I would like to belong to a science or math club.	1	2	3	4	5
I do not do very well in science.	1	2	3	4	5
I would like to be a scientist when I leave school.	1	2	3	4	5
I get bored when watching science or math related programs on TV at home.	1	2	3	4	5
Math is easy for me.	1	2	3	4	5
I would dislike becoming a scientist because it needs too much education.	1	2	3	4	5
I would like to be given a science or math book or a piece of scientific equipment as a present.	1	2	3	4	5
I usually understand what we are talking about in science.	1	2	3	4	5
A job as a scientist would be interesting.	1	2	3	4	5
I dislike reading books about science or math during my free time.	1	2	3	4	5
No matter how hard I try, I cannot understand math.	1	2	3	4	5
A job as a scientist would be boring.	1	2	3	4	5
I would like to do science experiments or math problems at home.	1	2	3	4	5
I often think, “I cannot do this,” when a science assignment seems hard.	1	2	3	4	5
I would like to teach science when I leave school.	1	2	3	4	5

Statement	Strongly Disagree	Disagree	Uncertain	Agree	Strongly Agree
I would like to teach math when I leave school.	1	2	3	4	5
Talking to friends about science or math after school would be boring.	1	2	3	4	5
I do not do very well in math.	1	2	3	4	5
A career in science would be boring.	1	2	3	4	5
A career in math would be boring.	1	2	3	4	5
I would enjoy having a job in a science laboratory during my summer vacation.	1	2	3	4	5
Science is easy for me.	1	2	3	4	5
Working in a science laboratory would be an interesting way to earn a living.	1	2	3	4	5
Watching a video about science or math would be boring.	1	2	3	4	5
I usually understand what we are talking about in math.	1	2	3	4	5
I would dislike a job in a science laboratory after I leave school.	1	2	3	4	5
I would enjoy visiting a science museum on the weekend.	1	2	3	4	5
No matter how hard I try, I cannot understand science.	1	2	3	4	5
When I leave school, I would like to work with people who make discoveries in science or math.	1	2	3	4	5
I dislike looking at websites about science or math.	1	2	3	4	5
I often think, “I cannot do this,” when a math problem seems hard.	1	2	3	4	5
I would dislike being a scientist after I leave school.	1	2	3	4	5

Appendix E: Justification of Survey Items

Teacher Survey Justification

Instrument	Pilot Group	Format	Scale	Alpha	Items (*= reverse code)
Science Teacher Efficacy Belief Instrument (STEBI) Riggs, I., & Knochs, L. (1990). Towards the development of an elementary teacher’s science teaching efficacy belief instrument. Science Education, 74, 625-637.	Field tested version—327 elementary teachers. Also later revised and used on pre-service elemen. teachers.	5 point Likert, SD to SA.	Personal Science Teaching Efficacy-- the extent to which teachers believe they have the skills to teach science	0.95	2. I am continually finding better ways to teach.
					3. Even when I try very hard, I don’t teach science/math well.*
					5. I know the steps necessary to teach science/math concepts effectively.
					6. I am not very effective in monitoring science/math hands-on activities or investigations.*
					8. I generally teach science/math ineffectively.*
					12. I understand science/math concepts well enough to be an effective teacher.
					17. I find it difficult to explain to students why science experiments or math problems work.*
					18. I am typically able to answer students’ science/math questions.
					19. I wonder if I have the necessary skills to teach science/math.*
					21. Given a choice I would not invite the principal to evaluate my science/math teaching.*
					22. When a student has difficulty understanding a concept, I am usually at a loss as to how to help the student understand it better.*
					23. When teaching science/math I usually welcome students’ questions.
					24. I don’t know what to do to turn students on to science/math.*
			2. Science teaching outcome expectancy (STOE)-- The extent to which teachers believe that certain behaviors lead to improved student outcomes	0.77	1. When a student does better than usual in science/math class, it is often because the teacher exerted a little effort.
					4. When the grades of students improve, it is most often due to their teacher having found a more effective teaching approach.
					7. If students are underachieving in science/math classes it is most likely due to ineffective science teaching.
					9. The inadequacy of a student’s science/math background can be overcome by good teaching.
					10. The low science/math achievement of some students cannot generally be blamed on their teachers.*
					11. When a low achieving child progresses in science/math it is usually due to extra attention given by the teacher.
					13. Increased effort in teaching produces little change in some students’ achievement.*
					14. The science/math teacher is generally responsible for the achievement of students in science/math.
					15. Students’ achievement in science/math is directly related to their teacher’s effectiveness in science/math teaching.
					16. If parents comment that their child is showing more interest in science/math at school, it is probably due to the performance of the child’s teacher.
					20. Effectiveness in science/math teaching has little influence on the achievement of students with low motivation.*
					25. Even teachers with good science/math teaching abilities cannot help some kids learn.*

Instrument	Pilot Group	Format	Scale	Alpha	Items (*= reverse code)
Horizon National Survey of Science and Math Education Weiss, IR, Bandilower, ER, McMahon, KC, Smith, PS (2001). Report on the 2000 National Survey of Science and Math Education, Horizon Research, Inc. www.horizon-research.com	Details of pilot and field testing not provided, but the survey was developed based on previous Horizon National surveys. The sample in this report consisted of 5,728 math and science teachers across all grade levels	5 pt Frequency scale from 0=never to 5= All or almost all lessons	Use of traditional teaching practices	0.78	26a. Introduce content through formal presentations
					26h. Assign science homework.
					27a. [Students] listen and take notes during presentation by teacher.
					27c. [Students] read from a science textbook in class.
					27e. [students] answer textbook or worksheet questions
					28a. Review student homework
					28b. Give predominantly short-answer tests (e.g., multiple choice, true/false, fill in the blank)
			Use of Strategies to Develop Students’ Abilities to Communicate Ideas	0.79	26b. Pose open-ended questions
					26c. Engage the whole class in discussions
					26d. Require students to supply evidence to support their claims
					26e. Ask students to explain concepts to one another.
					26f. Ask students to consider alternative explanations
					26g. Help students see connections between science and other disciplines.
			Use of Laboratory Activities	0.80	27b. [Students] work in groups
					27d. [Students] do hands-on activities or investigations
					27g. [Students] follow specific instructions in a hands-on activity or investigation.
					27f. [Students] record, represent and/or analyze data or numbers.

Student Survey Justification

Source	Pilot group	Scale	Format	Alpha	Items (*=reverse code)
mATSI (Modified Attitudes Toward Science Inventory) Weinburgh & Steele, 2000	Urban fifth grade students, n=1404. 49% male, 51% female; 69% African-American, 31% Caucasian. (Note: During the pilot, items were read aloud as students read them silently.)	Self-confidence in science	Six-point Likert-type, 1=SD to 6=SA	0.68	(2)I do not do very well in science.*
					(20)Science is easy for me.
					(8)I usually understand what we are talking about in science.
					(26)No matter how hard I try, I cannot understand science.*
					(14)I often think, “I cannot do this,” when a science assignment seems hard.*
Adapted from mATSI self-confidence in science scale	N/A	Self-confidence in math	Six-point Likert-type, 1=SD to 6=SA	TBD	(17)I do not do very well in math.*
					(5)Math is easy for me.
					(23)I usually understand what we are talking about in math.
					(11)No matter how hard I try, I cannot understand math.*
					(29)I often think, “I cannot do this,” when a math problem seems hard.*

Weinburgh & Steele (2000) did not provide information about the factor structure, although the authors state that they performed factor analysis on their data using the ATSI in order to guide modifications and create the current shorter version.

Source	Pilot group	Scale	Format	Alpha	Items (*=reverse code)
Adapted from TOSRA (Test of Science Related Attitudes) Fraser, 1981	Original scale (Career Interest in Science): Students in Sydney, Australia, metropolitan area, n=1337. One 7th, one 8th, one 9th, and one 10th grade class from each of 11 schools. N of 7th graders=340, 8th=335, 9th=338, 10th=324.	Career Interest in STEM	Five-point Likert-type, SA to SD	Original scale (Career Interest in Science): 7th grade =0.72 8th grade =0.70 Lott (2002): 0.91	(30)I would dislike being a scientist after I leave school.*
					(27)When I leave school, I would like to work with people who make discoveries in science or math.
					(24)I would dislike a job in a science laboratory after I leave school.*
					(21)Working in a science laboratory would be an interesting way to earn a living.
					(18)A career in science or math would be dull and boring.*
					(15)I would like to teach science or math when I leave school.
					(12)A job as a scientist would be boring.*
					(9)A job as a scientist would be interesting.
					(6)I would dislike becoming a scientist because it needs too much education.*
					(3)I would like to be a scientist when I leave school.

Source	Pilot group	Scale	Format	Alpha	Items (*=reverse code)
Adapted from TOSRA (Test of Science Related Attitudes) Fraser, 1981	Original scale (Career Interest in Science): Students in Sydney, Australia, metropolitan area, n=1337. One 7th, one 8th, one 9th, and one 10th grade class from each of 11 schools. N of 7th graders=340, 8th=335, 9th=338, 10th=324.	Leisure Interest in STEM	Five-point Likert-type, SA to SD	Original scale (Career Interest in Science): 7th grade =0.93 8th grade =0.92 Lott (2002): 0.89	(1)I would like to belong to a science or math club.
					(4)I get bored when watching science or math related programs on TV at home.*
					(7)I would like to be given a science or math book or a piece of scientific equipment as a present.
					(10)I dislike reading books about science or math during my free time*
					(13)I would like to do science experiments or math problems at home.
					(16)Talking to friends about science or math after school would be boring.*
					(19)I would enjoy having a job in a science laboratory during my summer vacation
					(22)Watching a video about science or math would be boring.*
					(25)I would enjoy visiting a science museum on the weekend.
					(28)I dislike looking at websites about science or math.*

Lott (2002): Used a modified version of all 3 of these scales in a study of 224 HS chemistry students. Alphas included above.

Ricks (2006) used Career Interest in Science, Enjoyment of science lessons, one additional TOSRA scale and 3 mATSI scales. She reported an overall alpha of 0.90.

Appendix F: Parent Consent Form

National Evaluation of NASA’s Summer of Innovation

NASA Office of Education is initiating a 3-year program called Summer of Innovation for middle school students. Two research companies—Abt Associates, and Education Development Center—were hired to do the national evaluation. The evaluation will include more than 12,000 students across the country.

NASA Office of Education is asking students participating in the Summer of Innovation program to complete a brief voluntary survey. We will ask students in the study to fill out a yearly survey for the next two years. The surveys will help us improve the Summer of Innovation program and we greatly value the feedback. All information collected will be used only for research purposes.

What it Means to Participate

We need students to complete the survey and participate in this study because it will help identify ways to improve NASA’s Summer of Innovation programs and other similar programs. However, your child’s participation is voluntary. Whether your child is part of the study will not affect whether he or she is eligible for any program or service. If you give permission, your child will be asked to fill out two brief surveys this summer, then a yearly survey for the next two years. Your child does not have to answer any question he or she does not want to.

Privacy

Protecting your child’s privacy is very important to us. NASA Office of Education and the companies doing the study will follow strict rules to protect your child’s privacy. Your child’s name, and the name of your child’s school, or the name of your child’s Summer of Innovation program, will not appear in any reports produced for this study. Your child’s answers and records will be kept private from Summer of Innovation staff, NASA, your child’s school staff, friends, and family. Your child’s name will be replaced with an identification number for the purpose of this study. Any information on your child entered into a database will be stored on a secure computer network. Paper copies of your child’s survey responses will also be stored securely. We will not provide information that identifies your child to anyone outside the study team, except as required by law.

As with all data collection, there is a very small chance that someone will see your child’s survey answers without permission. NASA Office of Education and the study team have many procedures in place to protect the privacy of the data collected on your child, so we do not think this will happen. However, if it does, all NASA Office of Education procedures will be followed to correct the situation.

Questions About the Study

If you have any questions about the survey, please call 1-877- 520-6840 (toll free), or email at [email protected]. If you have questions about the study, please call Dr. Hilary Rhodes, Study Director, at 617-520-3516 (toll call). . If you have any questions about subjects’ rights, please contact Abt’s Institutional Review Board Administrator, Dr. Teresa Doksum (617) 349-2896 (toll call).

1. Please read the following statement. NASA Summer of Innovation program to which your child is applying is participating in a national study. This NASA study is being conducted by Abt Associates and Education Development Center. The purpose of the study is to learn how Summer of Innovation helps students learn and engage in science, technology, engineering, and math. We would like your child to participate in this important study. For your child to participate, we ask your permission to collect survey information from your child this summer, and yearly for the next two years. The information gathered on your child will be kept strictly confidential to the extent allowed by law and will be used ONLY for the purpose of the study. 2. After reading the above statement, do you give permission for your child to participate in the National Evaluation of NASA’s Summer of Innovation?
	YES, my child _____________________________________________________________________
	First Name Last Name has my permission to participate in the national evaluation of NASA’s Summer of Innovation.
	NO, my child _____________________________________________________________________
	First Name Last Name does NOT have my permission to participate in the national evaluation of NASA’s Summer of Innovation.
Print Your Name: First Name Last Name
Signature: Date:
Your Telephone Number: ( ) ____________ - _____________________ Area Code Your E-mail Address: _________________________________________________

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless such collection displays a valid OMB control number The valid OMB control number for this information collection will be entered after clearance. The time required to complete this information collection is estimated to average 15 minutes per response, including the time to review instructions, search existing data resources, gather the data needed, and complete and review the information collection.

Appendix G: Student Assent Form

National Evaluation of NASA’s Summer of Innovation

NASA Office of Education is initiating a 3-year program called Summer of Innovation for middle school students. Researchers from Abt Associates and Education Development Center are doing the national study. The study will include more than 12,000 students across the country.

NASA Office of Education wants students in the Summer of Innovation program to complete a brief survey. We will ask you to fill out a short survey once for the next two years. If you are part of the study, you will help us learn ways in which to improve programs to help students learn and like science and math. However, whether you are part of the study is up to you.

What it Means to Be in the Study

Doing the survey is up to you, it is voluntary. You can still be part of the Summer of Innovation, whether you do the survey or not. Doing the survey will help us improve the program and help us learn how kids like you learn and like science and math. You do not have to answer any questions you don’t want to.

Privacy

NASA Office of Education, and the researchers at Abt Associates and Education Development Center, follow strict rules to protect your privacy and the privacy of any information you give us. No report will describe you in any way that could identify you. Your answers will be kept private from Summer of Innovation staff, your school staff, friends, and your family. No one will see your answers to this survey or future surveys besides trained members of the study team, except as required by law.

Questions About the Study

If you have any questions about the survey, please call 1-877-520-6840 (toll free), or email at [email protected]. If you have questions about the study, please call Dr. Hilary Rhodes, Study Director, at 617-520-3516 (toll call). . If you have any questions about subjects’ rights, please contact Abt’s Institutional Review Board Administrator, Dr. Teresa Doksum (617) 349-2896 (toll call).

1. Please read the following statement. The NASA Summer of Innovation program to which I am applying is part of a national study. This NASA Summer of Innovation study is being conducted by Abt Associates and Education Development Center. The purpose of the study is to learn how Summer of Innovation helps students learn and engage in science, technology, engineering, and math. We would like you to participate in this important study. If you participate, we will collect survey information this summer, and yearly for the next two years. The information gathered on you will be kept strictly confidential to the extent allowed by law and will be used ONLY for the purpose of the study. 2. After reading the above statement, do you agree to participate in the National Evaluation of NASA’s Summer of Innovation?
	YES, I agree to participate in the National Evaluation of NASA’s Summer of Innovation.
	NO, I do NOT agree to participate in the National Evaluation of NASA’s Summer of Innovation.
Print Your Name: First Name Last Name
Signature: Date:

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless such collection displays a valid OMB control number The valid OMB control number for this information collection will be entered after clearance. The time required to complete this information collection is estimated to average 15 minutes per response, including the time to review instructions, search existing data resources, gather the data needed, and complete and review the information collection.

Appendix H: Teacher Consent Form

National Evaluation of NASA’s Summer of Innovation

NASA Office of Education is initiating a 3-year program called Summer of Innovation for middle school students and their teachers. Two research companies—Abt Associates, and Education Development Center—were hired to do the national evaluation. The evaluation will include more than 600 teachers across the country.

NASA Office of Education is asking teachers participating in the Summer of Innovation program to complete a brief survey. The surveys will help us improve the Summer of Innovation program and we greatly value the feedback. All information collected will be used only for research purposes.

What it Means to Participate

We need teachers to complete the survey and participate in this study because it will help identify ways to improve NASA’s Summer of Innovation programs and other similar programs. However, your participation is voluntary. Whether you are part of the study will not affect whether you are eligible for any program or service. If you give permission, you will be asked to fill out two brief surveys this summer. You do not have to answer any question you choose.

Privacy

Protecting your privacy is very important to us. NASA Office of Education and the companies doing the study will follow strict rules to protect your privacy. Your name, and the name of your school, or the name of your Summer of Innovation program, will not appear in any reports produced for this study. Your answers and records will be kept private from Summer of Innovation staff, NASA, your school staff, friends, and family. Your name will be replaced with an identification number for the purpose of this study. Any information that you entered will be stored on a secure computer network. Paper copies of your survey responses will also be stored securely. We will not provide information that identifies you to anyone outside the study team, except as required by law.

As with all data collection, there is a very small chance that someone will see your survey answers without permission. NASA Office of Education and the study team have many procedures in place to protect the privacy of the data collected, so we do not think this will happen. However, if it does, all NASA Office of Education procedures will be followed to correct the situation.

Questions About the Study

If you have any questions about the survey, please call 1-877- 520-6840 (toll free), or email at [email protected]. If you have questions about the study, please call Dr. Hilary Rhodes, Study Director, at 617-520-3516 (toll call). . If you have any questions about subjects’ rights, please contact Abt’s Institutional Review Board Administrator, Dr. Teresa Doksum (617) 349-2896 (toll call).

1. Please read the following statement. The NASA Summer of Innovation program to which I am applying is part of a national study. This NASA Summer of Innovation study is being conducted by Abt Associates and Education Development Center. The purpose of the study is to learn how Summer of Innovation helps students learn and engage in science, technology, engineering, and math. We would like you to participate in this important study. If you participate, we will collect survey information this summer. The information gathered on you will be kept strictly confidential to the extent allowed by law and will be used ONLY for the purpose of the study. 2. After reading the above statement, do you agree to participate in the National Evaluation of NASA’s Summer of Innovation?
	YES, I agree to participate in the National Evaluation of NASA’s Summer of Innovation.
	NO, I do NOT agree to participate in the National Evaluation of NASA’s Summer of Innovation.
Print Your Name: First Name Last Name
Signature: Date:

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless such collection displays a valid OMB control number The valid OMB control number for this information collection will be entered after clearance. The time required to complete this information collection is estimated to average 15 minutes per response, including the time to review instructions, search existing data resources, gather the data needed, and complete and review the information collection.