ED School Climate Surveys (EDSCLS) Additional Item Cognitive Interviews - Set 2 Round 1

Table of Contents

Protocols for Student Survey 3

Protocols for Instructional and Non-Instructional Staff Survey 9

Student Survey

STEP 1: WELCOME AND INTRODUCTION SCRIPT

Interviewer Instructions: The following scripts should NOT be read verbatim. Text in italics is suggested content with which the interviewer should be thoroughly familiar in advance. Text in brackets [ ] presents instructions for the interviewer.

Hello, my name is <your name > and I work for Research Support Services. It’s nice to meet you, thanks for coming to help us out today.

[Create small talk to build rapport with the student]:

You are here today to help us out with a set of new survey questions that we are working on with the U.S. Department of Education. This survey asks about your opinions of the school you currently attend. A survey is just a set of questions about your experiences. This is not a test. I will ask you to take the survey as if you were taking a real survey. There are no right or wrong answers, and it’s ok if you do not know some of the answers. I will not be grading your work. Once you are done I will ask you questions about the survey. This will help us make the survey better and clearer for other students just like you.

You can choose whether or not you want to participate. You may also skip any question you do not want to answer. [We have discussed this research with your parent(s)/guardian and they know that we are also asking you for your agreement. If you are going to participate in the research, your parent(s)/guardian also have to agree. But if you do not wish to take part in the research, you do not have to do so, even if your parents have agreed.] Because the information you provide is so important, I am going to be taking notes while you are working and while we are talking [INSERT IF THERE IS AN OBSERVER: and my colleague will also be observing our session today].

Also, this interview will be recorded so researchers can review the tapes later. Your responses and any information that refers to you or the name of your school will not be included in our report. Your responses will be used for research purposes only and will not be disclosed, or used for any other purpose except as required by law. [if ask: Education Sciences Reform Act of 2002, 20 U.S. Code, Section 9573]

Remember, you are not being tested or graded; we want you to tell us what you think about the survey. Would you like to be a part of this study? [If respondent agrees, continue, if not, end the interview.]

If at any time you want to stop, just let me know.

[If, for any reason, the participant is no longer interested in participating, thank the participant for his/her time and end the interview.]

Before we continue, do you have any questions about what I just said? [Answer any questions the student may ask.]

PROVIDE RESPONDENT WITH A COPY OF THE ASSENT FORM. ASK THE RESPONDENT TO READ THE FORM (OR READ IT TO HIM/HER, DEPENDING ON RESPONDENT PREFERENCE), ANSWER ANY QUESTIONS, AND HAVE THE RESPONDENT SIGN THE FORM. LEAVE A SEPARATE COPY OF THE FORM WITH THE RESPONDENT.

• SIGNED ASSENT FORM COLLECTED

• COPY OF ASSENT FORM GIVEN TO RESPONDENT

• IF THE RESPONDENT HAS ASSENTED TO RECORDING, START THE RECORDER.

STEP 2: THINK-ALOUD PRACTICE

While you fill out the survey I want you to think out loud about how you are answering some of the questions. Hearing you talk about how you figure out your answers to the questions will help me understand how to make the questions better. I am more interested in how you arrive at your answers than in the answers you give me. To help you get started on thinking aloud, we are going to do a practice activity. I’m going to ask you a question and ask you to think aloud as you decide on your answer.

How many windows are there in the house or apartment where you live?

Please tell me what you are thinking while you are figuring out your answer.

Probe as needed for detail: How did you arrive at that answer? What were you visualizing when you were deciding on that answer? Can you tell me more about what you are including in your count?

When complete: Thank you; that’s the kind of detail I’m looking for when you talk to me about your answers to the questions in the survey.

HAND RESPONDENT A COPY OF THE QUESTIONNAIRE AND A PEN.

STEP 3: COMPLETION OF THE QUESTIONNAIRE

Now I would like you to complete the questionnaire. While you are answering the questions, please think about the school you currently attend. I’d like you to think aloud while you decide on your answer. Please read aloud anything that you would normally read to yourself if I were not here.

Remember that I am very interested in hearing you talk about what you are thinking about as you figure out your answers to the questions.

Say to respondent: Please answer these questions thinking about the school you are currently attending right now

Throughout the survey, when a question says "at this school," it refers to activities happening in school buildings, on school grounds, on school buses, and at places that hold school-sponsored events or activities. Unless otherwise specified, this refers to normal school hours or to times when school activities/events are in session.

How strongly do you agree or disagree with the following statements about this school? Mark one response.

1. It is easy to talk with teachers at this school

   • Strongly Agree

   • Agree

   • Disagree

   • Strongly Disagree

PROBES

Tell me more about your answer.

In your own words, what do you think this question is asking?

2. There are lots of chances for students at this school to get involved in sports, clubs, and other school activities outside of class.

   • Strongly Agree

   • Agree

   • Disagree

   • Strongly Disagree

PROBES

Tell me more about your answer.

3. Students at this school get along well with each other.

   • Strongly Agree

   • Agree

   • Disagree

   • Strongly Disagree

PROBES

Tell me more about why you [ANSWER TO Q3]

What do you think we mean when we say “get along well with each other?

4. Students at this school often make sexual comments, jokes, or gestures to others who do not want to hear or see them. This could happen in school or outside of school and it could be in person or by text, e-mail, or social media.

   • Strongly Agree

   • Agree

   • Disagree

   • Strongly Disagree

PROBES

Tell me more about your answer.

What do you think we mean when we say “sexual comments, jokes, or gestures”? Is there another word for “gestures”/ “sexual gestures” you would use?

This question asked about students who make sexual comments, jokes or gestures to “others”. When you read the term “others”, who were you thinking of (PROBE FOR WHETHER THEY WERE JUST INCLUDING OTHER STUDENTS AT THE SCHOOL OR OTHER KIDS AND ADULTS AS WELL)?

5. Students often spread sexual rumors or share sexual photos/videos of people who do not want them shared. This could happen in school or outside of school and it could be in person or by text, e-mail, or social media.

   • Strongly Agree

   • Agree

   • Disagree

   • Strongly Disagree

PROBES

Tell me more about your answer.

Can you give me examples of the types of things you were thinking about? [PROBE IF NECESSARY: When answering this question did you think about things that happen “in person or by text, e-mail, or social media”?]

What do you think they mean by “spread sexual rumors” in this question?

6. Students often get sent or shown sexual photos/videos that they do not want to see. This could happen in school or outside of school and it could be in person or by text, e-mail, or social media

   • Strongly Agree

   • Agree

   • Disagree

   • Strongly Disagree

PROBES

Tell me more about your answer.

In your own words, what do they mean here by “sexual photos or videos”?

Are there other ways, besides “in-person or by text, e-mail, or social media” that a student could be sent or shown sexual photos/videos that they do not want to see?

7. At this school, students often have their sexual body parts touched or grabbed (for example, forced kissing, touching of their butt, crotch, or breasts) when they do not agree to it and do not want it to happen. The person who does it could be another student, a teacher, or someone else.

   • Strongly Agree

   • Agree

   • Disagree

   • Strongly Disagree

PROBES

In your own words what does, “have their sexual body parts touched or grabbed” mean?

What do you think they mean in this question by “sexual body parts”? Would you prefer the term “private body parts”? Why (not)?

Without looking at the question now, do you remember if the questions mentions explicitly who could be doing the touching or grabbing? [PROBE IF NEEDED: Did you read or notice here where it says that “the person who does it could be another student, a teacher, or someone else”?]

8. At this school, students often have their sexual body parts touched or grabbed (e.g., forced kissing, touching of their butt, crotch, or breasts) when they cannot make a decision about whether they agree to it or want it to happen because they have been using drugs or alcohol.

   • Strongly Agree

   • Agree

   • Disagree

   • Strongly Disagree

PROBES

Tell me more about your answer.

In this question, what do you think they mean by “they cannot make a decision about whether they agree to it or want it to happen”?

9. Students would go to a teacher or some other adult for help if they had their sexual body parts touched or grabbed (e.g., forced kissing, touching of their butt, crotch, or breasts) when they did not agree to it and did not want it to happen.

   • Strongly Agree

   • Agree

   • Disagree

   • Strongly Disagree

PROBES

How did you come up with your answer?

What does “some other adult” mean to you? Who would you include in this definition?

10. Students would go to a teacher or some other adult for help if others were making sexual comments, jokes, or gestures to them, or sending them sexual photos/videos that they did not want to see, or spreading sexual rumors or sexual photos/videos about them to others.

    • Strongly Agree

    • Agree

    • Disagree

    • Strongly Disagree

PROBES

Tell me more about your answer.

Can you give me some examples of situations where someone“goes to a teacher or some other adult for help”?

11. Students would go to a teacher or some other adult for help if someone they were dating threatened to hurt them, or hit, pushed, or kicked them.

    • Strongly Agree

    • Agree

    • Disagree

    • Strongly Disagree

PROBES

In your own words, who do you think “someone they were dating” includes?

How did you feel about answering this question?

12. Students would go to a teacher or some other adult for help if someone they were dating tried to control them (for example, kept them from talking to their friends or family, or kept track of them at all times).

    • Strongly Agree

    • Agree

    • Disagree

    • Strongly Disagree

PROBES

What do you think they mean when they say someone they were dating “tried to control them”?

Tell me more about how you came up with your answer.

OVERALL PROBES

1. Overall, what are your thoughts on this survey?

2. How easy or difficult do you think it would be for other students to respond to this survey?

3. How helpful did you find the examples provided in some of the questions? [REFER TO A QUESTION WITH AN EXAMPLE, IF NECESSARY]

4. (Other than anything we may have already talked about,) Were there any terms or definitions you did not understand?

5. Were there any questions that seemed strange to you?

6. Were there any questions that made you uncomfortable?

7. If you had the opportunity, what would you do to improve this survey?

8. Do you have any other thoughts, comments or suggestions about the survey?

Those are all of the questions I have for you today. Thank you very much for taking the time to talk to me and for providing your valuable input.

STEP 4. THANK YOU FOR PARTICIPATING

We are finished. Thank you for helping us. The information you’ve provided will be very helpful for making this survey better.

Interviewer Instructions: Collect materials and escort student out of the interviewing room and back to his or her parent. Please give the student an incentive [$25], the parent an incentive [$25] if they brought their child to and from the interview, and thank them for time and effort.

Instructional/Non-instructional Staff Survey

STEP 1: WELCOME AND INTRODUCTION SCRIPT

Interviewer Instructions: The following scripts should NOT be read verbatim. Text in italics is suggested content with which the interviewer should be thoroughly familiar in advance. Text in brackets [ ] presents instructions for the interviewer.

Hello, my name is <your name > and I work for Research Support Services, Inc. It’s nice to meet you, thank you for coming to help us out today.

[Create small talk to build rapport with the teacher/Admin]:

You are here today to help us out with new survey questions that we are working on with the U.S. Department of Education. This survey asks about your opinions of the school where you work: [school name]. I will ask you to take the survey as if you were taking a real survey. There are no right or wrong answers, and it’s ok if you do not know some of the answers. As you are answering the survey questions I will ask you about them. I will also stop you once in a while to ask you questions. This will help us make the survey better and clearer for other teachers and school instructional staff just like you.

Because the information you provide is so important, I am going to be taking notes while you are working and while we are talking [INSERT IF THERE IS AN OBSERVER: and my colleague will also be observing our session today].

Also, this interview will be recorded so researchers can review the tapes later. Your responses and any information that refers to you or the name of your school will not be included in our report. Your responses will be used for research purposes only and will not be disclosed, or used for any other purpose except as required by law. [if ask: Education Sciences Reform Act of 2002, 20 U.S. Code, Section 9573]

Remember, in this study we want you to tell us what you think about the survey. Would you like to be a part of this study? [If respondent agrees, continue, if not, end the interview.]

If at any time you want to stop, just let me know.

[If, for any reason, the participant is no longer interested in participating, thank the participant for his/her time and end the interview.]

Before we continue, do you have any questions about what I just said? [Answer any questions the participant may ask.]

PROVIDE RESPONDENT WITH A COPY OF THE INFORMED CONSENT FORM. ASK THE RESPONDENT TO READ THE FORM (OR READ IT TO HIM/HER, DEPENDING ON RESPONDENT PREFERENCE), ANSWER ANY QUESTIONS, AND HAVE THE RESPONDENT SIGN THE FORM. LEAVE A SEPARATE COPY OF THE FORM WITH THE RESPONDENT.

• SIGNED CONSENT FORM COLLECTED

• COPY OF CONSENT FORM GIVEN TO RESPONDENT

• IF THE RESPONDENT HAS CONSENTED TO RECORDING, START THE RECORDER.

STEP 2: THINK-ALOUD PRACTICE

While you fill out the survey I want you to think out loud about how you are answering some of the questions. Hearing you talk about how you figure out your answers to the questions will help me understand how to make the questions better. I am more interested in how you arrive at your answers than in the answers you give me. To help you get started on thinking aloud, we are going to do a practice activity. I’m going to ask you a question and ask you to think aloud as you decide on your answer.

How many windows are there in the house or apartment where you live?

Please tell me what you are thinking while you are figuring out your answer.

Probe as needed for detail: How did you arrive at that answer? What were you visualizing when you were deciding on that answer? Can you tell me more about what you are including in your count?

When complete: Thank you; that’s the kind of detail I’m looking for when you talk to me about your answers to the questions in the survey.

HAND RESPONDENT A COPY OF THE QUESTIONNAIRE AND A PEN.

STEP 3: COMPLETION OF THE QUESTIONNAIRE

Now I would like you to complete the questionnaire. While you are answering the questions, please think about the school you currently work at: [school name]. I’d like you to think aloud while you decide on your answer. Please read aloud anything you would have read to yourself if I were not here.

Remember that I am very interested in hearing you talk about what you are thinking about as you figure out your answers to the questions.

Throughout the survey, when a question says "at this school," it refers to activities happening in school buildings, on school grounds, on school buses, and at places that hold school-sponsored events or activities. Unless otherwise specified, this refers to normal school hours or to times when school activities/events are in session.

How strongly do you agree or disagree with the following statements about this school? Mark One Response

1. I feel safe at this school.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

PROBES

Tell me more about your answer.

What do you think they mean by 'feeling safe' in this question?

2. The following types of problems occur often: physical conflicts among students.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

PROBES

What do you think “physical conflicts” means in this question?

Tell me more about your answer.

3. The following types of problems occur often: students making sexual comments, jokes, or gestures to others who do not want to hear or see. This could happen in school or outside of school, and it could be in person or by text, e-mail, or social media.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

PROBES

What do you think we mean when we say “sexual comments, jokes, or gestures”? Is there another word for “gestures”/ “sexual gestures” you would use?

When you read the term “others”, what groups were you thinking of (probe for whether they were just including other students at the school or other kids and adults as well)?

How did you decide what to answer?

4. The following types of problems occur often: students spreading sexual rumors or sharing sexual photos/videos of people who do not want them shared. This could happen in school or outside of school, and it could be in person or by text, e-mail, or social media.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

PROBES

When answering this question did you think about things that happen “in person or by text, by e-mail, or on social media,” or were you thinking of something else?

What does “spread sexual rumors” mean to you as it was used in this question?

5. The following types of problems occur often: students getting sent or shown sexual photos/videos that they do not want to see. This could happen in school or outside of school, or it could be in person or by text, e-mail, or social media.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

PROBES

In your own words, what does the phrase “sexual photos or videos” mean?

Are there other ways, besides “in-person or by text, e-mail, or social media” that a student could be sent or shown sexual photos/videos that they do not want to see?

4. The following types of problems occur often: students being threatened, hit, pushed, or kicked by someone they are dating. This could happen in school or outside of school.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

PROBES

In your own words, who do you think “someone they were dating” includes?

How did you feel about answering this question?

4. The following types of problems occur often: students having someone they are dating try to control them (for example, keeping them from talking to friends or family, keeping track of them at all times). This could happen in school or outside of school.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

PROBES

What do you think they mean in this question when they say someone they were dating “tried to control them”?

How did you decide what to answer?

6. The following types of problems occur at this school often: students having their sexual body parts touched or grabbed (e.g., forced kissing, touching of their butt, crotch, or breasts) when they do not agree to it and do not want it to happen.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

PROBES

What do you think we mean when we say “sexual body parts”? Would you prefer here the term “private body parts” ? Why (not)?

In your own words what does, “have their sexual body parts touched or grabbed” mean?

6. The following types of problems occur at this school often: students having their sexual body parts touched or grabbed (e.g., forced kissing, touching of their butt, crotch, or breasts) when they cannot make a decision about whether they agree to it or want it to happen because they have been using drugs or alcohol.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

PROBES

How did you select your answer to this question?

What do you think we mean when we say “they cannot make a decision about whether they agree to it or want it to happen”?

6. The following types of problems occur at this school often: students being sexually penetrated (e.g., intercourse, oral sex, anal sex, or other form of penetration) when they do not agree to it and do not want it to happen.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

PROBES

In your own words, what is this question asking?

Tell me more about your answer.

6. Outside of school/school-sponsored events, the following types of problems occur often: students having their sexual body parts touched or grabbed (e.g., forced kissing, touching of their butt, crotch, or breasts) when they do not agree to it and do not want it to happen.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

PROBES

What do you think they are referring to in this question when they say “outside of school/school-sponsored events”?

How did you feel about answering this question?

6. Outside of school/school-sponsored events, the following types of problems occur often: students being sexually penetrated (e.g., intercourse, oral sex, anal sex, or other form of penetration) when they do not agree to it and do not want it to happen.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

PROBES

Tell me more about your answer.

11. This school has effective resources for helping students who have experienced sexual contact (including forced kissing/touching of sexual body parts or sexual penetration such as intercourse, oral sex, or anal sex) that they did not agree to and did not want to happen.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

PROBES

What does the term “resources” include in this question?

What does “sexual contact” mean to you?

11. This school has effective resources for helping students who have had unwanted sexual comments, jokes, or gestures made to them, sexual photos/videos sent to them, or sexual rumors or sexual photos/videos spread about them.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

PROBES

What do you think they mean here by “effective resources”?

11. This school has effective resources for helping students who have had someone they were dating threaten to hurt, hit, push, kick, or try to control them (e.g., keep them from talking to their friends or family, keeping track of them at all times).

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

PROBES

Tell me more about your answer.

OVERALL PROBES

1. Overall, what are your thoughts on this survey?

2. How easy or difficult do you think it would be for other staff to respond to this survey?

3. Did you find the examples provided in some of the questions very helpful/helpful/somewhat helpful/not at all helpful?

4. (Other than anything we may have already talked about,) Were there any terms or definitions you did not understand?

5. Were there any questions that made you uncomfortable? IF YES: Tell me why.

6. If you had the opportunity, what would you do to improve this survey?

7. Do you have any other thoughts, comments or suggestions about the survey?

Those are all of the questions I have for you today. Thank you very much for taking the time to talk to me and for providing your valuable input.

STEP 4. THANK YOU FOR PARTICIPATING

We are finished. Thank you for helping us. The information you’ve provided will be very helpful for making this survey better.

Interviewer Instructions: Collect materials. Please give the respondent an incentive [$40].