ED School Climate Surveys (EDSCLS) Additional Item Cognitive Interviews – Set 2 Round 2

Table of Contents

Protocols for Student Survey 3

Protocols for Instructional and Non-Instructional Staff Survey 9

Student Survey

STEP 1: WELCOME AND INTRODUCTION SCRIPT

Interviewer Instructions: The following scripts should NOT be read verbatim. Text in italics is suggested content with which the interviewer should be thoroughly familiar in advance. Text in brackets [ ] presents instructions for the interviewer.

Hello, my name is <your name > and I work for Research Support Services. It’s nice to meet you, thanks for coming to help us out today.

[Create small talk to build rapport with the student]:

You are here today to give us your feedback and opinions on a set of new survey questions that we are working on with the U.S. Department of Education. The purpose of our conversation is to show you a survey with questions about your school and experiences as a student. I will ask you to take the survey as if you were taking a real survey. There are no right or wrong answers, so please answer these questions to the best of your knowledge. After some of the survey questions I may ask you follow-up questions to better understand your answer and the way you thought about the question. As we are going through the survey, please feel free to tell me anything that comes to mind or to ask me anything you are unclear about. We want your honest opinions about what you like and dislike and what you do and don’t understand so we can improve this survey. Once you have completed the survey, I will ask you a few follow-up questions about your overall thoughts on the survey.

The survey asks about student life, dating, and sexual activity. Because of the sensitive content, some of the questions might make you feel uncomfortable. You can choose whether or not you want to participate. You may also skip any question you do not want to answer. [We have discussed this research with your parent(s)/guardian and they know that we are also asking you for your agreement. If you are going to participate in the research, your parent(s)/guardian also have to agree. But if you do not wish to take part in the research, you do not have to do so, even if your parents have agreed.] Because the information you provide is so important, I am going to be taking notes while you are working and while we are talking [INSERT IF THERE IS AN OBSERVER: and my colleague will also be observing our session today].

Also, this interview will be recorded so researchers can review the tapes later. Your responses and any information that refers to you or the name of your school will not be included in our report. Your responses will be used for research purposes only and will not be disclosed, or used for any other purpose except as required by law. [if ask: Education Sciences Reform Act of 2002, 20 U.S. Code, Section 9573]

Remember, there are no right or wrong answers, instead we are more focused on what you think about the survey questions themselves. Would you like to be a part of this study? [If respondent agrees, continue, if not, end the interview.]

If at any time you want to stop, just let me know.

[If, for any reason, the participant is no longer interested in participating, thank the participant for his/her time and end the interview.]

Before we continue, do you have any questions about what I just said? [Answer any questions the student may ask.]

PROVIDE RESPONDENT WITH A COPY OF THE ASSENT FORM. ASK THE RESPONDENT TO READ THE FORM (OR READ IT TO HIM/HER, DEPENDING ON RESPONDENT PREFERENCE), ANSWER ANY QUESTIONS, AND HAVE THE RESPONDENT SIGN THE FORM. LEAVE A SEPARATE COPY OF THE FORM WITH THE RESPONDENT.

• SIGNED ASSENT FORM COLLECTED

• COPY OF ASSENT FORM GIVEN TO RESPONDENT

• IF THE RESPONDENT HAS ASSENTED TO RECORDING, START THE RECORDER.

STEP 2: THINK-ALOUD PRACTICE

While you fill out the survey I want you to think out loud about how you are answering some of the questions. Hearing you talk about how you figure out your answers to the questions will help me understand how to make the questions better. I am more interested in how you arrive at your answers than in the answers you give me. To help you get started on thinking aloud, we are going to do a practice activity. Please read the following questions and try to think aloud as you decide on your answer before marking your answer. Then I will ask you some follow-up questions about your thoughts and opinions.

How strongly do you agree or disagree with the following statements about this school? Please think about the school you currently attend. Mark one response.

1. It is easy to talk with teachers at this school

   • Strongly Agree

   • Agree

   • Disagree

   • Strongly Disagree

PROBES

How easy or difficult was it for you to answer this question?

2. Students at this school get along well with each other.

   • Strongly Agree

   • Agree

   • Disagree

   • Strongly Disagree

PROBES

In your own words, what is this question asking?

How did you come up with your answer to this question?

STEP 3: COMPLETION OF THE QUESTIONNAIRE

Now I would like you to complete the questionnaire. While you are answering the questions, please think about the school you currently attend.

Remember that I am very interested in hearing you talk about what you are thinking about as you figure out your answers to the questions.

HAND RESPONDENT A COPY OF THE QUESTIONNAIRE AND A PEN.

These questions ask how strongly you agree or disagree that the following things happen among students at this school.

1. At school, students make unwanted sexual comments, sexual jokes, sexual hand or body gestures; or spread unwanted sexual rumors, photos, or videos to other people.

This question is asking about things that the other person did not want to hear or see. “At school” includes school buses, school buildings, or school sponsored events. These things could happen in person, or by text, social media, or other forms of communication.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

PROBES

What do you think we mean when we say “sexual comments, sexual jokes, or sexual hand or body gestures”?

How easy or difficult was it to answer this question?

If needed: How did you come up with your answer?

2. Outside of school or outside of any school-sponsored events, students make unwanted sexual comments, sexual jokes, sexual hand or body gestures; or spread unwanted sexual rumors, photos, or videos to other people.

This question is asking about things that the other person did not want to hear or see. This asks about behaviors that happen outside of school or outside of any school sponsored events. These things could happen in person, or by text, social media, or other forms of communication.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

PROBES

Did you read or notice the difference between this question and the previous question? If YES: what was the difference between the two questions?

How easy or difficult was it to answer this question?

If needed: How did you come up with your answer?

3. At school, students have their private body parts touched or grabbed (e.g., touching of their butt, crotch, or breasts, or experiencing forced kissing) when they do not agree to it and do not want it to happen.

“At school” includes school buses, school buildings, or at school sponsored events. The person who does it could be another student, a teacher, or someone else.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

PROBES

In your own words what is this question asking?

Did you read or notice here where it says that “the person who does it could be another student, a teacher, or someone else”?

4. Outside of school or outside of any school sponsored events, students have their private body parts touched or grabbed (e.g., touching of their butt, crotch, or breasts, or experiencing forced kissing) when they do not agree to it and do not want it to happen.

This asks about behaviors that happen outside of school or outside of any school sponsored events. The person who does it could be another student, a teacher, or someone else.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

PROBES

Did you read or notice the difference between this question and the previous question? If YES: what was the difference between the two questions?

How easy or difficult was it to answer this question?

If needed: How did you come up with your answer?

The next questions ask how strongly you agree or disagree that students at this school would go to a teacher or some other adult for help in various situations.

5. Students would go to a teacher or some other adult for help if they had their private body parts touched or grabbed (e.g., touching of their butt, crotch, or breasts, or experiencing forced kissing) when they did not agree to it and did not want it to happen.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

PROBES

What kind of people would you include when thinking about “some other adult”?

Did you find the examples provided here helpful or not helpful?

How did you come up with your answer?

6. Students would go to a teacher or some other adult for help someone was making unwanted sexual comments, sexual jokes or sexual hand or body gestures to them; sending them sexual photos or videos that they did not want to see; or spreading unwanted sexual rumors, photos, or videos about them to other people.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

PROBES

In your own words, what does it mean for a student to “go to a teacher or some other adult for help”?

If needed: Tell me more about your answer.

7. Students would go to a teacher or some other adult for help if someone they were dating threatened to hurt them, or hit, pushed, or kicked them.

   • Strongly Agree

   • Agree

   • Disagree

   • Strongly Disagree

PROBES

In your own words, what is this question asking?

How did you feel about this question?

8. Students would go to a teacher or some other adult for help if someone they were dating tried to control them (for example, kept them from talking to their friends or family, or kept track of them at all times).

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

PROBES

What do you think we mean when we say someone the student was dating “tried to control them”?

Did you find the examples provided here helpful or not helpful?

Are there other ways in which someone could control someone else that is not listed here?

How easy or difficult was it to answer this question?

OVERALL PROBES

1. Overall, what are your thoughts on this survey?

2. How easy or difficult do you think it would be for other students to respond to this survey?

3. (Other than anything we may have already talked about,) Were there any terms or definitions you did not understand?

4. Were there any questions that seemed strange to you?

5. Were there any questions that made you uncomfortable?

6. If you had the opportunity, what would you do to improve this survey?

7. Do you have any other thoughts, comments or suggestions about the survey?

Those are all of the questions I have for you today. Thank you very much for taking the time to talk to me and for providing your valuable input.

STEP 4. THANK YOU FOR PARTICIPATING

We are finished. Thank you for helping us. The information you’ve provided will be very helpful for making this survey better.

Interviewer Instructions: Collect materials and escort student out of the interviewing room and back to his or her parent. Please give the student an incentive [$25], the parent an incentive [$25] if they brought their child to and from the interview, and thank them for time and effort.

Instructional/Non-instructional Staff Survey

STEP 1: WELCOME AND INTRODUCTION SCRIPT

Interviewer Instructions: The following scripts should NOT be read verbatim. Text in italics is suggested content with which the interviewer should be thoroughly familiar in advance. Text in brackets [ ] presents instructions for the interviewer.

Hello, my name is <your name > and I work for Research Support Services, Inc. It’s nice to meet you, thank you for coming to help us out today.

[Create small talk to build rapport with the teacher/Admin]:

You are here today to give us your feedback and opinions on a set of new survey questions that we are working on with the U.S. Department of Education. The purpose of our conversation is to show you a survey with questions about the school where you work, [school name]. I will ask you to take the survey as if you were taking a real survey. There are no right or wrong answers, so please answer these questions to the best of your knowledge. After some of the survey questions I may ask you follow-up questions to better understand your answer and the way you thought about the question. As we are going through the survey, please feel free to tell me anything that comes to mind or to ask me anything you are unclear about. We want your honest opinions about what you like and dislike and what you do and don’t understand so we can improve this survey. Once you have completed the survey, I will ask you a few follow-up questions about your overall thoughts on the survey.

Because the information you provide is so important, I am going to be taking notes while you are working and while we are talking [INSERT IF THERE IS AN OBSERVER: and my colleague will also be observing our session today].

Also, this interview will be recorded so researchers can review the tapes later. Your responses and any information that refers to you or the name of your school will not be included in our report. Your responses will be used for research purposes only and will not be disclosed, or used for any other purpose except as required by law. [if ask: Education Sciences Reform Act of 2002, 20 U.S. Code, Section 9573]

Remember, there are no right or wrong answers, instead we are more focused on what you think about the survey questions themselves. Would you like to be a part of this study? [If respondent agrees, continue, if not, end the interview.]

If at any time you want to stop, just let me know.

[If, for any reason, the participant is no longer interested in participating, thank the participant for his/her time and end the interview.]

Before we continue, do you have any questions about what I just said? [Answer any questions the participant may ask.]

PROVIDE RESPONDENT WITH A COPY OF THE INFORMED CONSENT FORM. ASK THE RESPONDENT TO READ THE FORM (OR READ IT TO HIM/HER, DEPENDING ON RESPONDENT PREFERENCE), ANSWER ANY QUESTIONS, AND HAVE THE RESPONDENT SIGN THE FORM. LEAVE A SEPARATE COPY OF THE FORM WITH THE RESPONDENT.

• SIGNED CONSENT FORM COLLECTED

• COPY OF CONSENT FORM GIVEN TO RESPONDENT

• IF THE RESPONDENT HAS CONSENTED TO RECORDING, START THE RECORDER.

STEP 2: THINK-ALOUD PRACTICE

While you fill out the survey I want you to think out loud about how you are answering some of the questions. Hearing you talk about how you figure out your answers to the questions will help me understand how to make the questions better. I am more interested in how you arrive at your answers than in the answers you give me. To help you get started on thinking aloud, we are going to do a practice activity. Please read the following questions and try to think aloud as you decide on your answer before marking your answer. Then I will ask you some follow-up questions about your thoughts and opinions.

How strongly do you agree or disagree with the following statements about this school? Please think about the school you currently work at: [school name]. Mark One Response.

1. I feel safe at this school.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

PROBES

How easy or difficult was it for you to answer this question?

2. The following types of problems occur often: physical conflicts among students.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

PROBES

In your own words, what is this question asking?

How did you come up with your answer to this question?

STEP 3: COMPLETION OF THE QUESTIONNAIRE

Now I would like you to complete the questionnaire. While you are answering the questions, please think about the school you currently work at: [school name].

Remember that I am very interested in hearing you talk about what you are thinking about as you figure out your answers to the questions.

HAND RESPONDENT A COPY OF THE QUESTIONNAIRE AND A PEN.

These questions ask how strongly you agree or disagree that certain behaviors are a problem among students at this school.

1. Students being threatened, hit, pushed, or kicked by someone they are dating is a problem among students at this school

This could happen in or outside of school.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

PROBES

In your own words, what is this question asking?

How did you feel about this question?

2. Students having someone they are dating try to control them (for example, keeping them from talking to friends or family, keeping track of them at all times) is a problem among students at this school.

This could happen in school or outside of school.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

PROBES

What do you think we mean when we say someone the student was dating “tried to control them”?

Did you find the examples provided here helpful or not helpful?

Are there other ways that someone could be controlled that are not listed here?

How easy or difficult was it to answer this question?

The next questions ask about behaviors that occur “at school”. This includes school buses, school buildings, or at school sponsored events.

3. Students making unwanted sexual comments, sexual jokes, sexual hand or body gestures; or spreading unwanted sexual rumors, photos, or videos to other people is a problem at this school.

This question is asking about things that the other person did not want to hear or see. These things could happen in person, or by text, social media, or other forms of communication.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

PROBES

What do you think we mean when we say “sexual comments, sexual jokes, or sexual hand or body gestures”?

How easy or difficult was it to answer this question?

If needed: Tell me more about your answer.

4. Students having their sexual body parts touched or grabbed (e.g., touching of their butt, crotch, or breasts, or experiencing forced kissing) when they do not agree to it and do not want it to happen is a problem at this school. The person who does it could be another student, a teacher, or someone else.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

PROBES

In your own words what is this question asking?

Did you read or notice here where it says that “the person who does it could be another student, a teacher, or someone else”?

5. Students having their sexual body parts touched or grabbed (e.g., touching of their butt, crotch, or breasts, or experiencing forced kissing) when they cannot make a decision about whether they agree to it or want it to happen because they have been using drugs or alcohol is a problem at this school. The person who does it could be another student, a teacher, or someone else.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

PROBES

In your own words, what does it mean if a student “cannot make a decision about whether they agree to it or want it to happen because they have been using drugs or alcohol”?

How did you select your answer to this question?

6. Students being sexually penetrated (e.g., intercourse, oral sex, anal sex, or other form of penetration) when they do not agree to it and do not want it to happen is a problem at this school. The person who does it could be another student, a teacher, or someone else.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

PROBES

In your own words, what is this question asking?

How easy or difficult was it to answer this question?

The next questions ask about behaviors that occur outside of school or outside of any school sponsored events.

7. Students making unwanted sexual comments, sexual jokes, sexual hand or body gestures; or spreading unwanted sexual rumors, photos, or videos to other people is a problem among students outside of school or outside of any school sponsored events.

This question is asking about things that the other person did not want to hear or see. These things could happen in person, or by text, social media, or other forms of communication.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

PROBES

How easy or difficult was it to answer this question?

If needed: Tell me more about your answer.

8. Students having their sexual body parts touched or grabbed (e.g., touching of their butt, crotch, or breasts, or experiencing forced kissing) when they do not agree to it and do not want it to happen is a problem among students outside of school or outside of any school sponsored events. The person who does it could be another student, a teacher, or someone else.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

PROBES

How did you feel about this question?

9. Students being sexually penetrated (e.g., intercourse, oral sex, anal sex, or other form of penetration) when they do not agree to it and do not want it to happen is a problem among students outside of school or outside of any school sponsored events. The person who does it could be another student, a teacher, or someone else.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

PROBES

How did you come up with your answer?

How easy or difficult was it to respond to questions about students when they are “outside of school or outside of any school sponsored events”?

The next questions ask about the school’s resources to help students with various situations.

10. This school has effective resources for helping students who have experienced sexual contact (including forced kissing/touching of sexual body parts or sexual penetration such as intercourse, oral sex, anal sex or other form of penetration) that they did not agree to and did not want to happen.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

PROBES

In your own words, what does the phrase “effective resources” mean?

How did you decide if the school’s resources were “effective”?

11. This school has effective resources for helping students who have had unwanted sexual comments, jokes, or gestures made to them, sexual photos/videos sent to them, or sexual rumors or sexual photos/videos spread about them.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

PROBES

How easy or difficult was it to respond to this question about the school’s resources?

12. This school has effective resources for helping students who have had someone they were dating threaten to hurt, hit, push, kick, or try to control them (e.g., keep them from talking to their friends or family, keeping track of them at all times).

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

PROBES

If needed: How did you come up with your answer?

OVERALL PROBES

1. Overall, what are your thoughts on this survey?

2. How easy or difficult do you think it would be for other staff to respond to this survey?

3. Did you find the examples provided in some of the questions, such as question 3 and 4, helpful or not helpful?

4. (Other than anything we may have already talked about,) Were there any terms or definitions you did not understand?

5. Were there any questions that made you uncomfortable? IF YES: Tell me why.

6. If you had the opportunity, what would you do to improve this survey?

7. Do you have any other thoughts, comments or suggestions about the survey?

Those are all of the questions I have for you today. Thank you very much for taking the time to talk to me and for providing your valuable input.

STEP 4. THANK YOU FOR PARTICIPATING

We are finished. Thank you for helping us. The information you’ve provided will be very helpful for making this survey better.

Interviewer Instructions: Collect materials. Please give the respondent an incentive [$40].