National Assessment of Educational Progress (NAEP) Social Studies Pretesting

Good morning/afternoon/evening. My name is <CONTACT NAME>, and I am contacting you from Educational Testing Service (ETS). We are carrying out an important study for the National Center for Education Statistics (NCES), part of the U.S. Department of Education. We are recruiting students who are currently in grades 8 or 12 to participate in research interviews supporting the development of new social studies tasks in civics, geography, and U.S. history for the National Assessment of Educational Progress (NAEP). The research interviews will explore how students work through tasks on a <tablet computer or computer> to ensure that the tasks are clear and understandable to a broad range of students. NCES simply wants to know how students would respond to the tasks if they were on an actual test and would like to hear students’ thoughts to improve the tasks and make them easier to understand.

The voluntary research interviews will last no more than <60 or 90> minutes. Students will be asked to respond to civics, geography, and/or U.S. history tasks. Research staff will take notes of students’ reactions to these tasks and will subsequently use students’ feedback to revise and refine the tasks. We will observe and ask about the students’ level of interest in, and understanding of, the tasks. Students will also be asked whether they have any suggestions about how to enhance the tasks.

Student responses will be used only for research purposes and will not be disclosed or used, in identifiable form, for any other purpose except as required by law. [Recruiter note: If asked what law is being referenced, please provide its name: Education Sciences Reform Act of 2002 (ESRA 2002) 20 U.S.C., § 9573.]

We would like your help in distributing and posting informational flyers. We will follow up directly with those who contact us. We will hold some sessions at <LOCATION>, <ADDRESS>; however, if the parents/guardians of students cannot provide transportation to <LOCATION>, <ADDRESS> we would like to set up sessions at your school.

Each participating student will receive a $25 gift card (from a major credit card company) in appreciation of the participants’ time. In addition, if transportation to the site is involved, a parent/guardian of each student will also receive a $25 gift card as appreciation for bringing the child to and from the interview site.

These interviews are an important stage of the NAEP test development process. The comments of students attending your school will help NCES improve test questions.

Would your school be interested in participating in or helping us to recruit students for this research study?

If NO: Thank you for your time. Have a wonderful day/evening. [END CALL]

If YES: Good, thank you very much. To ensure that we interview a broad mix of students and determine the best interview location, I have a few brief screening questions to ask you. It should take only a few minutes.

Screener

[Recruiter: Determine recruitment needs in advance. Look in public documents and school website to find what grades a school includes, the size of each grade, the SES, and racial/ethnic makeup of the student body. Record all responses to Screener questions. End Screener at whatever point the school staff member gives a response that confirms that their group has already been sampled adequately. End Screener Script: “Based on the requirements of this study, we are not able to include your school in the interviews at this time. Thank you for your time. Have a good day/evening.” [Interviewer: End call]]

Do you have <##> [specify recruitment needs] 8th graders who might be available to participate in the study?
Do you have <##> [specify recruitment needs] 12th graders who might be available to participate in the study?

Thank you for taking the time to answer the screening questions. Your school is generally eligible to participate in the research interviews.

Would you be able to distribute and post flyers regarding this study? After receiving a contact of interest, an ETS staff member would follow up with the parent/legal guardian by phone and ask them some screening questions to ensure that a diverse sample of students is included in this study.

If NO: Thank you for your time. Have a good day/evening. [Interviewer: End call]

If YES: Would you be able to make a room at your school available where ETS could conduct some <60- or 90-> minute research sessions?

If YES: That’s great, we will contact you to schedule some sessions at your school after we have some students signed up. (Interviewer: go to Closing]

If NO: (Interviewer: go to Closing]

(Closing)

Thank you for agreeing to disseminate information about this important research study. We will send you an email with the informational flyers attached.

We may need to reach you by telephone. Is the number I called the best number to use?

Do you have any questions at this time?

[Recruiter: Answer any questions.]

If you have any (additional) questions, please call <PHONE NUMBER>.

Thank you again for agreeing to be part of this important research project.

Appendix BB: ETS Recruitment Phone Script for Parents/Legal Guardians of Student Participants (Under Age 18): Social Studies Play Testing

Script when returning calls from potential participants, or calling parents from a previously compiled list:

Hello. May I please speak to <NAME OF Parent/Legal Guardian>?

Good morning/afternoon/evening. Is this the <NAME FROM CALL LIST> household? My name is <CALLER NAME>, and I am calling from Educational Testing Service (ETS). We are carrying out a study on behalf of the U.S. Department of Education’s National Center for Education Statistics. We are recruiting students currently in grades 8 or 12 to sign up to participate in a research study supporting the development of new social studies tasks in civics, geography, and U.S. history for the National Assessment of Educational Progress or NAEP. [If parent asks what this is: This national assessment is administered to students in elementary, middle, and high schools throughout the United States.]

Are there any 8th or 12th grade students in your household?

If Yes: (Great! Let me give you some additional information about the study.)

If No: (Thank you, have a good day/evening.)

We are carrying out research interviews to explore how students work through sample civics, geography, or U.S. history tasks on a <tablet computer or computer> to ensure that they are clear and understandable to a broad range of students. NCES simply wants to know if these tasks were to be included in an actual NAEP assessment, how would students respond to them.

If selected to participate, your child will receive a $25 gift card (from a major credit card company) as a thank you for his/her time. In addition, if transportation to the site is involved, you will also receive a $25 gift card as appreciation for bringing your child to and from the interview site.

The research interview will take place at <NAME OF INTERVIEW SITE AND LOCATION>. It can be scheduled at your convenience and will last no more than <60 or 90 minutes>. These research interviews are an important stage of the NAEP test development process. By sharing their feedback with us, students can directly guide the future of this nationwide assessment by helping NCES evaluate the tasks to create the best test possible.

Do you believe that your child would be interested in participating?

If No: (Thank you anyway, have a good day/evening.) (END CALL)

If Yes: Great! To ensure that we interview a broad mix of students, I have a few brief questions to ask about you and your child. It should take only a few minutes. First, I want to collect some additional information about your child to find out whether he/she meets the participation requirements.

(Skip to Screener, on following page.)

Script when calling a parent/legal guardian who responded to email or letter:

Hello. May I please speak to <NAME OF PARENT/LEGAL GUARDIAN>?

Good morning/afternoon/evening. My name is <NAME>. I’m calling from Educational Testing Service (ETS) on behalf of the National Center for Education Statistics, part of the U.S. Department of Education. Thank you for contacting us about the possibility of your child participating in a research interview to help develop social studies tasks in civics, geography, and U.S. history for the National Assessment of Educational Progress, or NAEP. To ensure that we interview a broad mix of students, I have a few brief screening questions to ask about you and your child. It should take only a few minutes. First, I want to collect some additional information about your child to find out whether he/she meets the participation requirements.

Screener

(INTERVIEWER: Determine recruitment needs in advance. Record all responses. End Screener at whatever point the parent/legal guardian gives a response that confirms that their group has already been sampled adequately. End Screener script: “Thank you for answering my questions. Based on the requirements of this study, we are not able to include your child in the interviews at this time. Thank you for your time. Have a good day/evening. [Interviewer: End call]”)

What is your child’s name?

What grade is your child currently in? (RECRUITER: DO NOT READ LIST)

 K – 7^th  (Unfortunately, we are interviewing only students currently in grade 8 or grade 12 for this study. Thank you for your time. Have a good day/evening.)

 8^th – Good.

 9^th – 11^th (Unfortunately, we are interviewing only students currently in grade 8 or grade 12 for this study. Thank you for your time. Have a good day/evening.)

 12^th – Good.

What school does your child attend? (INTERVIEWER: Note if homeschooled)

____________________________________________________________

What school district does he or she attend? (INTERVIEWER: Note if parochial, charter, homeschooled) __________________________________________________

Is your child currently enrolled in any of the following types of courses? (select all that apply)

 Advanced placement (i.e., AP courses)

 Honor courses

 Remedial courses

 Other courses

What is your child’s race or ethnicity? (INTERVIEWER: Read list only if parent doesn’t immediately reply. “Are you…?”)

Ethnicity:  Hispanic or Latino  Not Hispanic or Latino

Race (one or more of the categories below):

 American Indian or Alaska Native

 Asian

 Black or African American

 Hispanic or Latino

 Native Hawaiian or other Pacific Islander

 White or Caucasian

Just to be sure we're interviewing a broad range of students for this study, does your child participate in the National School Lunch program? [If the respondent indicates they prefer not to disclose this information, the interviewer should say “That’s fine.” And move on to the next question.]

 Yes

 No

Would you consider the area you live in to be …

 Urban

 Suburban

 Rural

Ok. Thank you and what is the zip code of the area that you live in?

________________________

Thank you. We would like to speak with your child to confirm if he/she is interested in participating in the research interview. It will take only a couple of minutes and then I will ask to speak with you again for another moment. Is your child available to speak with us now or should we call him/her back at another time?

 Yes, the child is available for screening questions  (SKIP TO SCRIPT FOR 8th or 12th GRADE STUDENTS)

 No, please call back.

Is the phone number you gave me earlier the best number to call?

Other phone numbers: _________________________________

Is there a good time for me to call? When would be best for you?

___________________________________________________

INTERVIEWER: IF STUDENT NEEDS TO RECEIVE A CALL BACK, RECORD NAME, PHONE NUMBER, DATE/TIME

Parent/Legal Guardian Name: ______________________________________________________________

Child Name: ____________________________________________________________________________

Parent/Legal Guardian Phone ______________________________________________________________

Date: __________________________________________________________________________________

Time: __________________________________________________________________________________

SCRIPT FOR 8th or 12th GRADE STUDENTS

Hello. Is this <NAME OF STUDENT>?

Good morning/afternoon/evening. My name is <NAME>, and I’m calling from Educational Testing Service (ETS). We are doing an important study for the National Center for Education Statistics, part of the U.S. Department of Education.

We have spoken to your parent/legal guardian and we are pleased that you may be interested in helping us develop social studies tasks in civics, geography, and U.S. history for a nationwide student assessment, the National Assessment of Educational Progress, or NAEP. Before I tell you more about what you would be asked to do, I need to ask you one question:

What grade are you currently in? (RECRUITER: DO NOT READ LIST)

 K – 7^th  (Unfortunately, we are interviewing only students currently in grade 8 or grade 12 for this study. Thank you for your time. Have a good day/evening.)

 8^th – Good.

 9^th – 11^th (Unfortunately, we are interviewing only students currently in grade 8 or grade 12 for this study. Thank you for your time. Have a good day/evening.)

 12^th – Good.

Thank you. For the research interview, you will be asked to visit <LOCATION>, <ADDRESS> and respond to sample social studies tasks on a <tablet computer or computer>. Your responses will not be graded. The National Center for Education Statistics simply wants to know how you would do each task if it were to be included in the National Assessment of Educational Progress. The tasks focus on civics, geography, and U.S. history and are interesting and fun to do. Your input will help evaluate the test that will be given to students throughout the nation. To thank you for participating, you will receive a $25 gift card (from a major credit card company). The interview will last no more than <60 or 90 minutes>. Your participation is totally voluntary. If you participate, no one but the study’s researchers will see your interview responses and all other interview materials, and we will not use your name to identify you.

Would you like to participate?

 Yes

 No  (THANK AND END CALL)

If NO: Thank you for your time. I will need to let your parent/legal guardian know that you would not like to participate. Is he/she available?

TO PARENT/LEGAL GUARDIAN: Your child has decided not to participate, but I would like to thank you very much for taking the time to speak with us about this opportunity.

If YES: Thank you for agreeing to participate in this important research study. I will need to speak to your parent/legal guardian to talk about the date/time of the interview. Is your parent/legal guardian available?

 Yes

 No

INTERVIEWER: IF CALLING PARENT/LEGAL GUARDIAN BACK, RECORD NAME, PHONE NUMBER, DATE/TIME

Parent/Legal Guardian Name: _______________________________________________________

Child Name: _____________________________________________________________________

Parent/Legal Guardian Phone: ______________________________________________________

Date: __________________________________________________________________________

Time: __________________________________________________________________________

IF PARENT/LEGAL GUARDIAN AVAILABLE AND CURRENTLY SCHEDULING INTERVIEWS

Thank you again for allowing your child to participate in the NAEP research interviews. We will be holding our current interviews from <DATE> to <DATE>. What would be the best date/time to schedule the interview?

Date ____________________

Time ____________________

Thank you. We will follow up with a confirmation email/letter and a consent form. What is your email or postal address for us to send these to?

____________________________________________________________________________________

IF PARENT/LEGAL GUARDIAN AVAILABLE AND NOT CURRENTLY SCHEDULING INTERVIEWS

Thank you again for allowing your child to participate in the NAEP research interviews. We are in between interview sessions and will need to call you back when we are actively scheduling the interviews to determine the date and time that will work best. What are your scheduling preferences? (SELECT ALL THAT APPLY)

 Morning

 Afternoon

 After school

 Weekends

Thank you so much for talking with us today. We look forward to communicating with you soon.

Appendix BC: ETS Recruitment Phone Script for Students Age 18 or Over Who Responded to Email, Advertisement, or Invitation: Social Studies Play Testing

Hello. May I please speak to <NAME OF STUDENT>?

Good morning/afternoon/evening. My name is <NAME>. I’m calling from Educational Testing Service (ETS) on behalf of the National Center for Education Statistics, part of the U.S. Department of Education. Thank you for contacting us about the possibility of participating in a research interview to help develop social studies: civics, geography, and U.S. history tasks for the National Assessment of Educational Progress, or NAEP. [If the student asks what this is: This national assessment is administered to students in elementary, middle, and high schools throughout the United States.]

To ensure that we interview a broad mix of students, I have a few brief screening questions to ask you. It should take only a few minutes. First, I want to collect some additional information about you to find out whether you meet the participation requirements.

Screener

(INTERVIEWER: Determine recruitment needs in advance. Record all responses. End Screener at whatever point the response confirms that this group has already been sampled adequately. End Screener script: “Thank you for answering my questions. Based on the requirements of this study, we are not able to include you in the interviews at this time. Thank you for your time. Have a good day/evening. [Interviewer: End call]”)

What grade are you currently in at school? (INTERVIEWER: DO NOT READ LIST)

 K – 7^th  (Unfortunately, we are interviewing only students currently in grade 8 or grade 12 for this study. Thank you for your time. Have a good day/evening.)

 8^th – Good.

 9^th – 11^th (Unfortunately, we are interviewing only students currently in grade 8 or grade 12 for this study. Thank you for your time. Have a good day/evening.)

 12^th – Good.

What school do you attend? (INTERVIEWER: Note if homeschooled)

____________________________________________________________

What school district do you attend? (INTERVIEWER: Note if parochial, charter, homeschooled) __________________________________________________

What is your race or ethnicity? (INTERVIEWER: Read list only if student doesn’t immediately reply. “Are you…?”)

Ethnicity:  Hispanic or Latino  Not Hispanic or Latino

Race (one or more of the categories below):

 American Indian or Alaska Native

 Asian

 Black or African American

 Hispanic or Latino

 Native Hawaiian or other Pacific Islander

 White or Caucasian

Are you currently enrolled in any of the following types of courses? (select all that apply)

Advanced placement (i.e., AP courses)

Honor courses

Remedial courses

Other courses

Just to be sure we're interviewing a broad range of students for this study, do you participate in the National School Lunch program?

[If the respondent indicates they prefer not to disclose this information, the interviewer should say “That’s fine” and move on to the next question.]

Would you consider the area you live in to be …

Urban

Suburban

Rural

Ok. Thank you and what is the zip code of the area that you live in?

________________________

Thank you. For the research interview, you will be asked to visit <LOCATION>, <ADDRESS> and respond to sample social studies tasks on a <tablet computer or computer>. Your responses will not be graded. The National Center for Education Statistics simply wants to know how you would do each task if it were to be included in the National Assessment of Educational Progress. The tasks focus on civics, geography, and U.S. history and are interesting and fun to do. Your input will help evaluate the test that will be given to students throughout the nation. To thank you for participating, you will receive a $25 gift card (from a major credit card company). The interview will last no more than <60 or 90 minutes>. Your participation is totally voluntary. If you participate, no one but the study’s researchers will see your interview responses and all other interview materials, and we will not use your name to identify you.

Would you like to participate?

 Yes

 No  (THANK AND END CALL)

IF CURRENTLY SCHEDULING INTERVIEWS

Thank you again for agreeing to participate in the NAEP research interviews. We will be holding our current interviews from <DATE> to <DATE>. What would be the best date/time to schedule the interview?

Date ____________________

Time ____________________

IF NOT CURRENTLY SCHEDULING INTERVIEWS

Thank you again for agreeing to participate in the NAEP research interviews. We are in between interview sessions and will need to call you back when we are actively scheduling the interviews to determine the date and time that will work best. What are your scheduling preferences? (SELECT ALL THAT APPLY)

 Morning

 Afternoon

 After school

 Weekends

Thank you so much for talking with us today. We look forward to communicating with you soon.

Appendix BD: ETS Consent Form for Parents/Legal Guardians of Student Participants (Under Age 18): Social Studies Play Testing

PARENTAL CONSENT FORM

FOR PARTICIPATION IN RESEARCH

Educational Testing Service (ETS)

Title of Research Study: Play Testing for NAEP Social Studies: Civics, Geography, and U.S. History

Thank you for your interest in having your child participate in the research study described below. This consent form explains the research study. Please read it carefully, and feel free to ask questions about anything you do not understand. If you do not have questions now, you may ask them later as they occur to you.

What is this study about?

The National Center for Education Statistics (NCES), part of the U.S. Department of Education, is conducting informal interviews called Play Testing with students currently in grades 8 and 12 about new test questions for the National Assessment of Educational Progress (NAEP). The questions we are testing focus on social studies: civics, geography, and U.S. history and will be part of a computer-based test. In these sessions, students will see early versions of computer-based questions. We may ask students whether they have any suggestions about how we might improve the questions.

What will happen during the play testing sessions?

Each participant will take an early version of a new NAEP social studies: civics, geography, and U.S. history task on a computer. We will ask questions that focus on your child’s level of interest and ability to understand the questions presented to them. We may ask whether they have any suggestions about how we might improve the questions. Their responses will not be graded.

ETS staff will audio record and take notes about your child’s reactions to these questions to revise and refine test content. There will be more than one adult present.

Where will the play testing sessions take place?

The sessions will take place at <NAME OF SESSION location>, located at <ADDRESS>.

How long will the play testing session last?

The session will last no more than 60 minutes. Each session requires only one visit; however your child may participate in multiple sessions, if interested.

Will you keep the information private and confidential?

Your child will be assigned a unique student identifier (ID), and at no time will his/her name be linked to any of his/her answers. Your child’s responses will be used only for research purposes and will not be disclosed or used, in identifiable form, for any other purpose except as required by law [Education Sciences Reform Act of 2002, 20 U.S.C §9573]. These sessions will be audio recorded, as long as you agree, and only used by the researchers.

Will the results go to my child’s school?

Your child’s answers will not be disclosed to his/her school, and will not be linked to any personally identifiable information such as the school’s name.

What are the possible risks of being in the study?

There are no known risks to participating in this research.

Can participation end early?

Participation in this study is completely voluntary. Your child has the right to refuse to answer particular questions. Your child may elect to withdraw from this study at any time and can end early if he/she wishes.

Whom can I contact with questions or for further information?

<CONTACT NAME> is coordinating this session at ETS on behalf of NCES. If you have any questions, please call <CONTACT PHONE> or email <CONTACT EMAIL>.

By signing below, you agree that your child, _________________________________, may participate in this research study. Your child will receive a $25 gift card (from a major credit card company) at the conclusion of the session. In addition, if transportation to the interview site is involved, you will also receive a $25 gift card as appreciation for the time bringing your child to and from the site.

Your signature below will indicate that your questions have been answered satisfactorily, and that you have read and understood the information provided above.

Print Name: ___________________________________________________________________________________

Signature: ______________________________________________________Date:__________________________

Email address: _________________________________________________________________________________

School student attends: _________________________________________________________________________

To be completed by ETS staff:

Student Payment Card Transaction Number: ________________________________________________________

Parent/Guardian Payment Card Transaction Number: _________________________________________________

Appendix BE: ETS Consent Form for Student Participants (Age 18 or Over): Social Studies Play Testing

CONSENT FORM

FOR PARTICIPATION IN RESEARCH

Educational Testing Service (ETS)

Title of Research Study: Play Testing for NAEP Social Studies: Civics, Geography, and U.S. History

Thank you for your interest in participating in the research study described below. This consent form explains the research study. Please read it carefully, and feel free to ask questions about anything you do not understand. If you do not have questions now, you may ask them later as they occur to you.

What is this study about?

The National Center for Education Statistics (NCES), part of the U.S. Department of Education, is conducting informal interviews called Play Testing with students currently in grades 8 and 12 about new test questions for the National Assessment of Educational Progress (NAEP). The questions we are testing focus on social studies: civics, geography, and U.S. history and will be part of a computer-based test. In these sessions, students will see early versions of computer-based questions. We may ask students whether they have any suggestions about how we might improve the questions.

What will happen during the play testing sessions?

Each participant will take an early version of a new NAEP social studies: civics, geography, and U.S. history task on a computer. We will ask questions that focus on your level of interest and your ability to understand the questions presented to you. We may ask whether you have any suggestions about how we might improve the questions. Your responses will not be graded.

ETS staff will audio record and take notes about your reactions to these questions to revise and refine test content.

Where will the play testing sessions take place?

The sessions will take place at <NAME OF SESSION location>, located at <ADDRESS>.

How long will the play testing sessions last?

The sessions will last no more than 60 minutes. Each session requires only one visit; however you may participate in multiple sessions, if you are interested.

Will you keep information private and confidential?

You will be assigned a unique student identifier (ID), and at no time will your name be linked to any of your answers. All of your responses will be used only for research purposes and will not be disclosed or used, in identifiable form, for any other purpose except as required by law [Education Sciences Reform Act of 2002 (ESRA 2002) 20 U.S.C., § 9573]. The interviews will be audio recorded, as long as you agree, and only used by the researchers.

Will the results go to my school?

Your answers will not be disclosed to your school, and will not be linked to any personally identifiable information such as the school’s name.

What are the possible risks of being in the study?

There are no known risks to participating in this research.

Can my participation end early?

Your participation in this study is completely voluntary and you have the right to refuse to answer particular questions. You may also elect to withdraw from this study at any time and you can end early if you wish.

Whom can I contact with questions or for further information?

<CONTACT NAME> is coordinating this session at ETS on behalf of NCES. If you have any questions, please call <CONTACT PHONE> or email <CONTACT EMAIL>.

By signing below, you agree that you will participate in this research study. You will receive a $25 gift card (from a major credit card company) at the conclusion of each session. In addition, if transportation to the site is involved, your parent/guardian will also receive a $25 gift card in appreciation for the time bringing you to and from the play testing site. Your signature below will indicate that your questions have been answered satisfactorily, and that you have read and understood the information provided above.

Print Name: __________________________________________________________________________________

Signature: ______________________________________________________Date: _________________________

Email address: _________________________________________________________________________________

School student attends: _________________________________________________________________________

To be completed by ETS staff:

Student Payment Card Transaction Number: ________________________________________________________

Parent/Guardian Payment Card Transaction Number: _________________________________________________

Appendices – Cognitive Interviews

Appendix BF: ETS Consent Form for Parents/Legal Guardians of Student Participants (Under Age 18): Social Studies Cognitive Interviews

PARENTAL CONSENT FORM

FOR PARTICIPATION IN RESEARCH

Educational Testing Service (ETS)

Title of Research Study: Cognitive Laboratory Interviews for NAEP Social Studies:

Civics, Geography, and U.S. History

What is this study about?

The National Center for Education Statistics (NCES), part of the U.S. Department of Education, is conducting research interviews with students currently in grades 8 and 12 about new test questions for the National Assessment of Educational Progress (NAEP). The questions we are testing focus on social studies: civics, geography, and U.S. history and will be part of a computer-based test. In these interviews, students will see early versions of computer-based tasks. We will ask questions that focus on students’ ability to understand the questions presented to them. Students will also be asked whether they have any suggestions about how we might improve the tasks.

What will happen during the research interview?

Where will the research interviews take place?

The interviews will take place at <NAME OF INTERVIEW LOCATION>, located at <ADDRESS>.

How long will the research interview last?

The interview will last no more than 90 minutes. The interview requires only one visit to the interview site.

Will you keep information private and confidential?

Your child will be assigned a unique student identifier (ID), and at no time will his/her name be linked to any of his/her answers. Your child’s responses will be used only for research purposes and will not be disclosed or used, in identifiable form, for any other purpose except as required by law [Education Sciences Reform Act of 2002, 20 U.S.C §9573]. Audio and screen activity will be recorded for analysis, but no personal identifying information will be recorded or retained.

Will the results go to my child’s school?

Your child’s answers will not be disclosed to his/her school and will not be linked to any personally identifiable information such as the school’s name.

What are the possible risks of being in the study?

There are no known risks to participating in this study.

Can participation end early?

Whom can I contact with questions or for further information?

<CONTACT NAME> is coordinating this study at ETS on behalf of NCES. If you have any questions, please call <CONTACT PHONE> or email <CONTACT EMAIL>.

By signing below, you agree that your child, _______________________________, may participate in this study. Your child will receive a $25 gift card (from a major credit card company) at the conclusion of the interview.

In addition, if transportation to the interview site is involved, you will also receive a $25 gift card as appreciation for the time bringing your child to and from the site.

Your signature below will indicate that your questions have been answered satisfactorily, and that you have read and understood the information provided above.

Print Name: ________________________________________________________________________________

Signature: ______________________________________________________Date: _______________________

Email address: ______________________________________________________________________________

School student attends: _______________________________________________________________________

To be completed by ETS staff:

Student Payment Card Transaction Number: ________________________________________________________

Parent/Guardian Payment Card Transaction Number: _________________________________________________

Appendix BG: ETS Consent Form for Student Participants (Age 18 or Over): Social Studies Cognitive Interviews

CONSENT FORM

FOR PARTICIPATION IN RESEARCH

Educational Testing Service (ETS)

Title of Research Study: Cognitive Laboratory Interviews for NAEP Social Studies:

Civics, Geography, and U.S. History

What is this study about?

The National Center for Education Statistics (NCES), part of the U.S. Department of Education, is conducting research interviews with students currently in grades 8 and 12 about new test questions for the National Assessment of Educational Progress (NAEP). The questions we are testing focus on social studies: civics, geography, and U.S. history and will be part of a computer-based test. In these interviews, students will see early versions of computer-based questions. We may ask students whether they have any suggestions about how we might improve the questions.

What will happen during the research interview?

Each participant will work through an early version of a new NAEP social studies: civics, geography, or U.S. history task on a tablet computer. An interviewer will ask what you were thinking as you completed the task, and whether you believe that the task is clear and understandable. Your responses will not be graded. In some cases, we may use a camera (also known as an eye-tracker) attached to the computer to record where on the screen students are looking during the task. Interviewers will also take notes about your reactions to the social studies tasks so that tasks can be revised and refined. We will observe and ask you questions about your understanding of the tasks.

Where will the research interviews take place?

The interviews will take place at <NAME OF INTERVIEW LOCATION>, located at <ADDRESS>.

How long will the research interview last?

The interview will last no more than 90 minutes. The interview requires only one visit to the interview site.

Will you keep information private and confidential?

You will be assigned a unique student identifier (ID), and at no time will your name be linked to any of your answers. Your responses will be used only for research purposes and will not be disclosed or used, in identifiable form, for any other purpose except as required by law [Education Sciences Reform Act of 2002, 20 U.S.C §9573]. Audio and screen activity will be recorded for analysis, but no personal identifying information will be recorded or retained.

Will the results go to my school?

Your answers will not be disclosed to your school and will not be linked to any personally identifiable information such as the school’s name.

What are the possible risks of being in the study?

There are no known risks to participating in this study.

Can participation end early?

Your participation in this study is completely voluntary. You have the right to refuse to answer particular questions. You may elect to withdraw from this study at any time.

Whom can I contact with questions or for further information?

<CONTACT NAME> is coordinating this study at ETS on behalf of NCES. If you have any questions, please call <CONTACT PHONE> or email <CONTACT EMAIL>.

By signing below, you agree that you will participate in this research study. You will receive a $25 gift card (from a major credit card company) at the conclusion of the interview. In addition, if transportation to the site is involved, your parent/guardian will also receive a $25 gift card in appreciation for the time bringing you to and from the interview site. Your signature below will indicate that your questions have been answered satisfactorily, and that you have read and understood the information provided above.

Print Name: ________________________________________________________________________________

Signature: ______________________________________________________Date: _______________________

Email address: ______________________________________________________________________________

School student attends: _______________________________________________________________________

To be completed by ETS staff:

Student Payment Card Transaction Number: ________________________________________________________

Parent/Guardian Payment Card Transaction Number: _________________________________________________

Appendix BH: EurekaFacts Recruitment Phone Script for Schools of Potential Student Participants: Social Studies Cognitive Interviews

Hello. May I please speak to <NAME of contact at school>?

Good morning/afternoon/evening. My name is <CONTACT NAME> and I am contacting you from EurekaFacts, a local research company. We are carrying out an important study for the National Center for Education Statistics (NCES), part of the U.S. Department of Education. We are recruiting students to participate in research sessions supporting the development of new social studies tasks in civics, geography, and/or U.S. history for the National Assessment of Educational Progress (NAEP). The research sessions will explore how students work through the tasks on a tablet computer to ensure that the tasks are clear and understandable to a broad range of students. NCES simply wants to know how students would do the tasks if they were on an actual test, and would like to hear students’ thoughts to improve the tasks and make them easier to understand.

The voluntary research sessions will last no more than 90 minutes. Students will be asked to respond to civics, geography, and U.S. history tasks and will be asked about them. EurekaFacts staff will take notes of students’ reactions to the tasks, and students’ feedback will be used to revise and refine the tasks. We will observe and ask about the students’ understanding of the tasks. Students may also be asked whether they have any suggestions about how to enhance the tasks.

Student responses will be used only for research purposes and will not be disclosed or used, in identifiable form, for any other purpose except as required by law.

[INTERVIEWER note: If asked what law is being referenced, please provide its name: Education Sciences Reform Act of 2002 (ESRA 2002) 20 U.S.C., § 9573.]

We would like your help in distributing flyers to students, and we will follow up directly with any respondents who contact us. We will hold some sessions at <LOCATION>; however, if the parents/guardians of students cannot provide transportation to <LOCATION>, we would like to set up sessions at your school.

For each participating student, a $25 gift card (from a major credit card company) will be issued in appreciation for participants’ time. In addition, if transportation to the site is involved, a parent or guardian of each student will receive a $25 gift card as appreciation for bringing the child to and from the research site.

These sessions are an important stage of the NAEP test development process. The comments of students attending your school will help NCES improve test questions.

Would your school be interested in participating in or helping us to recruit students for this research study?

If NO: Thank you for your time. Have a good day/evening.

If YES: Great, thank you very much. To ensure that we work with a broad mix of students and determine the best session location, I have a few brief screening questions to ask you. It should take only a few minutes.

Screener

[INTERVIEWER: Determine recruitment needs in advance. Look in public documents and school website to find what grades a school includes, the size of each grade, the SES, and racial/ethnic makeup of the student body. Record all responses to screener questions. End screener at whatever point the school staff member gives a response that confirms that their group has already been sampled adequately. End screener script: “Based on the requirements of this study, we are not able to include you in the study at this time. Thank you for your time. Have a good day/evening. [Interviewer: End call]”]

Are you at a school with a population that includes:

8th-grade students
12th-grade students

How many students are enrolled in your school?
1. 299 students or fewer
2. 300-599 students
3. 600 students or more

Eligible Participant Script

Thank you for taking the time to answer the screening questions. Students at your school are generally eligible to participate in the research sessions. After receiving a contact of interest, a EurekaFacts staff member will follow up with the parent/guardian by phone and ask them some screening questions to ensure that a diverse sample of students is included in this study.

Would you be able to send flyers to parents of potential student participants?

If NO: Thank you for your time. Have a good day/evening. [Interviewer: End call.]

If YES: Would you be able to make a room at your school available where EurekaFacts could conduct some 90-minute research sessions?

If YES: That’s great, we will contact you to schedule some sessions at your school after we have some students signed up. (Interviewer: go to Closing]

If NO: Go to “Closing.”

(Closing)

Thank you for agreeing to disseminate information about this important research study. We will send you email with informational flyers.

Is [INSERT EMAIL ADDRESS] the best email address to reach you? OR What is the best email address to reach you?

We may need to reach you by telephone. Is the number I called the best number to use?

Do you have any questions at this time?

[INTERVIEWER: Answer any questions.]

If you do have any questions, please call <PHONE NUMBER>.

Thank you again for agreeing to be part of this important research project. Have a wonderful day/evening.

Appendix BI: EurekaFacts Recruitment Phone Script for Parents/Legal Guardians of Student Participants (Under Age 18): Social Studies Cognitive Interviews

Script when calling parents from previously compiled list:

Good morning/afternoon/evening. Is this the <NAME FROM CALL LIST> household? My name is <CALLER NAME>, and I am calling from EurekaFacts, a local research company. We are carrying out a study on behalf of the U.S. Department of Education’s National Center for Education Statistics.

We are recruiting students currently in grades 8 or 12 to participate in a research study supporting the development of new questions for the National Assessment of Educational Progress or NAEP. [If parent asks what this is: This national assessment is administered to students in elementary, middle, and high schools throughout the United States.]

Are there any students in your household?

No: (Thank you, have a good day/evening.)

Yes: (Great! Let me give you some additional information about the study.)

We are looking for students to help us test new computer-delivered social studies: civics, geography, and U.S. history tasks. These tasks will be part of the National Assessment of Educational Progress (NAEP) delivered to students nationwide.

The research session will take place at <NAME OF SESSION SITE AND LOCATION>. It can be scheduled at your convenience and will last about 90 minutes. During the session, your child will be asked to respond to sample civics, geography, and/or U.S. history tasks on a tablet computer. NCES simply wants to know how students would do the tasks if they were on an actual test, and would like to hear students’ thoughts to improve the tasks and make them easier to understand.

If selected to participate, your child will receive a $25 gift card (from a major credit card company) as thanks for his or her time. In addition, if transportation to the site is involved, you will also receive a $25 gift card as appreciation for bringing your child to and from the research site.

These research sessions are an important stage of the NAEP test development process. By sharing their feedback with us, students can directly guide the future of this nationwide assessment by helping NCES evaluate the test questions and instructions to create the best test possible.

Do you believe that your child would be interested in participating?

No: (Thank you anyway, have a good day/evening.)

Yes: (Great! Thank you so much. To ensure that we work with a broad mix of students, I have a few brief questions to ask about you and your child. It should take only a few minutes. First, I want to collect some additional information about your child to find out whether he or she meets the participation requirements.)

(Skip to Screener, on following page.)

Script when calling a parent/guardian who responded to email or letter

Good morning/afternoon/evening. Is this the <NAME FROM CALL LIST> household? My name is <NAME>. I’m calling from EurekaFacts on behalf of the National Center for Education Statistics, part of the U.S. Department of Education. Thank you for contacting us about the possibility of your child participating in a research session to help assess test questions for the National Assessment of Educational Progress, or NAEP. To ensure that we work with a broad mix of students, I have a few brief screening questions to ask about you and your child. It should only take a few minutes. First, I want to collect some additional information about your child to find out whether he or she meets the participation requirements.

Screener

[INTERVIEWER: Determine recruitment needs in advance. Record all responses. End screener at whatever point the parent or legal guardian gives a response that confirms that their group has already been sampled adequately. End Screener script: “Thank you for answering my questions. Based on the requirements of this study, we are not able to include your child in the sessions at this time. Thank you for your time. Have a good day/evening. [Interviewer: End call]”)

What is your child’s name?

What grade is your child currently in? (INTERVIEWER: DO NOT READ LIST)

 K – 7^th  (Unfortunately, we need only students currently in grade 8 or grade 12 for this study. Thank you for your time. Have a good day/evening.)

 8^th

 9^th – 11^th (Unfortunately, we need only students currently in grade 8 or grade 12 for this study. Thank you for your time. Have a good day/evening.)

 12^th

What school does your child attend? (INTERVIEWER: Note if homeschooled)

____________________________________________________________

What school district does he or she attend? (INTERVIEWER: Note if parochial, charter, homeschooled) __________________________________________________

Is your child currently enrolled in any of the following types of courses (select all that apply)

Advanced placement (i.e., AP courses)

Honor courses

Remedial courses

Other courses

What is your child’s race or ethnicity? (INTERVIEWER: Read list only if parent doesn’t immediately reply. “Are you…?”)

Ethnicity:  Hispanic or Latino  Not Hispanic or Latino

Race (one or more of the categories below):

 American Indian or Alaska Native

 Asian

 Black or African American

 Hispanic or Latino

 Native Hawaiian or other Pacific Islander

 White or Caucasian

Just to be sure we're interviewing a broad range of students for this study, does your child participate in the National School Lunch program?

[If the respondent indicates they prefer not to disclose this information, the interviewer should say “That’s fine.” and move on to the next question.]

Would you consider the area you live in to be …

Urban

Suburban

Rural

Ok. Thank you and what is the zip code of the area that you live in?

______________

 Yes, the child is available for screening questions  (SKIP TO SCRIPT FOR 8th or 12th GRADE STUDENTS)

 No, please call back.

Is the phone number you gave me earlier the best number to call?

Other phone numbers: _________________________________

Is there a good time for me to call? When would be best for you?

___________________________________________________

INTERVIEWER: IF CHILD NEEDS TO RECEIVE A CALL BACK, RECORD NAME, PHONE NUMBER, DATE/TIME

Parent/Guardian Name ___________________________________________________________

Child Name ____________________________________________________________________

Date ____________________

Time ____________________

SCRIPT FOR 8th or 12th GRADE STUDENTS

Good morning/afternoon/evening. Is this <NAME OF STUDENT>? My name is <NAME>, and I’m calling from EurekaFacts. We are doing an important study for the National Center for Education Statistics, part of the U.S. Department of Education.

What grade are you currently in? (RECRUITER: DO NOT READ LIST)

 K – 7^th  (Unfortunately, we are interviewing only students currently in grade 8 or grade 12 for this study. Thank you for your time. Have a good day/evening.)

 8^th – Good.

 9^th – 11^th (Unfortunately, we are interviewing only students currently in grade 8 or grade 12 for this study. Thank you for your time. Have a good day/evening.)

 12^th – Good.

Thank you. For the research interview, you will be asked to visit <LOCATION>, <ADDRESS> and respond to sample social studies tasks on a tablet computer. Your responses will not be graded. The National Center for Education Statistics simply wants to know how you would do each task if it were to be included in the National Assessment of Educational Progress. The tasks focus on civics, geography, and U.S. history and are interesting and fun to do. Your input will help evaluate the test that will be given to students throughout the nation. To thank you for participating, you will receive a $25 gift card (from a major credit card company). The interview will last no more than 90 minutes. Your participation is totally voluntary. If you participate, no one but the study’s researchers will see your interview responses and all other interview materials, and we will not use your name to identify you.

Would you like to participate?

 Yes

 No  (THANK AND END CALL)

If NO: Thank you for your time. I will need to let your parent/legal guardian know that you would not like to participate. Is he/she available?

TO PARENT/LEGAL GUARDIAN: Your child has decided not to participate, but I would like to thank you very much for taking the time to speak with us about this opportunity.

 Yes

 No

INTERVIEWER: IF CALLING PARENT/LEGAL GUARDIAN BACK, RECORD NAME, PHONE NUMBER, DATE/TIME

Parent/Legal Guardian Name: _______________________________________________________

Child Name: _____________________________________________________________________

Parent/Legal Guardian Phone: ______________________________________________________

Date: __________________________________________________________________________

Time: __________________________________________________________________________

IF PARENT/LEGAL GUARDIAN AVAILABLE AND CURRENTLY SCHEDULING INTERVIEWS

Date ____________________

Time ____________________

Thank you. We will follow up with a confirmation email/letter and a consent form. What is your email or postal address for us to send these to?

____________________________________________________________________________________

IF PARENT/LEGAL GUARDIAN AVAILABLE AND NOT CURRENTLY SCHEDULING INTERVIEWS

 Morning

 Afternoon

 After school

 Weekends

Thank you so much for talking with us today. We look forward to communicating with you soon.

Appendix BJ: EurekaFacts Recruitment Phone Script for Students (Age 18 or Over): Social Studies Cognitive Interviews

Script when calling students from previously compiled list:

Good morning/afternoon/evening. Is this <NAME FROM CALL LIST>? My name is <CALLER NAME>, and I am calling from EurekaFacts, a local research company. We are carrying out a study on behalf of the U.S. Department of Education’s National Center for Education Statistics.

We are recruiting students to participate in a research study supporting the development of new questions for the National Assessment of Educational Progress or NAEP.

Are you a student 18 or over?

No: (Thank you, have a good day/evening.)

Yes: (Great! Let me give you some additional information about the study.)

We are looking for students to help us test new computer-delivered social studies: civics, geography, and U.S. history tasks. These tasks will be part of the National Assessment of Educational Progress (NAEP) delivered to students nationwide.

The research session will take place at <NAME OF SESSION SITE AND LOCATION>. It can be scheduled at your convenience and will last about 90 minutes. During the session, you will be asked to respond to sample civics, geography, and/or U.S. history tasks on a tablet computer. NCES simply wants to know how students would do the tasks if they were on an actual test, and would like to hear students’ thoughts to improve the tasks and make them easier to understand.

If selected to participate, you will receive a $25 gift card (from a major credit card company) as thanks for your time. In addition, if transportation to the site is involved, your parent or guardian will also receive a $25 gift card as appreciation for bringing you to and from the research site.

These research sessions are an important stage of the NAEP test development process. By sharing your feedback with us, students can directly guide the future of this nationwide assessment by helping NCES evaluate the test questions and instructions to create the best test possible.

Would you be interested in participating?

No: (Thank you anyway, have a good day/evening.)

Yes: (Great! Thank you so much. To ensure that we work with a broad mix of students, I have a few brief questions to ask you. It should take only a few minutes. First, I want to collect some additional information to find out whether you meet the participation requirements.)

(Skip to Screener, on following page.)

Script when calling a student age 18 or over who responded to email, advertisement, or invitation

Good <MORNING/AFTERNOON/EVENING>. Is this <NAME OF STUDENT>? My name is <NAME>. I’m calling from EurekaFacts on behalf of the U.S. Department of Education’s National Center for Educational Statistics, also known as NCES. Thank you for contacting us about the possibility of participating in a research session to help test new test questions for the National Assessment of Educational Progress, also known as the NAEP assessment. To ensure that we interview a broad mix of students, I have a few brief screening questions to ask you about yourself. It should take only a few minutes of your time. First, I want to collect some additional information about you to identify if you meet the participation requirements.

Before proceeding, please confirm that you are still interested in the possibility of participating in this research interview?

Yes: (INTERVIEWER: Proceed to “Screener.”)

No: (Thank you for your time. Have a good <DAY/EVENING>.)

Screener:

[INTERVIEWER: Determine recruitment needs in advance. Record all responses. End screener at whatever point the student gives a response that confirms that their group has already been sampled adequately. End Screener script: “Thank you for answering my questions. Based on the requirements of this study, we are not able to include you in the sessions at this time. Thank you for your time. Have a good day/evening. [Interviewer: End call]”)

What is your name?

What grade are you currently in? (If the student indicates that he/she is in any grade other than 8^th or 12^th, thank and end call: “Unfortunately we are interviewing only students currently in grades 8 or 12 for this study. Thank you for your time. Have a good <DAY/EVENING>.”)

What school do you attend? (INTERVIEWER: Note if homeschooled)

____________________________________________________________

What school district do you attend? (INTERVIEWER: Note if parochial, charter, homeschooled) __________________________________________________

Are you currently enrolled in any of the following types of courses? (select all that apply)

Advanced placement (i.e., AP courses)

Honor courses

Remedial courses

Other courses

What is your race or ethnicity? (INTERVIEWER: Read list only if respondent doesn’t immediately reply. “Are you…?”)

Ethnicity:  Hispanic or Latino  Not Hispanic or Latino

Race (one or more of the categories below):

American Indian or Alaska Native

Asian

Black or African American

Hispanic or Latino

Native Hawaiian or other Pacific Islander

White or Caucasian

Just to be sure we're interviewing a broad range of students for this study, do you participate in the National School Lunch program?

[If the respondent indicates they prefer not to disclose this information, the interviewer should say “That’s fine” and move on to the next question.]

Would you consider the area you live in to be …

Urban

Suburban

Rural

Ok. Thank you and what is the zip code of the area that you live in?

________________________

Would you like to participate?

Yes

No  (THANK AND END CALL)

If NO: Thank you for your time, and have a wonderful <DAY/EVENING.>

If YES:

Thank you for agreeing to participate in this important research study. For the research session, you will be asked to visit EurekaFacts headquarters in Rockville, MD and respond to a sample of test questions. Your responses will not be graded. The questions focus on social studies and are interesting and fun to answer. You will be asked to answer the questions on a tablet computer.

Your input will help evaluate the test that will be given to students throughout the nation. To thank you for participating, you will receive a $25 gift card (from a major credit card company). If your parent or legal guardian brings you to and from our office, he or she will also receive a $25 gift card. The session will last about 90 minutes. Your participation is totally voluntary. If you participate, no one but the study’s researchers will see your responses and all other session materials, and we will not use your name to identify you.

Please note that before your interview starts, you will be asked to sign a consent form to confirm that you have agreed to participate in the interview. You will not be able to participate until the form has been signed.

IF CURRENTLY SCHEDULING INTERVIEWS

Thank you again for agreeing to participate in the NAEP research sessions. We will be holding our current sessions from <DATE> to <DATE>. What would be the best date/time to schedule the session?

Date ____________________

Time ____________________

IF NOT CURRENTLY SCHEDULING SESSIONS

Thank you again for agreeing to participate in the NAEP research sessions. We are in between research sessions, and will need to call you back when we are actively scheduling the sessions to determine the date and time that will work best. What are your scheduling preferences? (SELECT ALL THAT APPLY)

 Morning

 Afternoon

 After school

 Weekends

Thank you so much for talking with us today. We look forward to communicating with you soon.

Appendix BK: EurekaFacts Consent Form for Parents/Legal Guardians of Student Participants (Under Age 18): Social Studies Cognitive Interviews

PARENTAL CONSENT FORM

FOR PARTICIPATION IN RESEARCH

EurekaFacts, LLC

Title of Research Study: Cognitive Laboratory Interviews for NAEP Social Studies: Civics, Geography, and U.S. History

Thank you for your interest in having your child participate in the research project described below. This consent form explains the research study. Please read it carefully, and feel free to ask questions about anything you do not understand. If you do not have questions now, you may ask them later as they occur to you.

What is this study about?

The National Center for Education Statistics (NCES), part of the U.S. Department of Education, is conducting research interviews with students currently in grades 8 and 12 about new test questions for the National Assessment of Educational Progress (NAEP). The questions we are testing focus on social studies: civics, geography, and U.S. history and will be part of a computer-based test. In these interviews, students will see early versions of computer-based questions. EurekaFacts staff will ask questions that focus on students’ level of interest and their ability to understand the questions presented to them. Students will also be asked if they have any suggestions about how we might improve the questions.

What will happen during the research interview?

Each participant will work through an early version of a new NAEP social studies: civics, geography, or U.S. history task on a tablet computer. An interviewer will ask what your child was thinking as he or she completed the task, and whether he or she believes the task is clear and understandable. Your child’s responses will not be graded. EurekaFacts staff will use computer software to record what students do on screen, as well as microphones to record students’ reactions to tasks, descriptions of their work, and responses to interviewer questions. In some cases, we may use a camera (also known as an eye-tracker) attached to the computer to record where on the screen students are looking during the task. Interviewers will also take notes about students’ reactions to the tasks so that tasks can be revised and refined. We will observe and ask about the students’ understanding of the tasks. There will be more than one adult present.

Where will the research interviews take place?

The interviews will take place at <NAME OF INTERVIEW LOCATION>, located at <LOCATION>, near <NEARBY LANDMARK OR METRO STATION>.

How long will the research interview last and when it will take place?

The interview will last up to 90 minutes and will require only one visit to our office. Interviews can be scheduled at your convenience.

Will you keep information private and confidential?

Your child will be assigned a unique student identifier (ID), and at no time will his/her name be linked to any of his/her answers. Your child’s responses will be used only for research purposes and will not be disclosed or used, in identifiable form, for any other purpose except as required by law [Education Sciences Reform Act of 2002, 20 U.S.C §9573]. Audio and screen activity will be recorded for analysis, but no personal identifying information will be recorded or retained.

Will the results go to my child’s school?

Your child’s answers will not be disclosed to his/her school, and will not be linked to any personally identifiable information such as the school’s name.

What are the possible risks of being in the study?

There are no known risks to participating in this study.

Can participation end early?

Your child’s participation in this study is completely voluntary. Your child has the right to refuse to answer particular questions. You or your child may elect to withdraw from this study at any time.

Whom can I contact with questions or for further information?

<CONTACT NAME> is coordinating this study at EurekaFacts on behalf of NCES. If you have any questions, please call him/her at <CONTACT PHONE> or by email <CONTACT EMAIL>.

In addition, if transportation to the site is involved, you will receive a $25 gift card as appreciation for the time bringing your child to and from the research site.

Your signature below will indicate that your questions have been answered satisfactorily, and that you have read and understood the information provided above.

Print Name: _________________________________________________________________________

Signature: ___________________________________________________________________________

Date:_______________________________________________________________________________

School that student attends: ____________________________________________________________

To be completed by EurekaFacts staff:

Student Payment Card Transaction Number: ________________________________________________________

Parent/Guardian Payment Card Transaction Number: _________________________________________________

Appendix BL: EurekaFacts Consent Form for Student Participants (Age 18 and Over): Social Studies Cognitive Interviews

CONSENT FORM

FOR PARTICIPATION IN RESEARCH

EurekaFacts, LLC

Title of Research Study: Cognitive Laboratory Interviews for NAEP Social Studies:

Civics, Geography, and U.S. History

Thank you for your interest in participating in the research project described below. This consent form explains the research study. Please read it carefully, and feel free to ask questions about anything you do not understand. If you do not have questions now, you may ask them later as they occur to you.

What is this study about?

The National Center for Education Statistics (NCES), part of the U.S. Department of Education, is conducting research interviews with students currently in grades 8 and 12 about new test questions for the National Assessment of Educational Progress (NAEP). The questions we are testing focus on social studies: civics, geography, and U.S. history and will be part of a computer-based test. In these interviews, students will see early versions of computer-based questions. EurekaFacts staff will ask questions that focus on students’ level of interest and their ability to understand the questions presented to them. Students will also be asked if they have any suggestions about how we might improve the questions.

What will happen during the research interview?

Each participant will work through an early version of a new NAEP social studies: civics, geography, or U.S. history task on a tablet computer. An interviewer will ask what you were thinking as you completed the task, and whether you believe that the task is clear and understandable. Your responses will not be graded. EurekaFacts staff will use computer software to record what students do on screen, as well as microphones to record students’ reactions to tasks, descriptions of their work, and responses to interviewer questions. In some cases, we may use a camera (also known as an eye-tracker) attached to the computer to record where on the screen students are looking during the task. Interviewers will also take notes about your reactions to the social studies tasks so that tasks can be revised and refined. We will observe and ask you questions about your understanding of the tasks.

Where will the research interviews take place?

The interviews will take place at <NAME OF INTERVIEW LOCATION>, located at <LOCATION>, near <NEARBY LANDMARK OR METRO STATION>.

How long will the research interview last and when it will take place?

The interview will last up to 90 minutes and will require only one visit to our office. Interviews can be scheduled at your convenience.

Will you keep information private and confidential?

Will the results go to my school?

Your answers will not be disclosed to your school, and will not be linked to any personally identifiable information such as the school’s name.

What are the possible risks of being in the study?

There are no known risks to participating in this study.

Can participation end early?

Your participation in this study is completely voluntary. You have the right to refuse to answer particular questions. You may elect to withdraw from this study at any time.

Whom can I contact with questions or for further information?

<CONTACT NAME> is coordinating this study at EurekaFacts on behalf of NCES. If you have any questions, please call him/her at <CONTACT PHONE> or by email <CONTACT EMAIL>.

By signing below, you agree that you will participate in this study. You will receive a $25 gift card (from a major credit card company) at the conclusion of the session.

In addition, if transportation to the site is involved, your parent/guardian will also receive a $25 gift card as appreciation for bringing you to and from the research site.

Your signature below will indicate that your questions have been answered satisfactorily, and that you have read and understood the information provided above.

Print Name: _________________________________________________________________________

Signature: ___________________________________________________________________________

Date:_______________________________________________________________________________

School that you attend: ________________________________________________________________

To be completed by ETS staff:

Student Payment Card Transaction Number: ________________________________________________________

Parent/Guardian Payment Card Transaction Number: _________________________________________________

Appendices – Tryouts

Appendix BM: Recruitment Screener – Parents/Legal Guardians of Student Participants (Under Age 18): Social Studies Tryouts

Script when calling parents from previously compiled list:

Good morning/afternoon/evening. Is this the <NAME FROM CALL LIST> household? My name is <NAME>, and I am calling from EurekaFacts. We are a local research company carrying out a study on behalf of the National Center for Education Statistics (NCES), part of the U.S. Department of Education. We are recruiting students to sign up to participate in a research study supporting the development of new social studies tasks in civics, geography, and U.S. history for the National Assessment of Educational Progress, or NAEP. [If parent asks what this is: This national assessment is administered to students in elementary and middle schools throughout the United States.]

Are there any students in your household?

Yes: (Great! Let me give you some additional information about the study.)

No: (Thank you, have a good day/evening.)

We are carrying out question tryout sessions to explore how students work through sample social studies test questions to ensure that they are clear and understandable to a broad range of students. If selected to participate, your child will receive a $25 gift card (from a major credit card company). You will also receive a $25 gift card to thank you for your time in bringing your child to and from the session site.

The question tryout session will take place at our office in Rockville, MD. It will last no more than 60 minutes and can be scheduled at your convenience. During the session, your child will be asked to respond to test questions on a computer. NCES simply wants to know how students would answer the questions if they were to be included on an actual NAEP assessment.

These question tryout sessions are an important stage of the NAEP test development process. By sharing their feedback with us, students can directly impact the future of this nationwide assessment by helping NCES evaluate the test questions to create the best test possible. Do you believe your child would be interested in participating?

Yes: (Great! To ensure that we conduct this research with a broad mix of students, I have a few brief questions to ask about you and your child. It should only take a few minutes of your time. First, I want to collect some additional information about your child to identify if <HE/SHE> meets the participation requirements.

(Skip to Screener, on following page.)

No: (Thank you, have a good day/evening. END CALL.)

Script when calling a parent who responded to email, advertisement, or invitation

Good morning/afternoon/evening. Is this <NAME OF STUDENT>? My name is <NAME>. I’m calling from EurekaFacts on behalf of the National Center for Education Statistics (NCES), part of the U.S. Department of Education.

Thank you for contacting us about the possibility of your child participating in a tryout session to help develop test questions for the National Assessment of Educational Progress, or NAEP. To ensure that we conduct research with a broad mix of students, I have a few brief questions to ask about you and your child. It should only take a few minutes of your time. First, I want to collect some additional information about your child to identify if <HE/SHE> meets the participation requirements.

Screener

What is your child’s name?

What grade is your child currently in? (INTERVIEWER: DO NOT READ LIST)
- K –7th  (Unfortunately, we are only interviewing students currently in grade 8 or grade 12 for this study. Thank you for your time. Have a good day/evening.)
- 8th
- 9th – 11th  (Unfortunately, we are only interviewing students currently in grade 8 or grade 12 for this study. Thank you for your time. Have a good day/evening.)
- 12th

What school does your child attend? (INTERVIEWER: Note if homeschooled)

____________________________

What school district does he/she attend? (INTERVIEWER: Note if parochial, charter, homeschooled)

____________________________

Is your child currently enrolled in any of the following types of courses (select all that apply)

Advanced placement (i.e., AP courses)

Honor courses

Remedial courses

Other courses

What is your race or ethnicity? (INTERVIEWER: Read list only if respondent doesn’t immediately reply. “Are you…?”)

Ethnicity:  Hispanic or Latino  Not Hispanic or Latino

Race (one or more of the categories below):

American Indian or Alaska Native

Asian

Black or African American

Hispanic or Latino

Native Hawaiian or other Pacific Islander

White or Caucasian

Just to be sure we're interviewing a broad range of students for this study, do you participate in the National School Lunch program?

[If the respondent indicates they prefer not to disclose this information, the interviewer should say “That’s fine” and move on to the next question.]

Would you consider the area you live in to be …
- Urban
- Suburban
- Rural

Ok. Thank you and what is the zip code of the area that you live in?

_________________

Thank you for your time. Is the phone number you gave me earlier the best number to call?

Other phone numbers: ____________________________

Is there a good time for me to call? When would be best for you?

Parent Name _____________________________________________

Child Name _______________________________________________

Date ____________________________________________________

Time ____________________________________________________

SCRIPT FOR 8th or 12th GRADE STUDENTS

What grade are you currently in? (RECRUITER: DO NOT READ LIST)

 K – 7^th  (Unfortunately, we are interviewing only students currently in grade 8 or grade 12 for this study. Thank you for your time. Have a good day/evening.)

 8^th – Good.

 9^th – 11^th (Unfortunately, we are interviewing only students currently in grade 8 or grade 12 for this study. Thank you for your time. Have a good day/evening.)

 12^th – Good.

Thank you. For the research interview, you will be asked to visit <LOCATION>, <ADDRESS> and respond to sample social studies tasks on a tablet computer. Your responses will not be graded. The National Center for Education Statistics simply wants to know how you would do each task if it were to be included in the National Assessment of Educational Progress. The tasks focus on civics, geography, and U.S. history and are interesting and fun to do. Your input will help evaluate the test that will be given to students throughout the nation. To thank you for participating, you will receive a $25 gift card (from a major credit card company). The interview will last no more than 60 minutes. Your participation is totally voluntary. If you participate, no one but the study’s researchers will see your interview responses and all other interview materials, and we will not use your name to identify you.

Would you like to participate?

 Yes

 No  (THANK AND END CALL)

If NO: Thank you for your time. I will need to let your parent/legal guardian know that you would not like to participate. Is he/she available?

TO PARENT/LEGAL GUARDIAN: Your child has decided not to participate, but I would like to thank you very much for taking the time to speak with us about this opportunity.

 Yes

 No

INTERVIEWER: IF CALLING PARENT/LEGAL GUARDIAN BACK, RECORD NAME, PHONE NUMBER, DATE/TIME

Parent/Legal Guardian Name: _______________________________________________________

Child Name: _____________________________________________________________________

Parent/Legal Guardian Phone: ______________________________________________________

Date: __________________________________________________________________________

Time: __________________________________________________________________________

IF PARENT/LEGAL GUARDIAN AVAILABLE AND CURRENTLY SCHEDULING INTERVIEWS

Date ____________________

Time ____________________

Thank you. We will follow up with a confirmation email/letter and a consent form. What is your email or postal address for us to send these to?

____________________________________________________________________________________

IF PARENT/LEGAL GUARDIAN AVAILABLE AND NOT CURRENTLY SCHEDULING INTERVIEWS

 Morning

 Afternoon

 After school

 Weekends

Thank you so much for talking with us today. We look forward to communicating with you soon.

Appendix BN: Recruitment Screener – Students Age 18 or Over: Social Studies Tryouts

Script when calling students from previously compiled list:

Good morning/afternoon/evening. Is this <NAME FROM CALL LIST>? My name is <NAME>, and I am calling from EurekaFacts. We are a local research company carrying out a study on behalf of the National Center for Education Statistics (NCES), part of the U.S. Department of Education. We are recruiting students to sign up to participate in a research study supporting the development of new social studies test questions for the National Assessment of Educational Progress, or NAEP. [If student asks what this is: This national assessment is administered to students in elementary and middle schools throughout the United States.]

Are you a student over the age of 18?

Yes: (Great! Let me give you some additional information about the study.)

No: (Thank you, have a good day/evening.)

We are carrying out question tryout sessions to explore how students work through sample social studies test questions to ensure that they are clear and understandable to a broad range of students. If selected to participate, you will receive a $25 gift card (from a major credit card company). If your parent or guardian accompanies you they will also receive a $25 gift card to thank them for your time in bringing you to and from the session site.

The question tryout session will take place at our office in Rockville, MD. It will last no more than 60 minutes and can be scheduled at your convenience. During the session, you will be asked to respond to test questions on a computer. NCES simply wants to know how students would answer the questions if they were to be included on an actual NAEP assessment.

These question tryout sessions are an important stage of the NAEP test development process. By sharing your feedback with us, students can directly impact the future of this nationwide assessment by helping NCES evaluate the test questions to create the best test possible. Would you be interested in participating?

Yes: (Great! To ensure that we conduct this research with a broad mix of students, I have a few brief questions to ask about you. It should only take a few minutes of your time. First, I want to collect some additional information to identify if you meet the participation requirements.

(Skip to Screener, on following page.)

No: (Thank you, have a good day/evening. END CALL.)

Script when calling students who responded to email, advertisement, or invitation

Thank you for contacting us about the possibility of you participating in a tryout session to help develop test questions for the National Assessment of Educational Progress, or NAEP. To ensure that we conduct research with a broad mix of students, I have a few brief questions to ask you. It should only take a few minutes of your time. First, I want to collect some additional information to identify if you meet the participation requirements.

Screener

What is your name?

What grade are you currently in? (INTERVIEWER: DO NOT READ LIST)

 K –7th  (Unfortunately, we are only interviewing students currently in grade 8 or grade 12 for this study. Thank you for your time. Have a good day/evening.)

 8th  (Unfortunately we need to speak to your parent or guardian. Are they available?)

 9th – 11th  (Unfortunately, we are only interviewing students currently in grade 8 or grade 12 for this study. Thank you for your time. Have a good day/evening.)

 12th

What school do you attend? (INTERVIEWER: Note if homeschooled)

____________________________

What school district do you attend? (INTERVIEWER: Note if parochial, charter, homeschooled)

____________________________

Are you taking any of the following types of courses (select all that apply)

Advanced placement (i.e., AP courses)

Honor courses

Remedial courses

Other courses

What is your race or ethnicity? (INTERVIEWER: Only read list if parent doesn’t immediately reply. Are you…?)

American Indian or Alaska Native
Asian
Black or African American
Hispanic or Latino
Native Hawaiian or other Pacific Islander
White or Caucasian

Just to be sure we're interviewing a broad range of students for this study, do you participate in the National School Lunch program?

[If the respondent indicates they prefer not to disclose this information, the interviewer should say “That’s fine” and move on to the next question.]

Would you consider the area you live in to be …

 Urban

 Suburban

 Rural

Ok. Thank you and what is the zip code of the area that you live in?

_________________

Thank you for your time. Is the phone number you gave me earlier the best number to call?

Other phone numbers: ____________________________

Is there a good time for me to call? When would be best for you?

Participant Name _______________________________

Date _________________________________________

Time _________________________________________

Appendix BO: EurekaFacts Consent Form for Parents/Legal Guardians of Student Participants (Under Age 18): Social Studies Tryouts

PARENTAL CONSENT FORM

FOR PARTICIPATION IN RESEARCH

EurekaFacts, LLC

Title of Research Study: Tryouts for NAEP Social Studies: Civics, Geography, and U.S. History

What is this study about?

The National Center for Education Statistics (NCES), part of the U.S. Department of Education, is conducting a study with students currently in grade 8 or grade 12 to try out new social studies test questions for the National Assessment of Educational Progress (NAEP). NCES needs to know if the questions on the test are clear, specifically questions on civics, geography, and U.S. history. Your child’s input will help NCES improve the test so that students like your child will understand it.

What will happen during the tryout?

Each participant will take an early version of a new NAEP social studies test on a computer. Participants will be asked to answer the questions as if it were an actual test. Participants will not be graded.

Where will the tryouts take place?

The tryouts will take place at <NAME OF TRYOUT LOCATION>, located at <LOCATION>, near <NEARBY LANDMARK OR METRO STATION>.

How long will the tryouts last and when will it take place?

The tryout session will last no more than 60 minutes and can be scheduled at your convenience. Each tryout requires only one visit to our office.

Whose consent do you require for participation?

For a student under age 18: Once we have received informed consent from the parent/legal guardian for their eligible student to participate, we will then also ask the student for his or her consent to participate. Participation in the study is completely voluntary. The parent/guardian or student may withdraw consent for the student’s participation in the study at any time. The student also has the right to refuse to answer particular questions. A student age 18 or above can sign the consent form directly, but again participation in the study is completely voluntary, and that student has the right to withdraw intent to participate and/or refuse to answer particular questions.

Will you keep information private and confidential?

All of your child’s responses will be used only for research purposes and will not be disclosed or used, in identifiable form, for any other purpose except as required by law [Education Sciences Reform Act of 2002 (ESRA 2002) 20 U.S.C., § 9573]. Your child will be assigned a unique student identifier (ID), and at no time will his/her name be linked to any of his/her answers. The tryouts will be audio and video recorded, as long as you agree, and are also used only for research purposes.

Will the results go to my child’s school?

Your child’s answers will not be disclosed to his/her school and will not be linked to any personally identifiable information such as the school’s name.

Are there are rewards or incentives for participating?

Yes, the student will receive a $25 gift card (from a major credit card company) from us at the end of his/her tryout. The student’s parent/guardian will also receive a $25 gift card to thank him/her for bringing their child to and from the tryout site.

What are the possible risks of being in the study?

There are no known risks to participating in this study.

Can participation end early?

Whom can I contact with questions or for further information?

<CONTACT NAME> is coordinating this study at EurekaFacts on behalf of NCES. If you have any questions, please call him/her at <CONTACT PHONE> or by email <CONTACT EMAIL>.

In addition, if transportation to the site is involved, you will also receive a $25 gift card as appreciation for the time bringing your child to and from the research site.

Your signature below will indicate that your questions have been answered satisfactorily, and that you have read and understood the information provided above.

Print Name: _________________________________________________________________________

Signature: ___________________________________________________________________________

Date:_______________________________________________________________________________

School that student attends: ____________________________________________________________

To be completed by ETS staff:

Student Payment Card Transaction Number: ________________________________________________________

Parent/Guardian Payment Card Transaction Number: _________________________________________________

Appendix BP: EurekaFacts Consent Form for Student Participants (Age 18 or Over): Social Studies Tryouts

CONSENT FORM

FOR PARTICIPATION IN RESEARCH

EurekaFacts, LLC

Title of Research Study: Tryouts for NAEP Social Studies: Civics, Geography, and U.S. History

What is this study about?

The National Center for Education Statistics (NCES), part of the U.S. Department of Education, is conducting a study with students currently in grade 8 or grade 12 to try out new social studies test questions for the National Assessment of Educational Progress (NAEP). NCES needs to know if the questions on the test are clear, specifically questions on civics, geography, and U.S. history. Your input will help NCES improve the test so that students like you will understand it.

What will happen during the research interview?

Where will the tryouts take place?

The tryouts will take place at <NAME OF TRYOUT LOCATION>, located at <LOCATION>, near <NEARBY LANDMARK OR METRO STATION>.

How long will the tryouts last and when will it take place?

The tryout session will last no more than 60 minutes and can be scheduled at your convenience. Each tryout requires only one visit to our office.

Whose consent do you require for participation?

For a student under age 18: Once we have received informed consent from the parent/legal guardian for their eligible student to participate, we will then also ask the student for his/her consent to participate. Participation in the study is completely voluntary. The parent/legal guardian or student may withdraw consent for the student’s participation in the study at any time. The student also has the right to refuse to answer particular questions. A student age 18 or above can sign the consent form directly, but again participation in the study is completely voluntary, and that student has the right to withdraw intent to participate and/or refuse to answer particular questions.

Will you keep information private and confidential?

All of your responses will be used only for research purposes and will not be disclosed or used, in identifiable form, for any other purpose except as required by law [Education Sciences Reform Act of 2002 (ESRA 2002) 20 U.S.C., § 9573]. You will be assigned a unique student identifier (ID), and at no time will your name be linked to any of your answers. The tryouts will be audio and video recorded, as long as you agree, and are also used only for research purposes.

Will the results go to my school?

Your answers will not be disclosed to your school and will not be linked to any personally identifiable information such as the school’s name.

Are there are rewards or incentives for participating?

What are the possible risks of being in the study?

There are no known risks to participating in this study.

Can participation end early?

Your participation in this study is completely voluntary. You have the right to refuse to answer particular questions. You may elect to withdraw from this study at any time.

Whom can I contact with questions or for further information?

<CONTACT NAME> is coordinating this study at EurekaFacts on behalf of NCES. If you have any questions, please call him/her at <CONTACT PHONE> or by email <CONTACT EMAIL>.

By signing below, you agree that you will participate in this study. You will receive a $25 gift card (from a major credit card company) at the conclusion of the session.

In addition, if transportation to the site is involved, your parent/guardian will also receive a $25 gift card as appreciation for the time bringing you to and from the research site.

Your signature below will indicate that your questions have been answered satisfactorily, and that you have read and understood the information provided above.

Print Name: _________________________________________________________________________

Signature: ___________________________________________________________________________

Date:_______________________________________________________________________________

School that you attend: ____________________________________________________________

To be completed by ETS staff:

Student Payment Card Transaction Number: ________________________________________________________

Parent/Guardian Payment Card Transaction Number: _________________________________________________

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Title	NATIONAL CENTER FOR EDUCATION STATISTICS
Author	joc
File Modified	0000-00-00
File Created	2021-01-27

National Assessment of Educational Progress (NAEP) Social Studies Pretesting

NCES Cognitive, Pilot, and Field Test Studies System

Appendices BA-BP NAEP Social Studies Pretesting Screeners & Consent Forms