National Assessment of Educational Progress (NAEP) Grade 8 Social Sciences Interactive Item Components (IICs) Pretesting

I am contacting you from EurekaFacts, a local research company. We are carrying out an important study for the National Center for Education Statistics (NCES), part of the U.S. Department of Education. We are contacting you to ask for your help in spreading the word about this exciting research opportunity.

We are looking for students in grade eight (ninth graders will also be considered) to help us test new computer-delivered social science questions in civics, geography, and U.S. history. These questions will be part of the National Assessment of Educational Progress (NAEP) delivered to students nationwide. It is important for us to have student feedback to develop the best ways to assess students’ skills in the social sciences.

[For Cognitive Interviews: The voluntary research interviews will last no more than 90 minutes. Students will be asked to answer civics, geography, or U.S. history questions and then give feedback. EurekaFacts staff may record audio and screen activity for analysis and take notes about students’ reactions to the questions in order to revise and refine test content.]

[For Tryouts: The voluntary research sessions will last no more than 90 minutes. The tryout sessions will explore how students work through civics, geography, or U.S. history test questions on a tablet computer to ensure that the questions are clear and understandable to a broad range of students. EurekaFacts staff may record audio to capture students’ reactions to the questions in order to revise and refine test content.]

If selected to participate, students will receive a $25 gift card from a major credit card company to thank them for their time. In addition, if transportation to the session site is involved, the student’s parent/legal guardian will also receive a $25 gift card as a thank-you for bringing his or her child to and from the interview site.

Students will have a key role in helping us improve our tests and will enhance the experience of thousands of students all over the United States who will take the tests. Student responses may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151).

If you are willing to help us recruit students for this study, please contact me, and I will follow up with flyers for you to distribute. I have included our Frequently Asked Questions (FAQs) sheet with additional information about this research study.

We appreciate your consideration of this request, and I invite you to please contact me with any questions regarding this project. I look forward to communicating with you.

Thank you,

EurekaFacts, LLC

Attachment: Frequently Asked Questions (FAQs) (See Appendix H)

Appendix B: Recruitment Email/Letter to After School Programs/Parent-Teacher-Student Associations/Youth Organizations: Social Science Cognitive Interviews and Tryouts

IF SENT AS EMAIL–Subject: Research Study Opportunity

<DATE>

Dear <NAME> [or Hello],

I am contacting you from EurekaFacts, a local research company. We are carrying out an important study in the social sciences for the National Center for Education Statistics (NCES), part of the U.S. Department of Education. We would like to ask for your help in spreading the word about this exciting research opportunity.

The voluntary research sessions will last no more than 90 minutes. Students will be asked to answer questions on civics, geography, or U.S. history and then give feedback about their experiences with the questions.

[For Cognitive Interviews: EurekaFacts staff may record audio and screen activity for analysis and take notes about students’ reactions to the questions to revise and refine test content.]

[For Tryouts: EurekaFacts staff may record audio to capture students’ reactions to the questions in order to revise and refine test content.]

I have included a flyer with a short description of this research study in the hope that you might pass it along to anyone who may be interested. Upon request, I can also send you our Frequently Asked Questions (FAQs) sheet with additional information about this research study.

We appreciate your consideration of this request, and I invite you to please contact me with any questions regarding this project. I look forward to communicating with you.

Thank you,

EurekaFacts, LLC

Attachment: Flyer (See Appendix N)

Appendix C: Recruitment Email/Letter to Parent(s)/Legal Guardian(s) of Student Participants (Under Age 18): Social Science Cognitive Interviews and Tryouts

(To be used only if the required number of participants is not recruited through schools and organizations and direct outreach to parents/legal guardians is necessary.)

IF SENT AS EMAIL–Subject: Research Study Opportunity

<DATE>

Dear <NAME>,

I am contacting you from EurekaFacts, a local research company. We are carrying out an important research study in the social sciences (civics, geography, and U.S. history) for the National Center for Education Statistics (NCES), part of the U.S. Department of Education.

We are looking for students currently in grade eight (ninth graders will also be considered) to help us test new computer-delivered questions in the social sciences for the National Assessment of Educational Progress (NAEP).

The <interviews or tryout sessions> will explore how students work through the questions to ensure that they are clear and understandable to a broad range of students. [For Cognitive Interviews: EurekaFacts staff may record audio and screen activity for analysis and take notes about students’ reactions to the questions to revise and refine test content.] [For Tryouts: EurekaFacts staff may record audio to capture students’ reactions to the questions in order to revise and refine test content.] NCES simply wants to know what your child thinks about each question and how your child might answer the questions on the test. These interviews are an important stage of the NAEP test development process. By sharing their feedback with us, students can help the future of this national assessment by helping NCES evaluate the questions to create the best tests possible.

The research session for your child would take place at <LOCATION> and would last no more than 90 minutes. The session can be scheduled at your and your child’s convenience. Your child will receive a $25 gift card from a major credit card company as a thank-you for his or her time. In addition, if transportation to the interview site is involved, you will also receive a $25 gift card as appreciation for bringing your child to and from the session site.

Student responses may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151).

I have included a Frequently Asked Questions (FAQs) sheet with additional information about this important research study. If your child is interested in participating in this study, please fill out an intake form at <LINK> or call/email me at <CONTACT PHONE> or <CONTACT EMAIL>. We look forward to communicating with you soon!

Thank you,

EurekaFacts, LLC

Attachment: Frequently Asked Questions (FAQs) (See Appendix H)

Appendix D: Confirmation (Email/Letter) to Schools/After School Programs/Parent-Teacher-Student Associations/Youth Organizations Where Interviews will be Administered: Social Science Cognitive Interviews and Tryouts

IF SENT AS EMAIL–Subject: Confirmation of NAEP Research Study

<DATE>

Dear <NAME>,

Thank you for agreeing to assist EurekaFacts in trying out new social science test questions in civics, geography, and U.S. history for the National Assessment of Educational Progress (NAEP).

Before giving the new questions to students in an assessment, the National Center for Education Statistics (NCES), part of the U.S. Department of Education, needs to know whether the questions are clear and what students think about the test. Students will be asked to respond to questions on a tablet computer and will be asked about the questions.

[For Cognitive Interviews (add to above paragraph): Students’ feedback and responses will be noted and audio recorded, and screen activity may be captured and used for analysis to help us revise and improve our tests.]

[For Tryouts (add to above paragraph): Students’ feedback and responses will be noted and audio recorded and used for analysis to help us revise and improve our tests.]

Research sessions have been scheduled at <SCHOOL/ORGANIZATION/ASSOCIATION> from <TIME> to <TIME> on <DAY, DATE>. The sessions will each last no more than 90 minutes. To thank them for their time, participating students will each receive a $25 gift card from a major credit card company.

[Include the following information, if applicable, to above paragraph: If a parent/legal guardian transports the student to or from the session site, he or she will also receive a $25 gift card for his or her time and effort.]

Thank you for assisting EurekaFacts in this important research for NAEP. If you have any questions, please contact me.

Thank you,

EurekaFacts, LLC

Appendix E: Confirmation of Interview (Email/Letter) to Parent(s)/Legal Guardian(s) of Student Participants (Under Age 18): Social Science Cognitive Interviews and Tryouts

(To be used if sessions will not take place at a school.)

IF SENT AS EMAIL–Subject: Confirmation of Your Child’s NAEP Research Study Session

<DATE>

Dear <NAME>,

Thank you for agreeing to allow your child to participate in a study to give us feedback on new social science test questions in civics, geography, and U.S. history for the National Assessment of Educational Progress (NAEP). Before giving the new questions to students in a nationwide assessment, the National Center for Education Statistics (NCES), part of the U.S. Department of Education, needs to know whether the questions are clear. Students will be asked to respond to social science questions on a tablet computer and asked about the questions. Your child’s input will help NCES improve the questions so that students understand them.

[For Tryouts (add to above paragraph): Students’ feedback and responses will be noted and audio recorded and used for analysis to help us revise and improve our tests.]

Your child has been scheduled for a research session at:

<DAY, DATE>

<TIME>

The research session will last no more than 90 minutes. Your child will receive a $25 gift card from a major credit card company as a thank-you for participating. In addition, if transportation to the research session site is involved, you will also receive a $25 gift card as appreciation for bringing your child to and from the site.

We have attached a consent form <to or with> this <email or letter> reminder. Please sign, date, and return the form when your child comes in for the research session, giving consent for your child to participate in the NAEP research. Without your consent, your child will not be able to participate.

Thank you for allowing your child to volunteer in this important research for NAEP. Our interviewer is looking forward to meeting with your child. If you have any questions, please contact me.

Thank you,

EurekaFacts, LLC

Attachment: EurekaFacts Consent Form for Parents/Legal Guardians of Student Participants (Under Age 18) (See Appendix Q)

Appendix F: Thank You Email/Letter to Schools/After School Programs/Parent-Teacher-Student Associations/Youth Organizations of Student Participants: Social Science Cognitive Interviews and Tryouts

IF SENT AS EMAIL–Subject: Thank You for Your Participation in the NAEP Research Study

<DATE>

Dear <NAME>,

On behalf of the National Center for Education Statistics, part of the U.S. Department of Education, EurekaFacts would like to thank you for your <School’s/After-School Program’s/Parent-Teacher-Student Association’s/Youth Organization’s> participation in our research study. Thanks to the students at <NAME OF SCHOOL/AFTER-SCHOOL PROGRAM/PARENT-TEACHER-STUDENT ASSOCIATION/YOUTH ORGANIZATION>, we are able to develop the best possible test questions for the National Assessment of Educational Progress (NAEP).

Your <school’s/after-school program’s/parent-teacher-student association’s/youth organization’s> assistance helped us better understand how students approach the test questions. Students’ open, candid, and insightful comments were very helpful.

Again, thank you for your assistance and participation.

Sincerely,

EurekaFacts, LLC

Appendix G: Thank You Email/Letter to Parents/Legal Guardians of Student Participants (Under Age 18): Social Science Cognitive Interviews and Tryouts

IF SENT AS EMAIL–Subject: Thank You for Your Child[ren]’s Participation in the NAEP Research Study

<DATE>

Dear <NAME>,

On behalf of the National Center for Education Statistics (NCES), part of the U.S. Department of Education, EurekaFacts would like to thank you for allowing your child[ren] to participate in our research study. Thanks to your child[ren], we are able to develop the best possible test questions for the National Assessment of Educational Progress (NAEP).

Your child[ren]’s assistance helped us better understand how students approach the test questions. Students’ open, candid, and insightful comments were very helpful.

Again, thank you for your assistance.

Sincerely,

<TITLE>

EurekaFacts, LLC

Appendix H: Frequently Asked Questions (FAQs): Social Science Cognitive Interviews and Tryouts

Frequently Asked Questions (FAQs)

NAEP Research Study

What is this study about?

The National Center for Education Statistics (NCES), part of the U.S. Department of Education, is conducting research sessions with students currently in grade eight (ninth graders will also be considered) about new test questions for the National Assessment of Educational Progress (NAEP). The questions we are testing focus on the social sciences (civics, geography, and U.S. history) and will be part of a digitally based test. NCES needs to know whether the questions on the test are clear. Students’ input will help NCES improve the test so that future students will understand it better.

What will happen during the research session?

[For Cognitive Interviews: Each participant will work through an early version of new NAEP social science questions in civics, geography, or U.S. history on a tablet computer. An interviewer will ask students what they are thinking as they complete the questions and whether they believe the questions are clear and understandable. EurekaFacts staff may record audio and screen activity for analysis and take notes about students’ reactions to these questions to revise and improve test content. No personally identifying information will be retained.]

[For Tryouts: Each participant will take an early version of a new NAEP social science test on a tablet computer. Participants will be asked to answer the questions as if it were an actual test. We may ask participants for their reactions to and feedback about the test questions. Audio recording may be used to capture students’ reactions and feedback. No personally identifying information will be retained.]

Social science-specific knowledge is not required, and participants’ responses will not be graded. There will be more than one adult present during each research session.

Where will the research session take place?

The research session will take place at <NAME OF INTERVIEW/SESSION LOCATION>, located at <ADDRESS>, near <NEARBY LANDMARK or METRO STATION>.

How long will the research session last and when will it take place?

The research session will last no more than 90 minutes and can be scheduled at your convenience. Each research session requires only one visit to our office.

Whose consent do you require for participation?

For students under the age of 18, we require written, informed consent from a parent/legal guardian for the eligible student to participate. We also ask the student for his or her verbal consent to participate during the prescreening process. Participation in the study is completely voluntary. The parent/legal guardian or student may withdraw consent for the student’s participation in the study at any time. The student also has the right to refuse to answer questions.

Will information be kept private and confidential?

Each participant will be assigned a unique student identifier (ID), and at no time will his or her name be linked to any of his or her answers. Your child’s responses may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151). [For Cognitive Interviews: Audio and screen activity will be recorded for analysis, but no personally identifying information will be retained.] [For Tryouts: Audio may be recorded for analysis, but no personally identifying information will be retained.]

Will the results go to the participants’ schools?

Participants’ responses will not be disclosed to their schools and will not be linked to any personally identifiable information such as the school’s name.

Are there rewards or incentives for participating?

Yes, the student will receive a $25 gift card from a major credit card company at the end of his or her session. If transportation is required to the research session site, the student’s parent/legal guardian will also receive a $25 gift card in appreciation for bringing his or her student to and from the site.

What are the possible risks of being in the study?

There are no known risks to participating in this study.

Can participation end early?

Students’ participation in this study is completely voluntary. Students have the right to refuse to answer questions. Students may choose to withdraw from this study at any time. Students will still receive a gift card if they refuse to answer questions or choose to end the research session early.

Whom may I contact with questions or for further information?

<CONTACT NAME> is coordinating this study at EurekaFacts on behalf of NCES. If you have any questions, please call <CONTACT PHONE> or email <CONTACT EMAIL>.

Appendix I: EurekaFacts Recruitment Email/Letter to Adults Who Have Requested Information from EurekaFacts on Research Study Opportunities: Social Science Cognitive Interviews and Tryouts

IF SENT AS EMAIL–Subject: Education Research Opportunity for Students in Grade 8

<DATE>

Hello,

I am contacting you from EurekaFacts, a local research company in Rockville, Maryland. You are receiving this email because you have previously given us your contact information in order to receive information regarding upcoming research opportunities.

If you are the parent/legal guardian of a child who is currently in grade eight (ninth graders will also be considered), or know anyone who is, we’d like to inform you that we are currently carrying out a research study on behalf of the National Center for Education Statistics (NCES), part of the U.S. Department of Education. This study supports the development of new social science test questions for the National Assessment of Educational Progress (NAEP).

We are carrying out research sessions to try out new computer-delivered social science questions in civics, geography, or U.S. history. Students will be asked to complete test questions on a tablet computer. The research sessions will explore how students work through the questions on a tablet computer to ensure that the questions are clear and understandable to a broad range of students. NCES simply wants to know how students would answer the questions if they were on an actual test and would like to hear students’ thoughts to improve the questions and make them easier to understand.

The voluntary research sessions will last no more than 90 minutes. Students will be asked to respond to civics, geography, or U.S. history questions and will be asked about them. EurekaFacts staff will take notes of students’ reactions to the questions, and students’ feedback will be used to revise and refine the questions. We will observe and ask about the students’ understanding of the questions. Students may also be asked whether they have any suggestions about how to improve the questions.

Each student who is selected to participate will receive a $25 gift card from a major credit card company as a thank- you for his or her participation. The student’s parent/legal guardian will also receive a $25 gift card to thank him or her for bringing his or her student to and from the site.

These research sessions are important in the NAEP test development process. Participating students can help the future of this nationwide assessment by helping NCES create the best test possible.

I have included a short description of this research study in the hope that you might pass it along to anyone who may be interested. Upon request, I can also send you our Frequently Asked Questions (FAQs).

Thank you for your consideration and assistance regarding this effort. Please do not hesitate to contact me with any questions you may have regarding our firm or this project. We look forward to communicating with you.

Sincerely,

EurekaFacts, LLC

Attachment: Flyer (See Appendix M/Appendix N)

Appendix J: EurekaFacts Recruitment Email/Letter to Parent-Teacher-Student Associations/Organizations/Schools That Have Requested Additional Information about Social Science Cognitive Interviews and Tryouts

IF SENT AS EMAIL–Subject: Education Research Opportunity for Students in Grade 8

<DATE>

Hello,

I am contacting you from EurekaFacts, a local research company in Rockville, Maryland. You are receiving this email because you have previously requested additional information about research studies on new social science test questions that we are carrying out on behalf of the National Center for Education Statistics (NCES), part of the U.S. Department of Education.

As we previously mentioned, we are currently recruiting students in grade eight (ninth graders will also be considered) for a research study supporting the development of new test questions for the National Assessment of Educational Progress (NAEP). We are carrying out research sessions to try out new computer-delivered social science questions in civics, geography, or U.S. history. Students will be asked to complete test questions on a tablet computer. The research sessions will explore how students work through the questions on a tablet computer to ensure that the questions are clear and understandable to a broad range of students. NCES simply wants to know how students would complete the questions if they were on an actual test and would like to hear their thoughts about how to improve the questions and make them easier to understand.

These research sessions are an important stage of the NAEP test development process. Participating students can help the future of this nationwide assessment by helping NCES create the best test possible.

Sincerely,

EurekaFacts, LLC

Attachment: Flyer (See Appendix N)

Appendix K: EurekaFacts Confirmation Phone Script for Parents/Legal Guardians of Student Participants (Under Age 18): Social Science Cognitive Interviews and Tryouts

Good <morning/afternoon/evening>. May I please speak to <NAME> about <STUDENT’S NAME(S)> <cognitive interview or tryout> session for the new National Assessment of Educational Progress, or NAEP, test questions?

Good <morning/afternoon/evening>. My name is <NAME>, and I’m calling from EurekaFacts about the NAEP research study that we are carrying out on behalf of the National Center for Education Statistics, or NCES, part of the U.S. Department of Education. We are pleased that you have agreed to let your child[ren] participate in this study and help develop the best possible test questions. This is a reminder that we have scheduled your child[ren]’s <cognitive interview(s) or tryout session(s)> for <TIME> on <DATE>.

The <cognitive interview(s) or tryout session(s)> will last no more than 90 minutes. It will take place at <LOCATION AND DIRECTIONS>.

During the <cognitive interview(s) or tryout session(s)>, your child[ren] will be asked to respond to sample social science questions in civics, geography, or U.S. history on a tablet computer. These <cognitive interviews or tryouts> are an important stage of the NAEP test development process. Your child[ren]’s participation will help NCES improve the test questions.

Your child[ren]’s responses may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required [Recruiter note: if asked what law is being referenced, please provide the following reference: law (20 U.S.C. §9573 and 6 U.S.C. §151.).]

As soon as your child[ren] <has or have> completed the <cognitive interview or tryout session>, <he or she or they> will receive a $25 gift card from a major credit card company as a thank-you for participating. You will also receive a $25 gift card to thank you for your time in bringing your child[ren] to and from the session location.

Please remember to bring the signed and dated consent form, which will give consent for your child[ren] to participate in the NAEP research when you bring your child[ren] in for the session. Without your consent, your child[ren] will not be able to participate in the <cognitive interview or tryout> session.

Do you have any questions at this time? [Answer any questions.]

We plan to see you at <TIME> on <DATE> at <LOCATION>. If you have any questions before then, please call <PHONE NUMBER>.

Thank you again for agreeing to be part of this very important research study.

Appendix L: EurekaFacts Confirmation Phone Script to School/Organization Where Sessions Will Be Conducted: Social Science Cognitive Interviews and Tryouts

Good <morning/afternoon/evening>. May I please speak to <NAME> about the <cognitive interview or tryout> sessions for the new National Assessment of Educational Progress, or NAEP, test questions?

[If a new person comes to the phone: Good <morning/afternoon/evening>.] My name is <NAME>, and I’m calling from EurekaFacts about the NAEP research study that we are carrying out on behalf of the National Center for Education Statistics, or NCES, part of the U.S. Department of Education. We are pleased that you will be assisting EurekaFacts in carrying out research studies to help develop the best possible test questions.

These <cognitive interviews or tryout sessions> are an important stage of the NAEP test development process. Students’ participation will help NCES improve the test questions. Their responses may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law. [Recruiter note: If asked what law is being referenced, please provide the following reference: law (20 U.S.C. §9573 and 6 U.S.C. §151.).]

Sessions have been scheduled at <SCHOOL/ORGANIZATION> from <TIME> to <TIME> on <DAY, DATE>. The sessions will each last approximately 90 minutes. To thank them for their time, participating students will receive a $25 gift card from a major credit card company. If a parent/legal guardian transports his or her student to or from the research site, the parent/legal guardian will also receive a $25 gift card for his or her time and effort.

Do you have any questions at this time? [Answer any questions.]

We plan to see you at <TIME> on <DATE> at <LOCATION>. If you have any questions before then, please call <PHONE NUMBER>.

Thank you again for assisting EurekaFacts in this very important research study.

Appendix M: EurekaFacts Recruitment Internet/Newspaper Advertisement/Flyer/ Advertisement Online: Social Science Cognitive Interviews and Tryouts

TEST THE TEST!

Do you have a child in 8th grade?

EurekaFacts, a local research company in Rockville, Maryland, is administering a study for the National Center for Educational Statistics (NCES), a part of the U.S. Department of Education.
We are looking for students in grade eight (ninth graders will also be considered) to help evaluate new test questions.
Each student will receive a $25 gift card from a major credit card company as a thank- you for participation, plus the parent/legal guardian will receive a $25 gift card if he or she brings the student to and from the study location.
The <research or tryout> session will last no more than 90 minutes and may be scheduled at your convenience after school or on weekends.
Student responses may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151).

Interested?

Call or text: <PHONE NUMBER>

Email: <EMAIL ADDRESS>

We are across the street from the Rockville Metro!

Appendix N: EurekaFacts Recruitment Informational Brochure Distributed Through Schools/Youth Organizations/Libraries/Parent-Teacher-Student Associations: Social Science Cognitive Interviews and Tryouts

TEST THE TEST!

Opportunity for Students in Grade 8!

EurekaFacts is a local research company administering an important study for the National Center for Education Statistics (NCES), part of the U.S. Department of Education. If your child is currently in grade 8 (9th graders will also be considered), we need his or her help to evaluate new social science test questions for the National Assessment of Educational Progress (NAEP), a nationwide educational test. Students’ participation will help to ensure that the test questions are clear and understandable to a broad range of students. Student responses may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151).

EurekaFacts will provide a $25 gift card from a major credit card company to each student who participates in the study, plus a $25 gift card to the parent/legal guardian who brings the student to and from the research session location for the 90-minute interview. Sessions will begin on <DATE> and may be scheduled at your convenience after school or on weekends. Please call, text, or email us now for more information about this opportunity!

Call or Text: <CONTACT PHONE> or Email: <CONTACT EMAIL>

Visit our website for more information: <WEBSITE INFORMATION>

Appendix O: Recruitment Phone Script for Schools of Potential Student Participants: Social Science Cognitive Interviews and Tryouts

Hello. May I please speak to <NAME OF CONTACT AT SCHOOL>?

[If a new person comes to the phone: <Good morning/afternoon/evening>.] My name is <CONTACT NAME>, and I am contacting you from EurekaFacts. We are carrying out an important study for the National Center for Education Statistics, or NCES, part of the U.S. Department of Education. We are recruiting students who are currently in grade eight (but will also consider ninth graders) to participate in research interviews supporting the development of new social science test questions for the National Assessment of Educational Progress, or NAEP. The research interviews will explore how students work through questions on a tablet computer to ensure that the questions are clear and understandable to a broad range of students. NCES simply wants to know how students would respond to the questions if they were on an actual test and would like to hear students’ thoughts about how to improve the questions and make them easier to understand.

The voluntary research <interviews or sessions> will last no more than 90 minutes. Students will be asked to respond to social science questions in civics, geography, or U.S. history. [For Cognitive Interviews: We will observe and ask about the students’ level of interest in, and understanding of, the questions. Students will also be asked whether they have any suggestions about how to improve the questions.] EurekaFacts research staff may record <audio and/or screen activity> for analysis and take notes of students’ reactions to these test questions and will subsequently use students’ feedback to revise and refine the test content.

Student responses may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law. [Recruiter note: if asked what law is being referenced, please provide the following reference: law (20 U.S.C. §9573 and 6 U.S.C. §151.).]

We would like your help in distributing and posting informational flyers. We will follow up directly with those who contact us. We will hold some sessions at <LOCATION>, <ADDRESS>; however, if the parents/legal guardians of students cannot provide transportation to <LOCATION>, <ADDRESS> we would like to set up sessions at your school.

Each participating student will receive a $25 gift card from a major credit card company in appreciation of his or her time. In addition, if transportation to the site is involved, a parent/legal guardian of each student will also receive a $25 gift card as appreciation for bringing his or her child to and from the interview site.

These <interviews or sessions> are an important stage of the NAEP test development process. The comments from students attending your school will help NCES improve test questions.

Would your school be interested in participating in or helping us to recruit students for this research study?

[If NO:] Thank you for your time. Have a wonderful <day or evening>. [END CALL]

[If YES:] Good, thank you very much. To ensure that we interview a broad mix of students and determine the best interview location, I have a few brief screening questions to ask you. It should take only a few minutes.

Screener Script

[Recruiter note: Determine recruitment needs in advance. Look in public documents and on schools’ websites to find what grades a school includes, the size of each grade, the socio-economic status, and the racial/ethnic makeup of the student body. Record all responses to screener questions.]

Are you at a school with a population that includes:

grade eight and/or grade nine students

How many students are enrolled in your school?

299 students or fewer
300-599 students
600 students or more

[End screener script at whatever point the school staff member gives a response confirming that his or her group has been sampled adequately.]

End screener script: Based on the requirements of this study, we are not able to include your school in the interviews at this time. Thank you for your time. Have a good <day or evening>. [End call]

Eligible Participant Script

Thank you for taking the time to answer the screening questions. Your school is generally eligible to participate in the research interviews.

Would you be able to distribute and post flyers regarding this study?

(After receiving a contact of interest, a EurekaFacts staff member would follow up with the parent/legal guardian by phone and ask him or her some screening questions to ensure that a diverse sample of students is included in this study.)

[If NO:] Thank you for your time. Have a good <day or evening>. [End call]

[If YES:] Would you be able to make a room at your school available where EurekaFacts could conduct some 90-minute research sessions?

[If YES:] That’s great. We will contact you to schedule sessions at your school after we have some students signed up. (Go to Closing)

[If NO:] (Go to Closing)

Closing Script

Thank you for agreeing to disseminate information about this important research study. We will send you an email with the informational flyers attached. What is your preferred email address? [Recruiter note: Record preferred email address on line.] ________________________________________________

We may need to reach you by telephone. Is the number I called the best number to use?

Do you have any questions at this time?

[Answer any questions.]

If you have any <additional> questions, please call <PHONE NUMBER>.

Thank you again for agreeing to be part of this important research project. Have a wonderful <day or evening>.

Appendix P: Recruitment Phone Script for Parents/Legal Guardians of Student Participants (Under Age 18): Social Science Cognitive Interviews and Tryouts

Script When Calling Parents from Previously Compiled List:

Hello. May I please speak to <NAME OF Parent/Legal Guardian>? [Thank and end call if not available.]

[If another person comes to the phone: Good <morning/afternoon/evening>.] Is this the <NAME FROM CALL LIST> household? My name is <NAME>, and I am calling from EurekaFacts, a local research company carrying out a study on behalf of the National Center for Education Statistics, or NCES, part of the U.S. Department of Education.

Are there any students in your household?

[If YES:] Great! Let me give you some additional information about the study.

[If NO:] Thank you. Have a good <day or evening>.

We are recruiting students to participate in a research study to help the development of new questions in the social sciences for the National Assessment of Educational Progress, or NAEP. [If parent asks what NAEP is: This national assessment is administered to students in elementary, middle, and high schools throughout the United States.]

(Skip to Screener Script.)

Script When Calling a Parent Who Responded to Email, Advertisement, or Invitation:

Good <morning/afternoon/evening>. Is this < NAME OF Parent/Legal Guardian>? My name is <NAME>. I’m calling from EurekaFacts on behalf of the National Center for Education Statistics, or NCES, part of the U.S. Department of Education.

Thank you for contacting us about the possibility of your child participating in a tryout session to help develop test questions for the National Assessment of Educational Progress, or NAEP.

Screener Script

We are carrying out <research interviews or sessions> to explore how students work through sample social science test questions to make sure that they are clear and understandable to a broad range of students. If selected to participate, your child will receive a $25 gift card from a major credit card company as a thank-you for his or her participation. You will also receive a $25 gift card to thank you for bringing your child to and from the session site.

The <research interview or session> will take place at <LOCATION>. It will last no more than 90 minutes and can be scheduled at your convenience.

[For Cognitive Interviews: We will observe and ask about the students’ level of interest in and understanding of the questions. Students will also be asked whether they have any suggestions about how to improve the questions. EurekaFacts staff may record audio and screen activity for analysis and take notes about students’ reactions to the questions in order to revise and refine test content. No personally identifying information will be retained.]

[For Tryouts: EurekaFacts staff may record audio and take notes to capture students’ reactions to the questions in order to revise and refine test content. No personally identifying information will be retained. NCES simply wants to know how students would answer the questions if they were to be included on an actual NAEP assessment.]

These research sessions are an important stage of the NAEP test development process. The feedback students provide can help NCES evaluate the test questions to create the best test possible. Do you think your child would be interested in participating?

[If YES:] Great! To ensure that we conduct this research with a broad mix of students, I have a few brief questions to ask about you and your child. It should only take a few minutes of your time. First, I want to collect some additional information about your child to identify if <HE or SHE> meets the participation requirements.

[If NO:] Thank you, have a good <day or evening>. [END CALL]

[Determine recruitment needs in advance. Record all responses. End screener at whatever point the parent/legal guardian gives a response that confirms that their group has already been sampled adequately. End Screener script: Thank you for answering my questions. Based on the requirements of this study, we are not able to include your child in the interviews at this time. Thank you for your time. Have a good <day or evening>. [End call]]

What is your child’s name?

____________________________

What grade is your child currently in? (DO NOT READ THE LIST.)
- K–7^th  [Unfortunately, we are only interviewing students currently in grades eight or nine for this study. Thank you for your time. Have a good <day or evening>.]
- 8^th–9^th (Good.)
- 10^th–12^th  [Unfortunately, we are only interviewing students currently in grades eight or nine for this study. Thank you for your time. Have a good <day or evening>.]

What school does your child attend? (Note if homeschooled.)

____________________________

What school district does he/she attend? (Note if charter, parochial, private, or homeschooled.)

____________________________

Is your child currently enrolled in any of the following types of courses? (Select all that apply.)

Advanced placement (e.g., AP, IB, or other college-level courses)

Honor courses

Remedial courses

Other courses

What is your child’s ethnicity and race? (Read list only if respondent doesn’t immediately reply: Is your child…? If the respondent indicates they prefer not to disclose this information, say: That’s fine. and move on to the next question.)

Ethnicity:

 Hispanic or Latino  Not Hispanic or Latino

Race (one or more of the categories below):

American Indian or Alaska Native

Asian

Black or African American

Native Hawaiian or other Pacific Islander

White or Caucasian

Which language did your child learn when he/she first began to talk? ______________________________

Which language does your child most frequently speak at home? _________________________________

Which language do you (the parent/legal guardian) most frequently use when speaking with your child? ______________________________________________________________________________________

[If the parent/legal guardian asks why you are asking this question respond with: Just to be sure we're interviewing a broad range of students for this study.]

What is your total annual household income range based on the following ranges? (If the respondent indicates they prefer not to disclose this information, say: That’s fine. and move on to the next question.)

Less than $20,000
$20,000 to $49,999
$50,000 to $99,999
$100,000 or more

Does your child participate in the National School Lunch program? [If the respondent indicates they prefer not to disclose this information, say [That’s fine.] and move on to the next question.]

 Yes

 No

Would you consider the area you live in to be…
- Urban
- Suburban
- Rural

Okay. Thank you and what is the zip code of the area that you live in?

_________________

Thank you. We would like to speak with your child to confirm if <he or she> is interested in participating in the research interview. It will take only a couple of minutes, and then I will ask to speak with you again for another moment. Is your child available to speak with us now or should we call <he or she> back at another time?

 Yes, the child is available for screening questions.  (Skip to script for eighth grade students)

 No, please call back.

Is the phone number you gave me earlier the best number to call?

Other phone numbers: _________________________________

Is there a good time for me to call? When would be best for you?

___________________________________________________

Interviewer: IF CHILD NEEDS TO RECEIVE A CALL BACK, RECORD NAME, PHONE NUMBER, DATE/TIME.

Parent/Legal Guardian Name: ______________________________________________________

Child Name: ____________________________________________________________________

Parent/Legal Guardian Phone: ______________________________________________________

Date: ____________________

Time: ____________________

Script for Grade Eight (Grade Nine) Students

Good <morning/afternoon/evening>. Is this <NAME OF STUDENT>? My name is <NAME>, and I’m calling from EurekaFacts. We are carrying out an important study for the National Center for Education Statistics, part of the U.S. Department of Education.

We have spoken to your parent/legal guardian and we are pleased that you may be interested in helping us test questions in the social sciences for a nationwide student assessment, the National Assessment of Educational Progress, or NAEP. Before I tell you more about what you would be asked to do, I need to ask you one question:

What grade are you currently in? (DO NOT READ THE LIST.)

 K–7^th  [Unfortunately, we are interviewing only students currently in grades eight or nine for this study. Thank you for your time. Have a good <day or evening>.]

 8^th–9^th  (Good.)

 10^th–12^th [Unfortunately, we are interviewing only students currently in grades eight or nine for this study. Thank you for your time. Have a good <day or evening>.]

Thank you. For the research <interview or session>, you will be asked to visit <LOCATION>, <ADDRESS> and respond to sample social science items on a <tablet computer or computer>. Your responses will not be graded. The National Center for Education Statistics simply wants to know how you would respond to each question if they were to be included in the National Assessment of Educational Progress. The questions focus on the social sciences and are interesting and fun to do. Your input will help evaluate the test that will be given to students throughout the nation. As a thank-you for participating, you will receive a $25 gift card from a major credit card company and your parent or legal guardian will also receive a $25 gift card if he or she brings you to and from the session site. The session will last no more than 90 minutes and will be recorded. Your participation is totally voluntary. If you participate, no one but the study’s researchers will see your responses and all other session materials, and we will not use your name to identify you.

Would you like to participate?

 Yes  (See script below “If YES.”)

 No  [Thank you for your time. I will need to let your parent/legal guardian know that you would not like to participate. Is <he or she> available?]

TO PARENT/LEGAL GUARDIAN: Your child has decided not to participate, but I would like to thank you very much for taking the time to speak with us about this opportunity.

[If YES:] Thank you for agreeing to participate in this important research study. I will need to speak to your parent/legal guardian to talk about the date/time of the interview. Could you please pass the phone back to your parent/legal guardian?

(IF CALLING PARENT/LEGAL GUARDIAN BACK, RECORD NAME, PHONE NUMBER, AND DATE/TIME.)

Parent/Legal Guardian Name: ______________________________________________________

Child Name: ____________________________________________________________________

Parent/Legal Guardian Phone: ______________________________________________________

Date: __________________________________________________________________________

Time: __________________________________________________________________________

If Parent/Legal Guardian Available and Currently Scheduling Interviews

Thank you again for allowing your child to participate in the NAEP research interviews. We will be holding our current interviews from <DATE> to <DATE>. What would be the best date/time to schedule the interview?

Date: ____________________

Time: ____________________

Thank you. We will follow up with a confirmation email/letter and a consent form. What is your email or postal address for us to send these to? ________________________________________________________________

If Parent/Legal Guardian Available and Not Currently Scheduling Interviews

Thank you again for allowing your child to participate in the NAEP research interviews. We are in between interview sessions and will need to call you back when we are actively scheduling the interviews to determine the date and time that will work best. What are your scheduling preferences? (Select all that apply.)

 Morning

 Afternoon

 After school

 Weekends

Thank you so much for talking with us today. We look forward to communicating with you soon.

Appendix Q: Consent Form for Parents/Legal Guardians of Student Participants (Under Age 18): Social Science Cognitive Interviews and Tryouts

PARENTAL CONSENT FORM FOR PARTICIPATION IN RESEARCH

EurekaFacts, LLC

Title of Research Study: <Cognitive Laboratory Interviews or Tryouts> for NAEP Social Sciences

Thank you for your interest in having your child participate in the research study described below. This consent form explains the research study. Please read it carefully, and feel free to ask questions about anything you do not understand. If you do not have questions now, you may ask them later as they occur to you.

What is this study about?

The National Center for Education Statistics (NCES), part of the U.S. Department of Education, is conducting research sessions with students currently in grade eight (ninth graders will also be considered) about new test questions for the National Assessment of Educational Progress (NAEP). The questions we are testing focus on the social sciences (civics, geography, and U.S. history) and will be part of a digitally based test. In these sessions, students will see early versions of digitally based questions. We may ask students whether they have any suggestions about how we might improve the questions.

NCES needs to know whether the questions on the test are clear. Students’ input will help NCES improve the test so that future students will understand it.

What will happen during the research <interviews or sessions>?

[For Tryouts: Each participant will take an early version of a new NAEP social science test on a tablet computer. Participants will be asked to answer the questions as if it were an actual test. We may ask participants for their reactions to and feedback about the test questions. EurekaFacts staff may record audio and take notes about your child’s reactions to the questions in order to revise and refine test content.]

Social science-specific knowledge is not required, and participants will not be graded. There will be more than one adult present.

[For Cognitive Interviews: EurekaFacts staff may record audio and screen activity for analysis and take notes about your child’s reactions to the questions in order to revise and refine test content.]

No names will be mentioned in the summary of findings and no personally identifying information will be retained once the study ends.

Where will the research session take place?

The sessions will take place at <NAME OF INTERVIEW/SESSION LOCATION>, located at <ADDRESS>, near <NEARBY LANDMARK or METRO STATION>.

[Insert if interview takes place at a school: For research activities conducted in schools, NAEP will not conduct pretesting interviews during school hours. If any pretesting interviews are conducted at schools, after regular school hours, we do not anticipate any disruption in school services to students.]

How long will the research <interview or session> last?

The <interview or session> will last no more than 90 minutes. Each session requires only one visit; however, your child may participate in multiple sessions, if interested.

Will information be kept private and confidential?

Your child will be assigned a unique student identifier (ID), and at no time will his or her name be linked to any of his or her answers. Your child’s responses may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151). [For Cognitive Interviews: Audio and screen activity will be recorded for analysis, but no personally identifying information will be retained.]

[For Tryouts: Audio may be recorded for analysis, but no personally identifying information will be retained.]

Will the results go to my child’s school?

Your child’s answers will not be disclosed to his or her school and will not be linked to any personally identifiable information such as the school’s name.

What are the possible risks of being in the study?

There are no known risks to participating in this study.

Can participation end early?

Participation in this study is completely voluntary. Your child has the right to refuse to answer questions. Your child may choose to withdraw from this study at any time and can end early if he or she wishes. Your child will still receive a gift card if he or she refuses to answer questions or chooses to end the <interview or session> early.

Whom can I contact with questions or for further information?

<CONTACT NAME> is coordinating this study at EurekaFacts on behalf of NCES. If you have any questions, please call <CONTACT PHONE> or email <CONTACT EMAIL>.

____ Yes, I agree to allow my child to participate in this study.

____ No, I do not want my child to participate in this study.

-------------------------------------------------------------------------------------------------------------------------------------------------------

By signing below, you agree that your child, _______________________________, may participate in this study. Your child will receive a $25 gift card from a major credit card company at the end of the interview.

In addition, if transportation to the interview site is involved, you will also receive a $25 gift card as appreciation for bringing your child to and from the site.

Your signature below will indicate that your questions have been answered satisfactorily, and that you have read and understood the information provided above.

Name (print): _______________________________________________________________________________

Signature: _____________________________________________________ Date: _______________________

Email address: ______________________________________________________________________________

School student attends: _______________________________________________________________________

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Title	NATIONAL CENTER FOR EDUCATION STATISTICS
Author	joc
File Modified	0000-00-00
File Created	2021-01-22

National Assessment of Educational Progress (NAEP) Grade 8 Social Sciences Interactive Item Components (IICs) Pretesting

NCES Cognitive, Pilot, and Field Test Studies System

Appendices NAEP Gr 8 Social Sciences IICs Pretesting