Communication Materials

You have been very helpful in the past in working with Educational Testing Service (ETS) to recruit students to help us test our tests. The students you helped to recruit provided rich information, and we are requesting your help again in recruiting more students to help us evaluate and improve test questions.

We are looking for students in grades 8 and 9 to help us test new computer-delivered questions, 3D models, and a virtual science lab for the NAEP (National Assessment of Educational Progress) SAIL (Survey Assessments Innovations Lab) Study. NAEP serves as the central means by which the United States measures student proficiency in core academic domains. It is vital for us to get student feedback while developing test questions.

These reviews are sessions where we show students versions of computer-delivered questions, 3D models, and a virtual science lab. Students will be informed that their participation is completely voluntary, and that they will not be evaluated. They will also be told they have a key role in helping us improve our tests and they can make a difference in the experience of students who may use these tests in the future.

ETS staff will take notes of students’ reactions to these questions and interactions with the virtual models and virtual lab and use students’ feedback to revise and refine the questions, virtual models, and virtual lab. We will record students as we observe and ask questions about the students’ level of interest and ability to understand the questions and use the simulations. Students will also be asked if they have any suggestions on how to improve them.

The sessions will be scheduled for up to 90 minutes, and students will be given $25 Visa gift cards for each session as a thank you for their time and as an incentive to help us. In addition, if transportation to the site is involved, a parent or legal guardian of each student will receive a $25 Visa gift card as a thank you for the time in bringing the child to and from the <playtesting/cognitive interview> site.

We would like your help in distributing flyers. We will hold sessions at ETS offices located in Princeton, New Jersey. If you are interested and willing to help us recruit students for this effort, or would like additional information about the effort, please contact <staff member at ETS organizing the sessions>. We are attaching a flyer that can be shared with your group, and will be happy to provide you with printed copies if you prefer.

Thank you for your time and interest. If you have any questions, please contact <CONTACT INFORMATION>.

Sincerely,

Educational Testing Service

660 Rosedale Rd

Princeton, NJ 08541

Appendix B: Informational Flyer

Can You Help Us Test Our Test?

Educational Testing Service (ETS) is looking for students in grades 8 and 9 to help us test new computer-delivered assessments in virtual environments for the National Center for Education Statistics (NCES), part of the U.S. Department of Education. These questions are part of NAEP SAIL (Survey Assessments Innovations Lab) research projects exploring new test approaches. You will be given early drafts of test questions and we will be recording as you respond to them and give your opinion about them. We call this <Playtesting/Cognitive Interviews>.

The information we collect from you as you explore and react to our test will only be used to help improve the test. The information will not be shared with or used by any school or district.

<Playtesting/cognitive interview> sessions will be held at ETS at 660 Rosedale Road in Princeton, NJ on <note days of the week when sessions are scheduled and the window of time when these sessions will be held>. These sessions will last no more than 90 minutes.

You will get a $25 gift card for your time in helping us to improve our tests. In addition, if transportation to the site is involved, your parent or legal guardian will receive a $25 Visa gift card to thank him or her for the time in bringing you to and from the <playtesting/cognitive interview> site.

If you are interested, please ask your parent or guardian to fill out an intake form at [WEBSURVEYOR LINK], or contact [CONTACT PERSON] at [PHONE] or [EMAIL] for more information or to give the necessary information over the phone.

Appendix C: Websurveyor Intake Form

NAEP SAIL Study

The study will require that students be audio and video recorded as they spend approximately 90 minutes completing the following tasks either individually or in small groups:

Use virtual models or virtual science lab to solve a science problem
Think aloud as they participate in the activity
Answer some questions about their experience with the tasks

Students will receive a $25 Visa gift card for participation in the study. In addition, if transportation to the site is involved, the parent or legal guardian will receive a $25 Visa gift card to thank him or her for the time in bringing participant(s) to and from the <playtesting/cognitive interview> site. The responses to questions and taking part in this study may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law [ESRA, 20 U.S.C §9573]. Participation is voluntary and the student may withdraw from the study at any time.

We would like to have a balanced group of students take part in the study. To that end, we’d like you to give us some information about your student by filling out the form on the following pages. We will let you know within <TIME> whether your student has been selected to take part in this <playtesting/cognitive interview> study. If your child is not selected for this study, we hope you will consider letting him or her be a part of other studies in the future.

If you have any questions, please contact (CONTACT INFORMATION).

Thanks for your interest in this study!

(NAME)

Student Name (Last, First):

Parent Name (Last, First):

Parent Phone Number:

Parent Email Address:

What grade is your child in? ____8th ___9th

Student’s school:

*Student’s Age:

*Student’s Gender:

*Student’s Race/Ethnicity:

Ethnicity:  Hispanic or Latino  Not Hispanic or Latino

Race (one or more of the categories below):

American Indian or Alaska Native
Asian
Black or African American
Native Hawaiian or other Pacific Islander
White or Caucasian

(* Indicates that the question is voluntary.)

Possible time for 90 minute session: Please mark all the sessions that would work for your child(ren)

(Not all these times will be used. We are trying to schedule students in a few convenient sessions)

(DATE)	(TIME)	(TIME)	(TIME)
(DATE)	(TIME)	(TIME)	(TIME)
(DATE)	(TIME)	(TIME)	(TIME)
(DATE)	(TIME)	(TIME)	(TIME)
(DATE)	(TIME)	(TIME)	(TIME)
(DATE)	(TIME)	(TIME)	(TIME)
(DATE)	(TIME)	(TIME)	(TIME)
(DATE)	(TIME)	(TIME)	(TIME)

If some other time during these weeks would be better, what 90 minute time frame would that be?

To complete submission of this form, please click the “Submit” button. You will be contacted within <TIME> to let you know whether or not your child has been selected to participate in this study.

(SUBMIT BUTTON)

Appendix D: Recruitment Phone Script for Parents or Legal Guardians of Student Participants

(To be used if direct outreach to parents is necessary)

Script when calling parents from previously compiled list:

Good morning/afternoon/evening. Is this the <NAME FROM CALL LIST> household? My name is <NAME>, and I am calling from Educational Testing Service (ETS). We are conducting a study on behalf of the U.S. Department of Education’s National Center for Education Statistics, or NCES. We are recruiting students currently in grades 8 and 9 to sign up to participate in a research study supporting the development of new science assessments and <interactive virtual models / lab> for a project being sponsored by the National Assessment of Educational Progress or NAEP. [If parent asks what this is: This national assessment is administered to students in elementary, middle, and high schools throughout the United States.]

Are there any 8th or 9th grade students in your household?

Yes (Great! Let me give you some additional information about the study.)

No (Thank you, have a good day/evening.)

We are conducting research interviews to explore how students work through sample science test questions and interact with virtual models / a virtual science lab to ensure that they are clear and understandable to a broad range of students. If selected to participate, your child will receive a $25 Visa gift card. In addition, if transportation to the site is involved, you will receive a $25 Visa gift card to thank you for the time in bringing your child to and from the interview site.

The research interview will take place at Educational Testing Service in Princeton, NJ. It can be scheduled at your convenience and will last about 90 minutes. During the interview, your child will be asked to respond to sample science test questions on a computer, and think aloud about the process they use for answering those questions. NCES simply wants to know how students would use the virtual models / virtual science lab and answer the questions if they were included on an actual NAEP assessment.

These research interviews are an important stage of the NAEP test development process. By sharing their feedback with us, students can directly impact the future of this nationwide assessment by helping NCES evaluate the new types of tests which are being explored.

Do you believe your child would be interested in participating?

Yes (Proceed to protocol script below)

No (Thank you, have a good day/evening.)

Great! To ensure that we interview a broad mix of students, I have a few brief questions to ask about you and your child. It should only take a few minutes of your time. First, I want to collect some additional information about your child to identify if <HE/SHE> meets the participation requirements. The information I collect from you may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law [ESRA, 20 U.S.C §9573].

(Skip to screener.)

Script when calling a parent who responded to email or letter

Good morning/afternoon/evening. My name is <NAME>. I’m calling from Educational Testing Service (ETS) on behalf of the National Center for Education Statistics (NCES), part of the U.S. Department of Education. I’d like to speak to <NAME>. Thank you for contacting us about the possibility of your child participating in a research interview to help develop innovative science test questions for the National Assessment of Educational Progress, or NAEP. To ensure that we interview a broad mix of students, I have a few brief screening questions to ask about you and your child. It should only take a few minutes of your time. First, I want to collect some additional information about your child to identify if <HE/SHE> meets the participation requirements. The information I collect from you may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law [ESRA, 20 U.S.C §9573].

Screener

(INTERVIEWER: Determine recruitment needs in advance. Record all responses. End screener at whatever point the parent or legal guardian gives a response that confirms that their group has already been sampled adequately. End screener script: Thank you for answering my questions. Based on the requirements of this study, we are not able to include your child in the interviews at this time.)

What is your child’s name?

What grade is your child currently in? (INTERVIEWER: DO NOT READ LIST)

 8th

 9th

 Other (Unfortunately, we are only interviewing students currently in grade 8 or grade 9 for this study. Thank you for your time. Have a good day/evening.)

What school does your child attend? (INTERVIEWER: Note if homeschooled)

____________________________________________________________

How old is your child?

What is your child’s gender?

What is your child’s race or ethnicity? (INTERVIEWER: Is she/he …..?)

Student’s Race/Ethnicity:

Ethnicity:  Hispanic or Latino  Not Hispanic or Latino

Race (one or more of the categories below):

American Indian or Alaska Native
Asian
Black or African American
Native Hawaiian or other Pacific Islander
White or Caucasian

[If the respondent indicates they prefer not to disclose this information, the interviewer should say “That’s fine” and move on to the next question.]

Thank you. We would like to speak with your child to confirm if (he/she) is interested in participating in the research interview. It will only take a couple of minutes and then I will ask to speak with you again for another moment. Is your child available to speak with us now or should we call (him/her) back at another time?

 Yes, the child is available for screening questions  (SKIP TO SCRIPT FOR STUDENTS)

 No, please call back.

Is the phone number you gave me earlier the best number to call?

Other phone numbers: ____________________________

Is there a good time for me to call? When would be best for you?

INTERVIEWER: IF CHILD NEEDS TO RECEIVE A CALL BACK, RECORD NAME, PHONE NUMBER, DATE/TIME

Parent Name ____________________

Child Name ____________________

Child Phone ____________________

Date ____________________

Time ____________________

SCRIPT FOR STUDENTS

Good morning/afternoon/evening. My name is <NAME> and I’m calling from Educational Testing Service (ETS). We are doing an important study for the National Center for Education Statistics (NCES), part of the U.S. Department of Education.

We have spoken to your parent/guardian and we are pleased that you may be interested in helping us develop innovative science test questions for a new test that is being developed by the National Assessment of Educational Progress, or NAEP. Before I tell you more about what you would be asked to do, I’d like to ask you one question first:

What grade are you currently in? (INTERVIEWER: DO NOT READ LIST)

 8th

 9th

 Other (Unfortunately, we are only interviewing students currently in grade 8 or grade 9 for this study. Thank you for your time. Have a good day/evening.)

Thank you. For the research interview, you will be asked to visit Educational Testing Service in Princeton and respond to a sample of science test questions. Your responses will not be graded. The National Center for Education Statistics simply wants to know how you would interact with a virtual model / virtual science lab and answer the questions if they were to be included on an actual test. The questions focus on science and are really interesting and fun to answer. You will be asked to answer the questions on a computer. Your input will help evaluate and improve the test questions and virtual environment. To thank you for participating, you will receive a $25 Visa gift card. The interview will last about 90 minutes and will be recorded. Your interactions with the virtual model / virtual science lab will be videotaped as well but the data will only be used for research purposes. Your participation is totally voluntary. If you participate, your interview responses and all other interview materials may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law [ESRA, 20 U.S.C §9573].

Would you like to participate?

 Yes

 No  (THANK AND TERMINATE)

If NO - Thank you for your time. I will need to speak to your parent/guardian to thank (him/her) for letting you take the time to speak with us.

TO PARENT: Your child has decided not to participate, but I would like to thank you very much for taking the time to speak with us today/this evening.

If YES - Thank you for agreeing to participate in this important research study. I will need to speak to your parent again to make arrangements for your participation. Is he/she available?

 Yes

 No

TO PARENT: Your child has indicated that he/she would be interested in participating in this study. We will be in touch with you within <TIME> to let you know whether or not your child has been selected to participate in this study, and to discuss scheduling and send you a consent form to fill out if he/she will be participating.

INTERVIEWER: IF CALLING PARENT BACK, RECORD NAME, PHONE NUMBER, DATE/TIME

Parent Name ____________________

Child Name ____________________

Parent Phone ____________________

Date ____________________

Time ____________________

Appendix E: Consent Form for Parents/Legal Guardians of Student Participants

PARTICIPATION CONSENT DOCUMENT

FOR PARTICIPATION IN RESEARCH

Dear Parent/Guardian:

Staff members at Educational Testing Service (ETS) and I are currently working on the NAEP (National Assessment of Educational Progress) SAIL (Survey Assessments Innovations Lab) Study, a federally funded project which is studying new types of computer delivered assessments. We are inviting 8th and 9th graders to participate in our study. For this project, a staff member from ETS will be meeting with students individually or in small groups for a study that lasts around 90 minutes at the ETS location on Rosedale Road in Princeton, NJ. Parents will not be in the room with their child, and should plan to return to pick up their child at the end of the session. Students will receive a $25 Visa gift card for their participation. In addition, if transportation to the site is involved, the parent or legal guardian will receive a $25 Visa gift card to thank him or her for the time in bringing the child to and from the <playtesting/cognitive interview> site.

The project team will video record students as they take part in problem-solving activities related to science. They will be asked to manipulate virtual objects on a screen and/or actual physical objects / interact with a virtual science lab while answering questions about science. Students will be asked to think aloud as they engage in these activities. Afterwards, they will answer a questionnaire about their experience with the activity.

Your child’s performance is not an indication of his or her abilities; it will inform us how the tasks work and help us to develop and refine our ideas for future tasks. All information obtained from your child may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law [ESRA, 20 U.S.C §9573].

We hope that you will give your consent for your child to participate in this important activity by signing the accompanying permission form. Students should bring the signed permission form with them when they come to ETS.

Please Note: Parents are responsible for providing transportation for their children

to and from ETS’s Princeton campus, where the sessions will be held.

To check or confirm your appointment, or if you have further questions, please call (CONTACT) of ETS at (PHONE NUMBER) or email (EMAIL).

Sincerely,

Parent Consent Form

Student’s name: ____________________________________________________

Please print student’s name

I am the parent/guardian of the child named above, and I hereby give my permission for my child to take part in the NAEP SAIL Study, including having the session video recorded for use by the project team. I understand that my child’s information may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law [ESRA, 20 U.S.C §9573].

Having read the accompanying letter, I voluntarily consent to my child’s participation in this study. I understand that my child’s participation is voluntary, my child may withdraw from the study at any time without prejudice, and that my child has the right to refuse to answer any questions they don’t feel comfortable responding to.

I understand that if I have any questions about any part of my child’s participation, I may call the Project Director, <SCIENTIST CONTACT INFORMATION>.

_____________________________________________ __________________________

Parent/Guardian Signature Date

Shape2 Yes, I consent to my child participating in this research project, including the video capture of my child completing the assigned task and answering interview questions.

Shape3 No, I do not consent to my child participating in this research project.

________________________________________________________________________________________

In the future, we may have additional opportunities for students to participate in studies. If you are interested in being contacted about your child participating in future studies, please check the box below.

Shape4 I consent to being contacted about opportunities for my child to participate in future NAEP SAIL studies.

________________________________________________________________________________________

The data collected in this project will be valuable information for the larger cognitive learning field, and as such, your child’s responses or clips from these videos may be selected for use in research reports, presentations and/or conferences. Your child’s name will not be identified.

Shape5 Yes, I consent to my child’s data (video and responses) being included in external publications and/or research conferences.

Shape6 No, I do not consent to my child’s data (video and responses) being included in external publications and/or research conferences.

Please bring this completed form with you at the time of the

child’s scheduled appointment.

Appendix F: Email/Letter Interview Confirmation for Parents/Legal Guardians of Student Participants

IF SENT AS EMAIL - Subject: Confirmation of <Playtesting/Cognitive Interview> Session

<DATE>

Dear <NAME>,

Thank you for agreeing to allow your child to participate in a study to give us feedback on the questions and simulation being developed for the NAEP (National Assessment of Educational Progress) SAIL (Survey Assessments Innovations Lab) project. During the session, your child will be asked to respond to sample test activities and interact with a virtual model / virtual lab which is being developed to assess knowledge of science. The National Center for Education Statistics (NCES), part of the U.S. Department of Education, simply wants to know what students such as your child think about the tests and how they might answer questions on that test. Your child will work individually or in a small group of several students who will also be reviewing and reacting to the potential test questions. Your child’s feedback and responses will be noted and audio and video recorded to help us revise and improve our tests.

All of your child’s responses may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law [ESRA, 20 U.S.C §9573].

Your child has been selected to attend a session at <START TO END TIME> on <DATE> located at the Princeton campus of ETS. To thank your child for <his/her> time, <he/she> will receive a $25 Visa gift card as soon as he/she has finished the session. In addition, if transportation to the site is involved, you will receive a $25 Visa gift card to thank you for the time in bringing your child to and from the <playtesting/cognitive interview> site.

Please confirm that this time slot still works for your child by <DATE>. We are attaching a consent form that must be signed and returned for your child to participate. Please fill it out and send it back to us, or have your child bring it with them to the <playtesting/cognitive interview> site.

We look forward to meeting with your child and getting your child’s feedback. If you have any questions or if your child is unable to attend this session, please contact me using the information below.

Sincerely,

Educational Testing Service

Appendix G: Acknowledgement Email to Parents/Legal Guardians of Students Not Selected to Participate

IF SENT AS EMAIL - Subject: NAEP SAIL Study

<DATE>

Dear <NAME>,

Thank you for your interest in having your son or daughter be a part of the NAEP (National Assessment of Educational Progress) SAIL (Survey Assessments Innovations Lab) <SUBJECT> Study. Unfortunately, due to the number of students who were interested in participating, we are not able to ask your child to be a part of the study at this time. However, we anticipate that we will be conducting additional studies related to this project in the future. If you are interested in possibly being contacted about having your son or daughter participate at a later time if another opportunity becomes available, please let us know (see contact information below).

Thank you again for taking the time to respond to our participation request.

Sincerely,

Educational Testing Service

Appendix H: Thank You Email/Letter to Parent or Legal Guardian of Student Participants

IF SENT AS EMAIL - Subject: Thank you for your Child’s Participation in a Research Study

<DATE>

Dear <NAME>,

On behalf of the National Center for Education Statistics (NCES), part of the U.S. Department of Education, Educational Testing Service (ETS) would like to thank you for allowing your child to participate in the NAEP (National Assessment of Educational Progress) SAIL (Survey Assessments Innovations Lab) research study. Thanks to your child’s participation, we are better able to refine, revise, and develop the simulation and questions that are part of this assessment.

Your child’s assistance helped us better understand how students approach these test questions and use the virtual environment. <HIS/HER> open, candid, and insightful comments were very helpful.

Again, thank you for your child’s assistance.

Sincerely,

Educational Testing Service

Appendix I: Thank You Email/Letter to School/ Organization/ Association of Student Participants

IF SENT AS EMAIL - Subject: Thank you for your Participation in a Research Study

<DATE>

Dear <NAME>,

On behalf of the National Center for Education Statistics (NCES), part of the U.S. Department of Education, Educational Testing Service (ETS) would like to thank you for your <school’s/ organization’s/ association’s> participation in the NAEP (National Assessment of Educational Progress) SAIL (Survey Assessments Innovations Lab) research study. Thanks to the students at your <school/ organization/ association> we are better able to refine, revise, and develop the virtual environment and questions which are part of this assessment.

Your <school’s/ organization’s/ association’s> assistance helped us better understand how students approach the <SUBJECT> test questions. Students’ open, candid, and insightful comments were very helpful.

Again, thank you for your assistance.

Sincerely,

< COMPANY REPRESENTATIVE >

Educational Testing Service

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Title	NATIONAL CENTER FOR EDUCATION STATISTICS
Author	joc
File Modified	0000-00-00
File Created	2021-01-27