National Assessment of Educational Progress (NAEP) Survey Assessments Innovations Lab (SAIL) English Language Arts (ELA) Collaboration and Inquiry Study 2017

You previously indicated you wished to be contacted about additional study opportunities for your child(ren) at Educational Testing Service (ETS). Staff members at Educational Testing Service (ETS) and I are currently working on the National Assessment of Educational Progress (NAEP) Survey Assessment Innovations Lab (SAIL) Collaboration in Virtual Worlds Study, a federally funded project to study the effects of collaboration on interactive computer delivery and content of NAEP SAIL assessments. NAEP serves as the central means by which the United States measures student proficiency in core academic domains. By sharing their feedback with us, students can directly impact the future of this nationwide assessment by helping NCES evaluate the new types of tests that are being explored.

We are looking for students in 9^th or 10^th grade to participate in study sessions in which students will work with a partner to help us test new computer-delivered English language arts activities in a virtual environment. Students will be informed that their participation is completely voluntary, and that they will not be evaluated. They will also be told they have a key role in helping us improve our tests and they can make a difference in the experience of students who may use these tests in the future.

Project staff will audio and video record pairs of students as they interact with a programmed version of a virtual world which is being developed to engage students in collaborating to conduct inquiry and research in English Language Arts. Student pairs will first work through the assessment under the guidance of a researcher and will be asked to talk aloud as they engage in these activities. Students may also be asked questions about their experience as they work through the assessment. Afterwards, they will answer a questionnaire about their experience with the virtual world assessment and their perceptions of the collaboration, and will provide any feedback they have about the scenario or the tools in the assessment. In some sessions, students may be asked to briefly discuss their experience with the researcher. ETS staff will take notes of students’ reactions and responses.

Notes and recordings from this session will be used to help project staff get a feel for how students interact with the virtual world, and will be reviewed to inform revisions to the questions and virtual environment. You will be asked separately to consent to the possibility that your child’s recordings and data may be included in future research presentations. If you consent to this, the recording and data will be used, but names or other identifying information will not be disclosed. If you do not consent to this, these recordings will be seen only by project staff. We follow strict rules to make sure that others do not know names of students who have participated.

The sessions will be held at the ETS location on Rosedale Road in Princeton, NJ, and will be scheduled for up to 2.5 hours (150 minutes). Students will be given $40 gift cards for each session to thank them for their time and effort. In addition, if transportation to the site is involved, a parent or legal guardian of each student will receive a $25 gift card to thank them for bringing their child to and from the play testing site. Parents will not be in the room with their child, but should plan to return to pick up their child at the end of the session.

If your child is interested in being a part of the study, please reply to me at <EMAIL ADDRESS>. I will then send you a form to complete with information about your child and his/her potential availability for a study session.

I look forward to hearing from you!

Sincerely,

Educational Testing Service

660 Rosedale Rd

Princeton, NJ 08541

APPENDICES- TRYOUTS (A)

Appendix B: Site Recruitment E-mail/Letter (To School or Community Site Administrators/Staff Members) for Tryouts (A)

IF SENT AS E-MAIL - Subject: Research Study Opportunity

<DATE>

My name is <CONTACT NAME> and I am contacting you from <INSTITUTION>. We are conducting an important study for the National Center for Education Statistics (NCES), part of the U.S. Department of Education.

We are currently working on a new assessment being developed by the National Assessment of Educational Progress (NAEP) for the Survey Assessments Innovations Lab (SAIL) project, which may someday be delivered to students across the country. NAEP serves as the central means by which the United States measures student proficiency in core academic domains. It is vital for us to get student feedback while developing test questions.

We are looking for a school to work with us to try out this assessment with pairs of students in grades 9-10 to help us test new computer-delivered questions in English language arts using a virtual world that allows students to collaborate on research activities. These voluntary tryouts will last no more than 150 minutes. Students will be audio/video recorded while working with a partner to respond to several test questions, using a platform to enable remote collaboration. Screen recordings will also be collected. Project staff will take notes of how students collaborate to use the virtual environment, and use this information to revise and refine the questions and collaborative tools.

Students will be informed that their participation is completely voluntary, and that they will not be evaluated. They will also be told they have a key role in helping us improve our tests and they can make a difference in the experience of thousands of students who may someday take these tests.

The <school/community site(s)> we work with will need to be able to allow project team researchers approximately 150 minutes of after-school time with students working in pairs being located in separate rooms or on opposite sides of a large room (i.e., students will collaborate over a remote Internet connection, each using a separate computer). The study will be conducted in a single after-school session, allowing students to complete the inquiry task in one sitting. The site will be responsible for distributing and collecting consent forms and providing some basic demographic information about the students. As a thank you for participating, the <school/community site(s)> will receive a gift card equivalent to $10 per participating student with signed consent forms, up to a maximum of 40 students (if all testing occurs at a single school or site). Participating schools should expect to be able to provide a minimum of 20 student participants (i.e., 10 pairs of students). In addition, participating students will receive $40 gift cards for each session to thank them for their time and effort.

If your <school/organization> would be interested in participating in this important research project, or you have any questions or would like more information, please contact <CONTACT INFORMATION>.

Sincerely,

<NAME>

< INSTITUTION>

Appendix C: Site Recruitment Phone Script (When Calling School Administrators/Staff Members) for Tryouts (A)

Good morning/afternoon/evening. May I please speak to <NAME of contact at school/organization>?

My name is <CONTACT NAME> and I am contacting you from <INSTITUTION>. We are administering an important study for the National Center for Education Statistics (NCES), part of the U.S. Department of Education.

We are looking for a school to work with us to try out this assessment with students in grades 9-10 to help us test new computer-delivered questions and simulations in English language arts. These voluntary tryouts will last no more than 150 minutes. Students will be audio/video recorded while working with a partner to respond to several test questions, using a platform to enable remote collaboration. Screen recordings will also be collected. Project staff will take notes of how students collaborate to use the virtual environment, and use this information to revise and refine the questions and collaborative tools.

The <school or schools/community site or sites> we work with will need to be able to allow project team researchers approximately 150 minutes of after-school time with students working in pairs being located in separate rooms or on opposite sides of a large room (i.e., students will collaborate over a remote Internet connection, each using a separate computer). The study will be conducted in a single after-school session, allowing students to complete the inquiry task in one sitting. The school/site will be responsible for distributing and collecting consent forms and providing some basic demographic information about the students. As a thank you for participating, the school(s)/site(s) we work with will receive a gift card equivalent to $10 per participating student with signed consent forms, up to a maximum of 40 students (if all testing occurs at a single school or site). Participating school(s)/site(s) should expect to be able to provide a minimum of 20 student participants (i.e., 10 pairs of students). In addition, participating students will receive $40 gift cards for each session to thank them for their time and effort.

Would your school/organization be interested in participating in this research study?

If NO: Thank you for your time. Have a wonderful day/evening.

If YES: Great, thank you very much. To ensure that we interview a broad mix of students and determine the best interview location, I have a few brief screening questions to ask you. It should only take a few minutes of your time.

Screener

[INTERVIEWER: Determine recruitment needs in advance. Look in public documents and school/organization website to find what grades a school/organization includes, the size of each grade, the SES, and racial/ethnic makeup of the student body. Record all responses to screener questions. End screener at whatever point the staff member gives a response that confirms that their group has already been sampled adequately. End screener script: Based on the requirements of this study, we are not able to include you in the interviews at this time.]

Do you have at least twenty 9^th or 10^th graders who might be available to participate in the study during after-school hours?
[Note: preference will be given to schools with existing after school programs for students]
Would you be able to give project staff room(s) in which to conduct the study at your <ORGANIZATION>?

Would you be able to send flyers and consent forms to parents of potential student participants?

If NO: Thank you for your time. Have a wonderful day/evening.

If YES: Thank you for taking the time to answer the screening questions. Your school has students who are eligible to participate in the tryouts. Again, each session will last up to 150 minutes, during after-school hours.

[Schedule test date(s) and make arrangements to send consent forms to the school.]

We may need to reach you by telephone. Is the number I called the best number to use?

Do you have any questions at this time?

[INTERVIEWER: Answer any questions.]

If you do have any questions, please call <PHONE NUMBER>.

Thank you again for agreeing to be part of this very important research project.

Appendix D: Consent Form for Parents/Legal Guardians of Student Participants (In-School or Community Site Administration) for Tryouts (A)

PARTICIPATION CONSENT DOCUMENT

FOR PARTICIPATION IN RESEARCH

Dear Parent/Guardian:

Your child’s <school/organization> is participating in a pilot study for the National Assessment of Educational Progress (NAEP), for a federally funded project that is studying new types of technology delivered assessments. As part of this study, your child will spend 150 minutes during after-school hours taking a test in which they work with a partner to conduct research activities designed to assess English language arts proficiency within a virtual environment. After they finish the activity, students will complete a brief questionnaire about their experience. Because the sessions will take place after school, students will receive a $40 gift card to thank them for their participation.

The project team will audio/video and screen record students as they work in pairs to take part in problem-solving activities related to English language arts. Students will complete some questions about themselves, preferences for working collaboratively, and prior knowledge. Then they will be asked to interact with a virtual environment while working together to answer questions about a research task. Students will be asked to talk aloud (via audio/video chat) with their partner as they engage in these activities. Afterwards, they will answer a questionnaire about their experience with the activity.

Your child’s performance on the assessment will not be used as an indication of his or her abilities; it will inform us how well the assessment tasks work and help us to develop and refine our ideas for future tasks. Unless you give specific permission, all information obtained from your child may be used only for research purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law [ESRA, 20 U.S.C §9573]. Participation is voluntary and a student may withdraw from the study at any time.

We hope that you will give your consent for your child to participate in this important activity by signing the accompanying consent form. Students should return the attached permission form to the classroom teacher by <DATE>.

If you have any questions, please call <CONTACT> of <INSTITUTION> at <PHONE NUMBER> or email <EMAIL>.

Should you have any questions, please do not hesitate to contact me.

Sincerely,

<NAME>

Parent Consent Form

Student’s name: ____________________________________________________

Please print student’s name

I am the parent/guardian of the child named above, and I hereby give my permission for my child to take part in the National Assessment of Educational Progress (NAEP) for the Survey Assessments Innovations Lab (SAIL) research study, including having the session video recorded for use by the project team. I understand that my child will not be identified by name, and, unless I give a separate consent for the information to be shared in other ways, that my child’s information may be used only for research purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law [ESRA, 20 U.S.C §9573].

Having read the accompanying letter, I voluntarily consent to my child’s participation in this study. I understand that my child’s participation is voluntary, my child may withdraw from the study at any time without prejudice, and that my child has the right to refuse to answer any questions <he/she> doesn’t feel comfortable responding to.

I understand that if I have any questions about any part of my child’s participation, I may call the Project Director, <CONTACT INFORMATION>.

Yes, I consent to my child participating in this research project, including the video capture of my child completing the assigned task and answering interview questions.

Shape2 No, I do not consent to my child participating in this research project.

_____________________________________________ __________________________

Parent/Guardian Signature Date

In the future, we may have additional opportunities for students to participate in studies. If you are interested in being contacted about your child participating in future studies, please check the box below.

Shape3 I consent to being contacted about opportunities for my child to participate in future NAEP studies.

The data collected in this project will be valuable information for the larger cognitive learning field, and as such, your child’s responses or clips from these videos may be selected for use in research reports, presentations, and/or conferences. Your child’s name will not be identified, and you do not need to consent to your child’s recording being included in external research publications and/or research conferences in order for your child to participate in this study.

Yes, I consent to my child’s data (video and responses) being included in external research publications and/or research conferences.

No, I do not consent to my child’s data (video and responses) being included in external research publications and/or research conferences.

Students should return this permission form to their <teacher/organization leader> by <XXXX Date>.

Appendix E: Thank You E-mail/Letter to Schools/Organizations of Student Participants for Tryouts (A)

IF SENT AS E-MAIL - Subject: Thank you for your Participation in a Research Study

<DATE>

Dear <NAME>,

On behalf of the National Center for Education Statistics (NCES), part of the U.S. Department of Education, we would like to thank you for your <school’s/organization’s/association’s> participation in the NAEP SAIL research study. Thanks to the students at your <school/organization/association> we are better able to refine, revise, and develop the virtual environment, collaborative tools, and questions that are part of this assessment.

Your <school’s/organization’s/association’s> assistance helped us better understand how students approach the <SUBJECT> test questions. The students’ help was very much appreciated.

Again, thank you for your assistance.

Sincerely,

APPENDICES- TRYOUTS (B/C)

Appendix F: Recruitment E-mail/Letter (If Additional Out-Of-School Recruitment is Required - To be Sent to Teachers/Staff Members) for Tryouts (B/C)

IF SENT AS E-MAIL - Subject: Research Study Opportunity

<DATE>

Dear <Teacher or Staff Member at After School Program>,

You have been very helpful in the past in working with <Educational Testing Service (ETS)/EurekaFacts> to recruit students to help us test our tests. The students you helped to recruit provided rich information, and we are requesting your help again in recruiting more students to help us evaluate and improve test questions.

We are looking for students in grades 9 and 10 to help us test new computer-delivered questions for the NAEP (National Assessment of Educational Progress) SAIL (Survey Assessments Innovations Lab) research study. NAEP serves as the central means by which the United States measures student proficiency in core academic domains. It is vital for us to get student feedback while developing test questions.

In these sessions, students will complete a pre-task questionnaire about their relevant prior experiences, perspectives, and knowledge and then work in pairs while interacting with a virtual environment and answering questions that have been developed for assessing collaborative research skills in English language arts. After completing the tasks, they will respond to a post-task survey about their experiences with the virtual environment and the collaborative activities. Students will be informed that their participation is completely voluntary, and that they will not be evaluated. They will also be told they have a key role in helping us improve our tests and they can make a difference in the experience of students who may someday take these or similar tests.

Project staff will collect audio/video and screen recordings of students as they observe the way in which student pairs interact with the virtual environment and related questions. Student responses may be used only for research purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law [ESRA 2002, 20 U.S.C §9573].

The sessions will be scheduled for 150 minutes, and students will receive $40 gift cards for each session to thank them for their time and effort. In addition, if transportation to the site is involved, a parent or legal guardian of each student will receive a $25 gift card to thank them for the time in bringing the child to and from the study site.

We would like your help in distributing flyers. We will hold sessions at <ETS offices located in Princeton, New Jersey/EurekaFacts offices located in Rockville, Maryland>. If you are interested and willing to help us recruit students for this effort, or would like additional information about the effort, please contact <staff member>. We are attaching a flyer that can be shared with your group, and will be happy to provide you with printed copies if you prefer.

Thank you for your time and interest. If you have any questions, please contact <CONTACT INFORMATION>.

Sincerely,

Attachment: Flyer (Appendix I)

Appendix G: Recruitment Email/Letter for Parents/Guardians of Student Participants (Under Age 18) for Tryouts (B/C)

IF SENT AS EMAIL - Subject: Research Study Opportunity for Students in Grades 9 and 10

<DATE>

Dear <NAME>,

My name is <CONTACT NAME> and I am contacting you from <ETS/EurekaFacts>, a local research company. We are carrying out an important research study on behalf of the NAEP (National Assessment of Educational Progress) SAIL (Survey Assessments Innovations Lab), a federally funded project which is studying the effects of collaboration on interactive computer delivery and content of NAEP SAIL assessments. NAEP serves as the central means by which the United States measures student proficiency in core academic domains. By sharing their feedback with us, students can directly impact the future of this nationwide assessment by helping NCES evaluate the new types of tests that are being explored.

We are recruiting students currently in 9^th and 10^th grade to work in pairs to take part in problem-solving activities related to English language arts. Your child’s input will help NCES improve the test to help ensure that other students will find the test questions to be clear and easily understandable. If selected to participate, your child will receive a $40 gift card [from a major credit card company] to thank them for their time and effort. You will also receive a $25 gift card to thank you for your time in bringing your child to and from the session site.

The tryout session will take place at <LOCATION> and will last approximately 150 minutes. The sessions can be scheduled at your convenience, including after school and on weekends. Your child will be asked to work with a partner to use a computer-based virtual world and platform to answer questions about conducting research in the English language arts, and then answer some questions about their experience with the collaborative tasks.

NCES simply wants to know how students would answer the questions if it were an actual test. These sessions will be audio and/or video recorded. All of your child’s responses will be used only for research purposes and will not be disclosed or used, in identifiable form, for any other purpose except as required by law [Education Sciences Reform Act of 2002 (ESRA 2002) 20 U.S.C., § 9573].

These question tryouts are an important stage of the NAEP test development process. The students can directly impact the future of this national assessment by helping NCES evaluate the test questions and instructions to create the best test possible.

If your child is interested in participating in this research study, please call <CONTACT PHONE> or email <CONTACT EMAIL>. We look forward to speaking with you soon!

Thank you,

Appendix H: Frequently Asked Questions (FAQs) (for Option B or C)

Frequently Asked Questions

NAEP SAIL Research Study

What is this study about?

The National Center for Education Statistics (NCES), part of the U.S. Department of Education, is conducting a study with students currently in grade 9 and 10 to try out a new English Language Arts task for the National Assessment of Educational Progress (NAEP)’s Survey Assessments Innovations Lab (SAIL). NCES needs to know how students perform on the task while working collaboratively. Students’ input will help NCES better understand how to assess students as they work together to complete activities in English Language arts.

What will happen during the tryout?

Each participant will work in pairs with a partner to take an early version of a new NAEP English Language arts test on a tablet computer. Participants will work in pairs, being located in separate rooms or on opposite sides of a large room (i.e., students will collaborate over a remote Internet connection), each using a separate computer to use a computer-based virtual world to complete a collaborative activity. Students will be asked to answer the questions as if it were an actual test. Participants will not be graded. In addition, students will answer a pre-task about their relevant prior experiences, perspectives, and knowledge, as well as a post-task questionnaire about their experience with virtual world and the collaborative tasks.

Where will the tryouts take place?

The tryouts will take place at <NAME OF TRYOUT LOCATION>, located at <LOCATION ADDRESS>, near <NEARBY LANDMARK OR METRO STATION>.

4. How long will the tryout last and when will it take place?

The tryout session will last no more than 150 minutes and can be scheduled at your convenience. Each tryout requires only one visit to our office.

5. Whose consent do you require for participation?

For students under the age of 18, we require written informed consent from a parent/legal guardian in order for the eligible student to participate. We also ask the student for his/her verbal consent to participate during the prescreening process. Participation in the study is completely voluntary. The parent/legal guardian or student may withdraw consent for the student’s participation in the study at any time. The student also has the right to refuse to answer particular questions.

6. Will you keep information confidential and private?

All of the participants’ responses will be used only for research purposes and will not be disclosed or used, in identifiable form, for any other purpose except as required by law [Education Sciences Reform Act of 2002 (ESRA 2002) 20 U.S.C., § 9573]. Each participant will be assigned a unique student identifier (ID), and at no time will his/her name be linked to any of his/her answers. The tryouts will be audio and video recorded, as long as you agree, and are also used only for research purposes.

7. Will the results go to the participant’s school?

Participant’s answers will not be disclosed to his/her school and will not be linked to any personally identifiable information such as the school’s name.

8. Are there any rewards or incentives for participating?

Yes, the student will receive a $40 gift card (from a major credit card company) from us at the end of his or her tryout to thank them for their time and effort. The student’s parent or guardian will also receive a $25 gift card to thank him or her for bringing the child to and from the tryout site.

9. What are the possible risks of being in the study?

There are no known risks to participating in this study.

Can participation end early?

Students’ participation in this study is completely voluntary. Students have the right to refuse to answer particular questions. Students may elect to withdraw from this study at any time.

Whom can I contact with questions or for further information?

<CONTACT NAME> is coordinating this study at <ETS/EurekaFacts> on behalf of NCES. If you have any questions, please call him/her at <CONTACT PHONE> or by email <CONTACT EMAIL>.

APPENDICES- MATERIALS COMMON TO PLAY TESTING AND TRYOUTS

Appendix I: Informational Flyer/Recruitment Ad Text for Play Testing and Tryouts (B/C)

Can You Help Us Test Our Test?

<Educational Testing Service (ETS) or EurekaFacts> is looking for students in grades 9 and 10 to help us test new computer-delivered assessments in virtual environments for the National Center for Education Statistics (NCES), part of the U.S. Department of Education. These questions are part of NAEP (National Assessment of Educational Progress) SAIL (Survey Assessments Innovations Lab) research projects exploring new approaches to collaborative assessment. [PLAY TESTING: You will be shown early versions of computer-based assessment tasks and we will be recording as you work with a partner to respond to them and give your opinion about them. We call this Play Testing. TRYOUTS: You will work with a partner as you interact with a virtual environment and answer test questions as part of a tryout of materials that have been developed.]

We will collect audio/video and screen recordings of the tryout sessions. The information we collect from you as you explore and react to our test will only be used to help improve the test. The information will not be shared with or used by any school or district.

These sessions will be held at <locatioN> on <note days of the week when sessions are scheduled and the window of time when these sessions will be held>. These sessions will last no more than 150 minutes.

You will get a $40 gift card for your time and effort in helping us to improve our tests. In addition, if transportation to the site is involved, your parent or legal guardian will receive a $25 gift card to thank him or her for the time in bringing you to and from the [play testing/tryouts] site.

If you are interested, please ask your parent or guardian to fill out an intake form at [WEBSURVEYOR LINK], or contact [CONTACT PERSON] at [PHONE] or [E-MAIL] for more information or to give the necessary information over the phone.

Appendix J: Websurveyor Intake Form for Play Testing and Tryouts (B/C)

NAEP SAIL Collaboration in Virtual Worlds Study

Staff members at <Educational Testing Service (ETS)/EurekaFacts> and I are currently working on the NAEP (National Assessment of Educational Progress) SAIL (Survey Assessments Innovations Lab) research study, a federally funded project which is studying the effects of collaboration on interactive computer delivery and content of NAEP SAIL assessments. We are inviting 9^th and 10^th graders to participate in our study. This will inform us how the tasks work and help us to develop and refine our ideas for future tasks.

The study will require that students be audio/video and screen recorded as they spend approximately 150 minutes completing the following tasks:

Answer a pre-task questionnaire about their relevant prior experiences, perspectives, and knowledge.
Work with a partner to use a virtual environment to answer English language arts questions
Talk aloud [TRYOUTS ONLY: (via audio/video chat)] with a partner as they participate in the activity together
Answer some written [PLAY TESTING ONLY: and oral] questions about their experience with the collaborative tasks

Students will receive a $40 gift card to thank them for their time and effort after participating in the study. In addition, if transportation to the site is involved, the parent or legal guardian will receive a $25 gift card to thank him or her for the time in bringing participant(s) to and from the study site. Unless you give specific permission, your child’s responses to questions may be used only for research purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law [ESRA, 20 U.S.C §9573]. Participation is voluntary and a student may withdraw from the study at any time.

We would like to have a balanced group of students take part in the study. To that end, we’d like you to give us some information about your student by filling out the form on the following pages. We will let you know within <TIME> whether your student has been selected to take part in this study. If your child is not selected for this study, I hope you will consider letting him or her be a part of other studies in the future.

If you have any questions, please contact <CONTACT INFORMATION>.

Thanks for your interest in this study!

<NAME>

Student Name (Last, First): __________________

Parent Name (Last, First): __________________

Parent Phone Number: __________________

Parent E-mail Address: __________________

What grade is your child in? ____9th ___10th

Student’s school: __________________

Student’s most recent grade in English Language Arts: __________________

Students’ most recent grade in U.S. History: __________________

*Student’s Age:

*Student’s Gender:

*Student’s Race/Ethnicity:

Ethnicity:  Hispanic or Latino  Not Hispanic or Latino

Race (one or more of the categories below):

American Indian or Alaska Native
Asian
Black or African American
Native Hawaiian or other Pacific Islander
White or Caucasian

* What is your total annual household income range based on the following ranges? (Choose one)

Less than $20,000
$20,000 to $49,999
$50,000 to $99,999
$100,000 or more

* Does your child participate in the National School Lunch program (receive Free/Reduced Price Lunch)?

Yes  No

* Would you consider the area you live in to be …

Urban
Suburban
Rural

* What is the zip code of the area that you live in? _________________

(* Indicates that the question is voluntary.)

Possible times for 150-minute session: Please mark all the sessions that would work for your child(ren).

(It is possible that not all these times will be used.)

(DATE)	(TIME)	(TIME)	(TIME)
(DATE)	(TIME)	(TIME)	(TIME)
(DATE)	(TIME)	(TIME)	(TIME)
(DATE)	(TIME)	(TIME)	(TIME)
(DATE)	(TIME)	(TIME)	(TIME)
(DATE)	(TIME)	(TIME)	(TIME)
(DATE)	(TIME)	(TIME)	(TIME)
(DATE)	(TIME)	(TIME)	(TIME)

If some other time during these weeks would be better, what 150-minute time frame would that be?

To complete submission of this form, please click the “Submit” button. You will be contacted within <TIME> to let you know whether or not your child has been selected to participate in this study.

(SUBMIT BUTTON)

Appendix K: Recruitment Phone Script for Parents/Legal Guardians of Student Participants for Play Testing and Tryouts (B/C)

Good morning/afternoon/evening. Is this the <NAME FROM CALL LIST> household? My name is <NAME>, and I am calling from <Educational Testing Service (ETS)/EurekaFacts>. We are administering a study on behalf of the U.S. Department of Education’s National Center for Education Statistics, or NCES. We are recruiting students currently in grades 9 and 10 to participate in a research study supporting the development of new English language arts assessments and interactive virtual environments for a project being conducted by the National Assessment of Educational Progress or NAEP. [If parent asks what this is: This national assessment is administered to students in elementary, middle, and high schools throughout the United States.]

Are there any 9^th or 10^th grade students in your household?

Yes (Great! Let me give you some additional information about the study.)

No (Thank you, have a good day/evening.)

We are conducting <play testing/tryouts> to explore how students work through sample test questions and interact with a virtual environment to ensure that they are clear and understandable to a broad range of students. If selected to participate, your child will receive a $40 gift card to thank them for their time and effort. In addition, if transportation to the site is involved, you will receive a $25 gift card to thank you for the time in bringing your child to and from the interview site.

The study sessions will take place at <COMPANY LOCATION>. It can be scheduled at your convenience and will last about 150 minutes (or 2 and a half hours). During the interview, your child will be asked to work with a partner to conduct information gathering and evaluation activities while exploring a virtual world on a computer, and talking aloud [TRYOUTS ONLY: (via audio/video chat)] about the process they use for answering those questions. NCES simply wants to know how students and their partners would use the virtual environment and answer the questions if they were included on an actual NAEP assessment.

These sessions are an important stage of the NAEP test development process. By sharing their feedback with us, students can directly impact the future of this nationwide assessment by helping NCES evaluate the new types of tests that are being explored.

Do you believe your child would be interested in participating?

Yes (Proceed to protocol script below)

No (Thank you, have a good day/evening.)

Great! To ensure that we interview a broad mix of students, I have a few brief questions to ask about you and your child. It should only take a few minutes of your time. First, I want to collect some additional information about your child to identify if <HE/SHE> meets the participation requirements. The information I collect from you may be used only for research purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law [ESRA, 20 U.S.C §9573].
(Skip to screener.)

Script when calling a parent who responded to e-mail or letter

Good morning/afternoon/evening. My name is <NAME>. I’m calling from <ORGANIZATION: ETS or EurekaFacts> on behalf of the National Center for Education Statistics (NCES), part of the U.S. Department of Education. I’d like to speak to <NAME>. Thank you for contacting us about the possibility of your child participating in a research interview to help develop innovative English language arts assessments for the National Assessment of Educational Progress, or NAEP. To ensure that we interview a broad mix of students, I have a few brief screening questions to ask about you and your child. It should only take a few minutes of your time. First, I want to collect some additional information about your child to identify if <HE/SHE> meets the participation requirements. The information I collect from you may be used only for research purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law [ESRA, 20 U.S.C §9573].

Screener

(INTERVIEWER: Determine recruitment needs in advance. Record all responses. End screener at whatever point the parent or legal guardian gives a response that confirms that their group has already been sampled adequately. End screener script: Thank you for answering my questions. Based on the requirements of this study, we are not able to include your child in the interviews at this time.)

What is your child’s name?

What grade is your child currently in? (INTERVIEWER: DO NOT READ LIST)

 9th

 10th

 Other (Unfortunately, we are only interviewing students currently in grade 9 or grade 10 for this study. Thank you for your time. Have a good day/evening.)

What school does your child attend? (INTERVIEWER: Note if homeschooled)

____________________________________________________________

What is your child’s most recent grade in English Language Arts?

What is your child’s most recent grade in U.S. History?
How old is your child?

What is your child’s gender?

What is your child’s race or ethnicity? (INTERVIEWER: Only read list if parent doesn’t immediately reply. Are you…?)

Ethnicity:  Hispanic or Latino  Not Hispanic or Latino

Race (one or more of the categories below):

American Indian or Alaska Native
Asian
Black or African American
Native Hawaiian or other Pacific Islander
White or Caucasian

[If the respondent indicates they prefer not to disclose this information, the interviewer should say “That’s fine” and move on to the next question.]

Just to be sure we're interviewing a broad range of students for this study…

What is your total annual household income range based on the following ranges? [INTERVIEWER: If the respondent indicates they prefer not to disclose this information, say: “That’s fine,” and move on to the next question.]

Less than $20,000
$20,000 to $49,999
$50,000 to $99,999
$100,000 or more

Does your child participate in the National School Lunch program? [INTERVIEWER: If the respondent indicates they prefer not to disclose this information, say “That’s fine,” and move on to the next question. If the respondent is not sure what this refers to, ask “Does your child receive free or reduced price lunch?”]

Would you consider the area you live in to be…

Urban
Suburban
Rural

Ok, thank you. And what is the zip code of the area that you live in?

_________________

Thank you. We would like to speak with your child to confirm if (he/she) is interested in participating in the tryout. It will only take a couple of minutes and then I will ask to speak with you again for another moment. Is your child available to speak with us now or should we call (him/her) back at another time?

 Yes, the child is available for screening questions  (SKIP TO SCRIPT FOR STUDENTS)

 No, please call back.

Is the phone number you gave me earlier the best number to call?

Other phone numbers: ____________________________

Is there a good time for me to call? When would be best for you?

INTERVIEWER: IF CHILD NEEDS TO RECEIVE A CALL BACK, RECORD NAME, PHONE NUMBER, DATE/TIME

Parent Name ____________________

Child Name ____________________

Child Phone ____________________

Date ____________________

Time ____________________

SCRIPT FOR 9th/10th GRADE STUDENTS

Good morning/afternoon/evening. My name is <NAME> and I’m calling from <ORGANIZATION: Educational Testing Service (ETS)/EurekaFacts>. We are doing an important study for the National Center for Education Statistics (NCES), part of the U.S. Department of Education.

We have spoken to your parent/guardian and we are pleased that you may be interested in helping us try out innovative English language arts test questions for a new test that is being developed by the National Assessment of Educational Progress, or NAEP. Before I tell you more about what you would be asked to do, I’d like to ask you one question first:

What grade are you currently in? (INTERVIEWER: DO NOT READ LIST)

 9th

 10th

 Other (Unfortunately, we are only interviewing students currently in grade 9 or grade 10 for this study. Thank you for your time. Have a good day/evening.)

Thank you. For the study, you will be asked to visit <COMPANY LOCATION> and work with a partner to respond to a sample of English language arts test questions. Your responses will not be graded. The National Center for Education Statistics simply wants to know how you and a partner would interact with a virtual model environment and answer the questions if they were to be included on an actual test. The questions focus on English language arts and are really interesting and fun to answer. You will be video recorded as you answer the questions on a computer but the data will only be used for research purposes. Your input will help evaluate and improve the test questions and simulation. To thank you for participating, you will receive a $40 gift card. In addition, if transportation to the site is involved, your parent or legal guardian will receive a $25 gift card to thank him or her for the time in bringing you to and from the study site. The study will last about 150 minutes and will be recorded. Your interaction with the virtual environment will be video recorded as well. Your participation is totally voluntary. Your interview responses and all other interview materials may be used only for research purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law [ESRA, 20 U.S.C §9573].

Would you like to participate?

 Yes

 No  (THANK AND TERMINATE)

If NO - Thank you for your time. I will need to speak to your parent/guardian to thank (him/her) for letting you take the time to speak with us.

TO PARENT: Your child has decided not to participate, but I would like to thank you very much for taking the time to speak with us today/this evening.

If YES - Thank you for agreeing to participate in this important research study. I will need to speak to your parent/guardian again. Is he/she available?

 Yes

 No

TO PARENT: Your child has indicated that he/she would be interested in participating in this study. We will be in touch with you within <TIME> to let you know whether or not your child has been selected to participate in this study, and to discuss scheduling and send you a consent form to fill out if he/she will be participating.

INTERVIEWER: IF CALLING PARENT BACK, RECORD NAME, PHONE NUMBER, DATE/TIME

Parent Name ____________________

Child Name ____________________

Parent Phone ____________________

Date ____________________

Time ____________________

Appendix L: Consent Form for Parent/Legal Guardian of Student Participants for Play Testing and Tryouts (B/C)

PARTICIPATION CONSENT DOCUMENT

FOR PARTICIPATION IN RESEARCH

Dear Parent/Guardian:

Staff members at <Educational Testing Service (ETS)/EurekaFacts> and I are currently working on the NAEP (National Assessment of Educational Progress) SAIL (Survey Assessments Innovations Lab) Study, a federally funded project to study new types of technology delivered assessments. We are inviting 9^th – 10^th graders to participate in our study. For this project, a staff member from <ETS/EurekaFacts> will be meeting with students in pairs for a study that lasts around 150 minutes at the <ETS location on Rosedale Road in Princeton, NJ/EurekaFacts location in Rockville, Maryland>. Parents will not be in the room with their child, and should plan to return to pick up their child at the end of the session. Students will receive a $40 gift card to thank them for participation. In addition, if transportation to the site is involved, the parent or legal guardian will receive a $25 gift card to thank him or her for the time in bringing the student to and from the <play testing/tryout> site.

At the beginning of the <play testing/tryout> session, students will individually complete a pre-task questionnaire about their relevant prior experiences, perspectives, and knowledge. Next, the project team will audio/video and screen record students as they work in pairs to take part in problem-solving activities related to English language arts. They will be asked to interact with a virtual environment while working together to answer questions about a research task. Students will be asked to talk aloud <TRYOUTS ONLY: (via audio/video chat)] with their partner as they engage in these activities. Afterwards, they will answer a questionnaire about their experience with the activity>. <PLAY TESTING ONLY: They may also answer additional oral questions posed by the researcher.>

Your child’s performance is not an indication of his or her abilities; it will inform us how the tasks work and help us develop and refine our ideas for future tasks. Unless you give specific permission, your child’s responses to questions may be used only for research purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law [ESRA, 20 U.S.C §9573]. Participation is voluntary and a student may withdraw from the study at any time.

We hope that you will give your consent for your child to participate in this important activity by signing the accompanying consent form. Students should bring the signed permission form with them when they come to <ETS/EurekaFacts>.

Please Note: Parents/Guardians are responsible for providing transportation for their children to and from <ETS’ Princeton campus/EurekaFacts’ Rockville, MD headquarters>, where the sessions will be held.

To check or confirm your appointment, or if you have further questions, please call (CONTACT) of (ORGANIZATON) at <PHONE NUMBER> or email <EMAIL>.

We thank you for considering supporting our research. Should you have any questions, please do not hesitate to contact me.

Sincerely,

<NAME>

Parent Consent Form

Student’s name: ____________________________________________________

Please print student’s name

I understand that if I have any questions about any part of my child’s participation, I may call the Project Director, <CONTACT INFORMATION>.

Yes, I consent to my child participating in this research project, including the video capture of my child completing the assigned task and answering interview questions.

Shape7 No, I do not consent to my child participating in this research project.

_____________________________________________ __________________________

Parent/Guardian Signature Date

Shape8 I consent to being contacted about opportunities for my child to participate in future NAEP studies.

Yes, I consent to my child’s data (video and responses) being included in external research publications and/or research conferences.

No, I do not consent to my child’s data (video and responses) being included in external research publications and/or research conferences.

Please bring this completed form with you at the time of the child’s scheduled appointment.

Appendix M: E-mail/Letter Interview Confirmation for Parents/Legal Guardians of Student Participants for Play Testing and Tryouts (B/C)

IF SENT AS E-MAIL - Subject: Confirmation of <Play testing/Tryout> Session

<DATE>

Dear <NAME>,

Thank you for agreeing to allow your child to participate in a study to give us feedback on the questions and simulation being developed for the National Assessment of Educational Progress (NAEP) SAIL (Survey Assessment Innovations Lab) research project. During the session, your child will be asked to respond to sample test activities and interact with a virtual environment that is being developed to assess knowledge of research skills in English language arts. In addition, your child will be asked to answer a pre-task questionnaire about their relevant prior experiences, perspectives, and knowledge, and a post-task questionnaires about their experience with the virtual environment and the collaborative tasks. The National Center for Education Statistics (NCES), part of the U.S. Department of Education, simply wants to know what your child thinks about the tests and how your child might work with a partner to answer questions on that test. Your child will work with another student who will also be reviewing and reacting to the potential test questions. Your child’s feedback/responses will be noted, audio/video recorded, and used to help us revise and improve our tests.

Unless you give specific permission, your child’s responses to questions may be used only for research purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law [ESRA, 20 U.S.C §9573]. Participation is voluntary and a student may withdraw from the study at any time.

Your child has been selected to attend a session at <START TO END TIME> on <DATE> located at the <ETS campus on Rosedale Road in Princeton, NJ/EurekaFacts offices in Rockville, MD>. To thank your child for <his/her> time and effort, <he/she> will receive a $40 gift card as soon as <he/she> has finished the session. In addition, if transportation to the site is involved, you will receive a $25 gift card to thank you for the time in bringing your child to and from the [play testing / tryout] site.

Please confirm that this time slot still works for your child by <DATE>. We are attaching a consent form that must be signed and returned for your child to participate. Please fill it out and send it back to us, or have your child bring it with them to the [play testing / tryout] site.

We look forward to meeting with your child and getting your child’s feedback. If you have any questions or if your child is unable to attend this session, please contact me using the information below.

Sincerely,

Appendix N: Acknowledgement E-Mail to Parents/Legal Guardians of Students Not Selected to Participate in Play Testing and Tryouts (B/C)

IF SENT AS E-MAIL - Subject: NAEP SAIL Study

<DATE>

Dear <NAME>,

Thank you for your interest in having your child be a part of the NAEP (National Assessment of Educational Progress) SAIL (Survey Assessments Innovations Lab) <SUBJECT> Study. Unfortunately, due to the number of students who were interested in participating, we are not able to ask your child to be a part of the study at this time. However, we anticipate that we may be conducting additional studies related to this project in the future. If you are interested in possibly being contacted about having your child participate at a later time if another opportunity becomes available, please let us know (see contact information below).

Thank you again for taking the time to respond to our participation request.

Sincerely,

Appendix O: Thank You E-mail/Letter to Parent/Legal Guardian of Student Participants for Play Testing and Tryouts (A/B/C)

IF SENT AS E-MAIL - Subject: Thank you for your Child’s Participation in a Research Study

<DATE>

Dear <NAME>,

On behalf of the National Center for Education Statistics (NCES), part of the U.S. Department of Education, Educational Testing Service (ETS) would like to thank you for allowing your child to participate in our National Assessment of Educational Progress (NAEP) SAIL (Survey Assessment Innovations Lab) research study. Thanks to your child’s participation, we are better able to develop and refine the virtual environment, collaborative tools, and questions that are part of this assessment.

Your child’s assistance helped us better understand how students approach these test questions and use the virtual environment. <His/Her> <PLAY TESTING: open, candid, and insightful comments were very helpful. TRYOUTS: help was very much appreciated>.

Again, thank you for your child’s assistance.

Sincerely,

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Title	NATIONAL CENTER FOR EDUCATION STATISTICS
Author	joc
File Modified	0000-00-00
File Created	2021-01-23