Appendices - Recruitment and Communication Materials

Fulcrum, a NAEP contractor, will be conducting ongoing user testing for the National Center for Education Statistics (NCES). The focus of user testing will be on new tools and item types being considered/developed for NAEP assessments.

School Participation

The study will take place during two consecutive school days, scheduled on dates convenient for the school
Eighteen students (nine per day) are expected to participate
Participating students will be in grades 4, 8, and 12
The study will be conducted at a quiet area to work, such as a library or empty classroom, with easy access to electrical outlets
A $50 gift card will be provided to the school as a “thank you” for participating in the study

Student Participation

Each student will work one-on-one with a study facilitator for 1 hour and 15 minutes
Students will be seen in groups of 3 or 4
Each student will get to keep the earbuds used during the user testing

Equipment

Fulcrum staff will provide all equipment used in the study

Schedule

Scheduling the students over the two-day testing period would be up to the school, and should take into consideration student lunch periods and academic schedule. A typical schedule might be as follows:

Sample Schedule

	Day 1	Day 2
8:30-9:45	Students 1, 2, 3	Students 10, 11, 12
10:00-11:15	Students 4, 5, 6	Students 13, 14, 15
11:15-12:30	Lunch	Lunch
12:30-1:45	Students 7, 8, 9	Students 16, 17, 18

For more information on how to participate, please contact the usability team at Fulcrum.

Appendix B: Usability Study Sample Email Contact with Participating Schools

Email Subject Line: Arrangements for NAEP Usability Study Date: <DATE>

Dear <Name of School Coordinator>,

My name is <Researcher’s Name>, and I represent Fulcrum. <Name of NAEP State Coordinator> forwarded your name to me to work out arrangements for my colleagues to visit <Name of School> in the near future. We are requesting to work with <number of students> students as part of a usability study to test tools and item types being considered for NAEP assessments. This study will be carried out by Fulcrum, on behalf of the National Center for Education Statistics for the National Assessment of Educational Progress, or NAEP.

In the study, students will be asked to complete tasks on a tablet and answer questions about how easy or hard it is to accomplish computer functions, answer questions in different formats, and understand what the system expects them to do next. Before working on the tablets, students will be asked about their familiarity and frequency regarding usage of laptops, smart phones, and tablets. Student information may only be used for research purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (Education Sciences Reform Act of 2002, 20 U.S.C §9573). Students may decline participation for any reason.

There are a few logistical matters to address before we conduct the study. I will summarize them in this email, and then follow up with a phone call to finalize these arrangements, unless you prefer to finalize them via email.

Study Date(s)

We expect each user testing session to take approximately 75 minutes to complete. We could come to your school the week of <Date>, or we can arrange for different dates with you.

Student Selection for Study

Please identify up to <number of students> <appropriate grade> students—approximately half boys and half girls—to participate in the study. Students should be generally representative of your school makeup (e.g., race, SES, familiarity with computers), as the purpose of this study is to examine the usability of assessment tools and item types for a wide range of students.

Space Requirements

When we visit to conduct the study, we will need to use a conference room, classroom, or space in your media center or library (if private) with room for up to three students with tablets and four research associates with laptops. We will bring the tablets and related equipment, and set up everything that we need. We would like to meet with up to three students at a time for up to 75 minutes each, but we are flexible depending on your class schedules and space considerations.

Parent/Guardian Notification

Parents/guardians of the students participating in the study should be notified by letter. A sample letter is attached to this email; please provide this letter to the parents/guardians.

Other Logistics

It would be helpful to know your school day schedule so that we can plan to arrive 45 minutes before the first testing session. In addition, please let me know where we should park, and where we should check in with school staff upon arrival.

A representative will call you within the next few days to determine which dates might work for you and to answer any questions you might have. Feel free also to contact me by email or phone, as indicated below. We greatly appreciate your school’s cooperation, and will offer a $50 gift card to a school supply store as a token of our appreciation. Thank you for your help.

Sincerely,

<Researcher’s Name>

ATTACHMENT: Sample Parental Notification Letter <Appendix D>

Appendix C: Usability Study Sample School Contact Script

NAEP 2015-2016 Usability Study

School Contact Script

Note: Information in grey boxes is not to be read; rather, these are instructions to the Fulcrum staff.

Hello, _________________________ <NAME OF SCHOOL COORDINATOR>. My name is __________________ <NAME OF INTERVIEWER> and I am calling from Fulcrum, which has been contracted to carry out a study on behalf of the National Center for Education Statistics about the National Assessment of Educational Progress, or NAEP. I am following up on an email I sent to you a few days ago describing the study. This call should only take a few minutes. Would now be a good time to finalize arrangements for the study?

	Yes	 Go to question 1.
	No	 Continue

If not, when would be a good time? ___________________________

Thank you for your time. I look forward to speaking to you then. [end call]

1. In my email, I proposed the week of <DATE> for us to conduct the study. Is there a day during that week that works for you?

	Yes	 Date 1: ________________
	No	 Arrange new date: _____________________________________

2. Now let’s discuss the study logistics. We’ll need a quiet space, with desks or tables, in which to set up tablets and laptop computers for the students and research associates. We will be working with the students in groups of three, and we need enough room for four tables on which to set up our equipment and work with the students. Three of the tables will be for student-research associate pairs and their equipment, while the fourth will be for the study supervisor and server equipment. In the past, we have worked in the library or an empty classroom. We will also need access to electrical outlets for the laptops and tablets. We have our own extension cords, so we would just need a place to plug them in. If you know where you’d like us to work, I can write that down now, or you can just show us when we arrive.

 Testing Location: ___________________________ (if known)

3. We usually schedule three groups of three students, for a total of nine students, per day. Even though the students will be working for no more than 75 minutes, sessions will need to be spaced at least 90 minutes apart to allow for transition time between groups of students. Sessions should be scheduled taking into account lunch times and other daily routines that a student might need to participate in. I’d like to write down the start times for the three sessions now, and then you can give us a list of which students are in which sessions when we arrive.

	Date:_____________________________
Students Group #1	Time: ___________	Location: ______________
Students Group #2	Time: ___________	Location: ______________
Students Group #3	Time: ___________	Location: ______________

My team and I will need about 45 minutes before the first group is scheduled to set up and prepare. What time should we arrive? ______ <TIME>, is that OK? May we have access to __________________ <TESTING LOCATION> at that time?

4. Now, let’s review the status of parent/guardian notification. Have the parents and guardians been notified?

	Yes	 Continue
	No	 Skip to question 5.

Please keep a copy of the letter for me, which I’ll collect from you on the morning of the usability study.

Were notifications sent to only the parents and guardians of the selected students, or to the parents and guardians of all the students in the [specify grade] grade?

	Parents/guardians of selected students only	 Skip to question 6.
	Parents/guardians of all students in the [specify grade] grade.	On the morning of the usability study, if any of the selected students are absent, we’d like to substitute another [specify grade] grade student in their place so that we can test as many students as possible.
		 Skip to question 6.

5. As a reminder, the parents and guardians of the selected students must be notified before the usability study can take place. I’d like to call you back to confirm the letter has been sent. When is a good time for me to make that call?

 Callback date: _______________________ Time: ________________

Also, I’ll need a copy of the letter for my records, which I’ll collect from you on the morning of the usability study.

Do you plan to notify the parents/guardians of only the selected students or the parents and guardians of all the students in the [specify grade] grade?

	Parents/guardians of selected students only	 Skip to question 6.
	Parents/guardians of all students in the [specify grade] grade.	On the morning of the usability study, if any of the selected students are absent, we’d like to substitute another [specify grade] grade student in their place so that we can test as many students as possible.

6. I have just a few other logistical questions to ask you, so that the study runs as smoothly as possible.

When my team and I arrive at your school on the morning of the study, where should we park?

_______________________________________________________________________________________

We will check in at the main office. Are there other procedures or protocols specific to your school that we need to follow?

_______________________________________________________________________________________

Is there anything scheduled that might interrupt the usability session, such as a fire drill or assembly?

_______________________________________________________________________________________

How should we contact the office from the study location if we need assistance should there be an emergency?

_______________________________________________________________________________________

How will we get the students for each session? Should we notify the office when we are ready for the next group of students?

_______________________________________________________________________________________

How do you announce school closings or delays, such as a snow day or weather delay? By what time are these decisions made?

_______________________________________________________________________________________

Our team lead will want to let you know when we are leaving, and will have a gift card for the school as a thank you for participating in our study. Do you know which store you’d like the gift card for?

_______________________________________________________________________________________

7. Those are all of the questions I have for you at this time. Do you have any questions for me?

If you have any questions before the usability study date, I can be reached at ________________________ <TELEPHONE NUMBER> or this email address _________________________ <EMAIL ADDRESS>.

Thank you for your time.

NOTES:

Appendix D: Usability Study Parent/Guardian Notification Letter

NAEP PARENT/GUARDIAN NOTIFICATION LETTER

Usability Evaluation of NAEP Assessment Tools & Item Types

[School Letterhead] <DATE>

Dear Parent or Guardian:

We are pleased to inform you that <School Name> has been selected to participate in the usability study for tools and item types for the National Assessment of Educational Progress (NAEP). This study will be used to guide development of future NAEP assessments.

NAEP is the largest continuing and nationally representative assessment of what our nation’s students know and can do. The results of NAEP are released as The Nation’s Report Card, which provides information about student achievement to educators, parents, policymakers, and the public.

In our school, the usability study will be conducted on <DATE>. Your child may be selected to participate. The study involves observing individual students working for 75 minutes on touch screen tablets, and asking them questions about how easy or hard it is to perform tasks, answer questions in different formats, and understand what the system expects them to do next. Their comments and actions will be recorded for later analysis. Before working on the tablets, students will be asked about their familiarity and frequency regarding usage of laptops, smart phones, and tablets.

It will take about 75 minutes for most students to complete the study. The screens of the tablets will be recorded, along with the audio from the session. No cameras will be used, and no images or videos of your child or their surroundings will be captured. The researchers will collect no identifying information on the students. Each student will be identified only by a participant number in the study. All the information your child provides may only be used for research purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law [Education Sciences Reform Act of 2002, 20 U.S.C §9573].

Your child will represent many other students, so participation is very important. However, your child’s participation in the study is voluntary. In addition, your child may decide not to participate or to end his or her participation before the study is complete. If you do not want your child to participate, please notify me by <DATE>.

There is no need to prepare for this study. As a token of appreciation for participating in the study, your child will get to keep a set of earbuds given to them during the study.

Students are given an opportunity to ask questions before, during, and after the study. If you have any questions before or after the study, contact the lead field researcher, <CONTACT NAME>, Fulcrum Services, LLC, <CONTACT PHONE NUMBER AND EMAIL>. If you would like additional information about the NAEP assessments, please visit http://nces.ed.gov/nationsreportcard/. If you have any other questions, please contact me at <telephone number> or via email at <EMAIL ADDRESS>.

Sincerely,

<Principal’s Name>

School Principal

Appendix E: Thank You Email to Participating School

Usability Evaluation of NAEP Assessment Tools and Item Types

<DATE>

Dear <Head of School>:

I wanted to write you personally to thank you for participating in our NAEP usability study. The results of this study will be used to improve the delivery of NAEP assessments, and make participation in these assessments a positive experience for everyone involved. If you have any feedback for us about your experience participating in this usability study, don’t hesitate to contact me or the project manager at Fulcrum.

Sincerely,

<Researcher’s Name>

Appendix F: EurekaFacts Recruitment Phone Script for Schools of Potential Participants

Hello. May I please speak to <NAME of contact at school>?

Good morning/afternoon/evening. My name is <CONTACT NAME> and I am contacting you from EurekaFacts, a local research company. We are carrying out an important study for the National Center for Education Statistics (NCES), part of the U.S. Department of Education. We are recruiting students who are currently in grades 4, 8, and 12 to participate in research sessions supporting the development of new tools and item types for the National Assessment of Educational Progress (NAEP). The research sessions will explore how students work through the tasks on a tablet computer to ensure that the tasks are clear and understandable to a broad range of students.

The voluntary research sessions will last no more than 75 minutes. Students will be asked to complete tasks on a tablet and answer questions about how easy or hard it is to accomplish computer functions, answer questions in different formats, and understand what the system expects them to do next. Before working on the tablets, students will be asked about their familiarity and frequency regarding usage of laptops, smart phones, and tablets.

Student responses will be used only for research purposes and will not be disclosed or used, in identifiable form, for any other purpose except as required by law.

[INTERVIEWER note: If asked what law is being referenced, please provide its name: Education Sciences Reform Act of 2002 (ESRA 2002) 20 U.S.C., § 9573.]

We would like your help in distributing flyers to students, and we will follow up directly with any who contact us. For each participating student, a $25 gift card (from a major credit card company) will be issued in appreciation for their participation. In addition, if a parent or legal guardian brings the student to and from the site, he or she will also receive a $25 thank you gift card.

These sessions are an important stage of the NAEP test development process. The comments of students attending your school will help NCES improve the NAEP test.

Would your school be interested in participating in or helping us to recruit students for this research study?

If NO: Thank you for your time. Have a good day/evening. [Interviewer: End call]

If YES: Great, thank you very much. Would you be able to send flyers to parents of potential student participants?

If NO: Thank you for your time. Have a good day/evening. [Interviewer: End call]

If YES: Would you be able to make a room at your school available where EurekaFacts could conduct some 75-minute research sessions?

If YES: That’s great, we will contact you to schedule some sessions at your school after we have some students signed up. [Interviewer: go to Closing]

If NO: Thank you for your time. Have a good day/evening. [Interviewer: End call]

(Closing)

Thank you for agreeing to disseminate information about this important research study. We will send you email with informational flyers.

Is <EMAIL ADDRESS> the best email to reach you? OR What is the best email address to reach you?

We may need to reach you by telephone. Is the number I called the best number to use?

Do you have any questions at this time?

[INTERVIEWER: Answer any questions.]

If you do have any questions, please call <PHONE NUMBER>.

Thank you again for agreeing to be part of this important research project. Have a wonderful day/evening.

Appendix G: EurekaFacts Recruitment Phone Script for Parents/Guardians of Participants (Below Age 18)

Script when calling parents from previously compiled list:

Good <MORNING/AFTERNOON/EVENING>. May I please speak to <NAME> about <STUDENT’S NAME> session for the National Assessment of Educational Progress (NAEP) usability study for tools and item types study?

My name is <CALLER NAME>, and I am calling from EurekaFacts, a local research company. We are carrying out a study on behalf of the U.S. Department of Education’s National Center for Education Statistics.

We are recruiting students currently in grades 4, 8, or 12 to participate in a research study supporting the development of new tools and item types for the National Assessment of Educational Progress or NAEP. [If parent asks what this is: This national assessment is administered to students in elementary, middle, and high schools throughout the United States.]

Are there any students at grades 4, 8, or 12 in your household?

No: (Thank you, have a good day/evening.)

Yes: (Great! Let me give you some additional information about the study.)

The research session will take place at <NAME OF SESSION SITE AND LOCATION>. It can be scheduled at your convenience and will last about 75 minutes. During the session, your child will be asked to complete several tasks on a tablet computer using a variety of tools and interactions. NCES simply wants to know how students would do the tasks if they were on an actual test, how intuitive and easy to understand the tasks are, and to hear students’ thoughts on how to improve the tasks and make them easier to understand.

If selected to participate, your child will receive a $25 gift card (from a major credit card company) as thanks for his or her time. In addition, if you bring your child to and from the research site, you will also receive a $25 gift card.

These research sessions are an important stage of the NAEP development process. By sharing their feedback with us, students can directly guide the future of this nationwide assessment by helping NCES evaluate the test and instructions to create the best test possible.

Do you believe that your child would be interested in participating?

No: (Thank you anyway, have a good day/evening.)

Yes: (Great! Thank you so much. I have a few brief questions to ask about you and your child. It should take only a few minutes.

[Skip to Screener.]

Script when calling a parent/guardian who responded to email or letter

Good morning/afternoon/evening. My name is <NAME>. I’m calling from EurekaFacts on behalf of the National Center for Education Statistics, part of the U.S. Department of Education.

Thank you for contacting us about the possibility of your child participating in a research session to help assess testing tools for the National Assessment of Educational Progress, or NAEP. First, I have a few brief screening questions to ask about you and your child. It should only take a few minutes.

Screener

[INTERVIEWER: Record all responses.]

What is your child’s name?

What grade is your child currently in (OR, IF INTERVIEW OCCURS IN THE SUMMER, What grade will your child be in this fall)? [INTERVIEWER: Do not read list]

 Grade 4, 8, or 12 – Good.

 Other  (Unfortunately, we need only students currently in grade 4, grade 8, or grade 12 for this study. Thank you for your time. Have a good day/evening.)

What school does your child attend? [INTERVIEWER: Note if homeschooled]

____________________________________________________________

What school district does he or she attend? [INTERVIEWER: Note if parochial, charter, homeschooled] __________________________________________________

What is your child’s race or ethnicity? [INTERVIEWER: Read list only if parent doesn’t immediately reply. “Is he/she…?”]

Ethnicity:  Hispanic or Latino  Not Hispanic or Latino

Race (one or more of the categories below):

 American Indian or Alaska Native

 Asian

 Black or African American

 Hispanic or Latino

 Native Hawaiian or other Pacific Islander

 White or Caucasian

[If the respondent indicates they prefer not to disclose this information, the interviewer should say “That’s fine” and move on to the next question.]

Just to be sure we're working with a broad range of students for this study, does your child participate in the National School Lunch program?

 Yes

 No

 Refused to disclose

[If the respondent indicates they prefer not to disclose this information, the interviewer should say “That’s fine” and move on to the next question.]

Would you consider the area you live in to be …

 Urban

 Suburban

 Rural

Ok. Thank you and what is the zip code of the area that you live in?

______________

Thank you. We would like to speak with your child to confirm if (he/she) is interested in participating in the research session. It will take only a couple of minutes and then I will ask to speak with you again for another moment.

Is your child available to speak with us now or should we call (him/her) back at another time?

 Yes, the child is available for screening questions  [Skip to script for students]

 No, please call back.

Is the phone number you gave me earlier the best number to call?

Other phone numbers: ____________________________

Is there a good time for me to call? When would be best for you?

[INTERVIEWER: If child needs to receive a call back, record name, phone number, date/time]

Parent/Guardian Name ___________________________________________________________

Child Name ____________________________________________________________________

Child Phone ____________________________________________________________________

Date ____________________

Time ____________________

SCRIPT FOR STUDENTS

Good morning/afternoon/evening. My name is <NAME> and I’m calling from EurekaFacts, a local research company. We are doing an important study for the National Center for Education Statistics, part of the U.S. Department of Education.

We have spoken to your parent and we are pleased that you may be interested in helping us develop testing tools for a nationwide student assessment, the National Assessment of Educational Progress, or NAEP.

Before I tell you more about what you would be asked to do, I need to ask you one question:

What grade are you currently in (OR, IF INTERVIEW OCCURS IN THE SUMMER, What grade will you be in this fall)? [INTERVIEWER: Do not read list]

 Grade 4, 8, or 12 – Good.

 Other  (Unfortunately, we need only students currently in grade 4, grade 8, or grade 12 for this study. Thank you for your time. Have a good day/evening.)

Thank you. For the research session, you will be asked to visit <LOCATION> and complete several tasks on a tablet computer using a variety of tools and interactions. Your responses will not be graded.

Your input will help evaluate the test that will be given to students throughout the nation. To thank you for participating, you will receive a $25 gift card (from a major credit card company). If your parent or guardian brings you to and from <LOCATION>, they will also receive a $25 gift card. The session will last about 75 minutes. Your participation is totally voluntary. If you participate, no one but the study’s researchers will see your responses and all other session materials, and we will not use your name to identify you.

Would you like to participate?

 Yes

 No

If NO: Thank you for your time. I will need to let your parent know that you would not like to participate. Is he or she available?

TO PARENT: Your child has decided not to participate, but I would like to thank you very much for taking the time to speak with us about this opportunity.

If YES: Thank you for agreeing to participate in this important research study. I will need to speak to your parent to talk about the date/time of the session. Is your parent available?

 Yes

 No

[INTERVIEWER: If calling parent back, record name, phone number, date/time]

Parent/Guardian Name ______________________________________________

Child Name ________________________________________________________

Parent Phone ______________________________________________________

Date ______________________________________________________________

Time ______________________________________________________________

IF PARENT AVAILABLE AND CURRENTLY SCHEDULING INTERVIEWS

Thank you again for allowing your child to participate in the NAEP research sessions. We will be holding our current sessions from <DATE> to <DATE>. What would be the best date/time to schedule the session?

Date ____________________

Time ____________________

Thank you. We will follow up with a confirmation email/letter and a consent form. What is your email or postal address for us to send these to?

____________________________________________________________________________________

IF PARENT AVAILABLE AND NOT CURRENTLY SCHEDULING SESSIONS

Thank you again for allowing your child to participate in the NAEP research sessions. We are in between research sessions, and will need to call you back when we are actively scheduling the sessions to determine the date and time that will work best. What are your scheduling preferences? [Select all that apply]

 Morning

 Afternoon

 After school

 Weekends

Thank you so much for talking with us today. We look forward to communicating with you soon.

Appendix H: EurekaFacts Recruitment Phone Script for Students Age 18 or Above Who Responded to Email, Advertisement, or Invitation

May I please speak to <STUDENT’S NAME>? Good <MORNING/AFTERNOON/EVENING>. My name is <NAME>. I’m calling from EurekaFacts on behalf of the U.S. Department of Education’s National Center for Educational Statistics, also known as NCES. Thank you for contacting us about the possibility of participating in a research session to help test new testing tools for the National Assessment of Educational Progress, also known as the NAEP assessment. First, I have a few brief screening questions to ask you about yourself. It should take only a few minutes of your time.

Before proceeding, please confirm that you are still interested in the possibility of participating in this research interview?

Yes: [INTERVIEWER: Proceed to “Screener.”]

No: (Thank you for your time. Have a good <day/evening>.)

Screener:

[INTERVIEWER: Record all responses]

What is your name?

What grade are you currently in (OR, IF INTERVIEW OCCURS IN THE SUMMER, What grade will you be in this fall)? (If the student indicates that he/she is in any grade other than 12, thank the student and end call: Unfortunately we are interviewing only students currently in grades 4, 8, or 12 for this study. Thank you for your time. Have a good <day/evening>.)

What school do you attend? [INTERVIEWER: Note if homeschooled]

____________________________________________________________

What school district do you attend? [INTERVIEWER: Note if parochial, charter, homeschooled] __________________________________________________

What is your race or ethnicity? [INTERVIEWER: Read list only if respondent doesn’t immediately reply. “Are you…?”]

Ethnicity:  Hispanic or Latino  Not Hispanic or Latino

Race (one or more of the categories below):

American Indian or Alaska Native
Asian
Black or African American
Hispanic or Latino
Native Hawaiian or other Pacific Islander
White or Caucasian

[If the respondent indicates they prefer not to disclose this information, the interviewer should say “That’s fine” and move on to the next question.]

Just to be sure we're working with a broad range of students for this study, do you participate in the National School Lunch program?

Yes
No
Refused to disclose

[If the respondent indicates they prefer not to disclose this information, the interviewer should say “That’s fine” and move on to the next question.]

Would you consider the area you live in to be …

Urban
Suburban
Rural

Ok. Thank you and what is the zip code of the area that you live in?

________________________

Would you like to participate?

Yes
No  (THANK AND END CALL)

If NO: Thank you for your time, and have a wonderful <day/evening>.

If YES:

Thank you for agreeing to participate in this important research study. For the research session, you will be asked to visit <LOCATION> and complete several tasks on a tablet computer using a variety of tools and interactions. Your responses will not be graded.

Your input will help us evaluate the test that will be given to students throughout the nation. To thank you for participating, you will receive a $25 gift card (from a major credit card company). If your parent or legal guardian brings you to and from our office, he or she will also receive a $25 gift card. The session will last about 75 minutes. Your participation is totally voluntary. If you participate, no one but the study’s researchers will see your responses and all other session materials, and we will not use your name to identify you.

Please note that before your interview starts, you will be asked to sign a consent form to confirm that you have agreed to participate in the interview. You will not be able to participate until the form has been signed.

IF CURRENTLY SCHEDULING INTERVIEWS

Thank you again for agreeing to participate in the NAEP research sessions. We will be holding our current sessions from <DATE> to <DATE>. What would be the best date/time to schedule the session?

Date ____________________

Time ____________________

IF NOT CURRENTLY SCHEDULING SESSIONS

Thank you again for agreeing to participate in the NAEP research sessions. We are in between research sessions, and will need to call you back when we are actively scheduling the sessions to determine the date and time that will work best. What are your scheduling preferences? [Select all that apply]

 Morning

 Afternoon

 After school

 Weekends

Thank you so much for talking with us today. We look forward to communicating with you soon.

Appendix I: EurekaFacts Recruitment Email/Letter to Schools of Potential Participants

<DATE>

IF SENT AS EMAIL - Subject: Research Study Opportunity

I am contacting you from EurekaFacts, a local research company. We are carrying out an important study for the National Center for Education Statistics (NCES), part of the U.S. Department of Education. We are contacting you to ask for your help in spreading the word about this exciting research opportunity.

We are looking for students in grades 4, 8, and 12 to complete several tasks on a tablet computer using a variety of tools and interactions. These tasks will be part of the National Assessment of Educational Progress (NAEP) delivered to students nationwide. It is vital for us to have student feedback to develop intuitive tools and item types.

The voluntary research sessions will last no more than 75 minutes. Students will be asked to interact with a variety of tools and item types on a tablet computer, then provide feedback about their experiences with completed tasks. If selected to participate, the student will receive a $25 gift card (from a major credit card company) from us. The student’s parent or guardian will also receive a $25 gift card to thank him or her for bringing their child to and from the interview site.

Students will have a key role in helping us improve our tests, and can enhance the experience of thousands of students all over the U.S. who will take them. Student responses will be used only for research purposes and will not be disclosed or used, in identifiable form, for any other purpose except as required by law, under the Education Sciences Reform Act of 2002 (ESRA 2002) 20 U.S.C., § 9573.

If you are willing to help us recruit students for this study, or would like additional information about it, please contact me, and we will follow up as soon as possible. I have included a fact sheet with additional information about this research study.

We appreciate you considering this request, and I invite you to please contact me with any questions regarding this project. I look forward to communicating with you.

Thank you,

<NAME>

EurekaFacts, LLC

ATTACHMENT: Study Fact Sheet <Appendix V>

Appendix J: EurekaFacts Recruitment Email/Letter to Parent-Teacher-Student Associations/Youth Organizations

<DATE>

IF SENT AS EMAIL - Subject: Research Study Opportunity

Hello,

I am contacting you from EurekaFacts, a local research company. We are carrying out an important study for the National Center for Education Statistics (NCES), part of the U.S. Department of Education. We would like to ask for your help in spreading the word about this exciting research opportunity.

The voluntary research sessions will last no more than 75 minutes. Students will be asked to interact with a variety of tools and item types on a tablet computer, then provide feedback about their experiences with completed tasks. If selected to participate, the student will receive a $25 gift card (from a major credit card company) from us. The student’s parent or guardian, who brings their child to and from the interview site, will also receive a $25 gift card.

Participating students will have a key role in helping us improve our tests, and can enhance the experience of thousands of students all over the U.S. who will take them. Student responses will be used only for research purposes and will not be disclosed or used, in identifiable form, for any other purpose except as required by law (Education Sciences Reform Act of 2002, 20 U.S.C §9573).

I have included a short description of this research study in the hope that you might pass it along to anyone who may be interested.

We appreciate you considering this request, and I invite you to please contact me with any questions regarding this project. I look forward to communicating with you.

Thank you,

EurekaFacts, LLC

ATTACHMENT: Study Fact Sheet <Appendix V>

Appendix K: EurekaFacts Recruitment Email/Letter to Adults who have Requested Information on Research Study Opportunities

<DATE>

IF SENT AS EMAIL - Subject: Education Research Opportunity for Students in Grades 4, 8, or 12

Hello,

I am contacting you from EurekaFacts, a local research company in Rockville, MD. You are receiving this email because you have previously given us your address in order to receive information regarding upcoming research opportunities.

In case you are the parent or legal guardian of a child who is currently in grades 4, 8, or 12, or know anyone who is, we’d like to inform you that we are currently carrying out a research study on behalf of the National Center for Education Statistics (NCES), part of the U.S. Department of Education. This study supports the development of new testing tools and item types for the National Assessment of Educational Progress (NAEP).

We are conducting research sessions to try out new approaches to test questions using interactive computer software. Students will be asked to complete tasks on a tablet computer. The research sessions will explore how students work through the tasks on a tablet computer to ensure that the tasks are clear and understandable to a broad range of students. NCES simply wants to know how students would do the tasks if they were on an actual test, and would like to hear students’ thoughts to improve the tasks and make them easier to understand.

Each student who participates will receive a $25 gift card (from a major credit card company). The student’s parent or guardian will also receive a $25 gift card for bringing the student to and from the session site.

Students’ responses will not be graded, or shared with their school or anyone other than the researchers. These research sessions are an important stage of the NAEP development process. The students can directly guide the future of this nationwide assessment by helping NCES create the best test possible.

If a student chooses to participate, his or her responses will be used only for research purposes and will not be disclosed or used, in identifiable form, for any other purpose except as required by law (Education Sciences Reform Act of 2002, 20 U.S.C §9573).

I have included a short description of this research study in the hope that you might pass it along to anyone who may be interested.

Thank you for your consideration and assistance regarding this effort. Please do not hesitate to contact me with any questions you may have regarding our firm or this project. We look forward to communicating with you.

Sincerely,

EurekaFacts, LLC

ATTACHMENT: Study Fact Sheet <Appendix V>

Appendix L: EurekaFacts Recruitment Email/Letter to Organizations that have Requested Additional Information About Usability Studies

<DATE>

IF SENT AS EMAIL - Subject: Education Research Opportunity for Students in Grades 4, 8, or 12

Hello,

I am contacting you from EurekaFacts, a local research company in Rockville, MD. You are receiving this email because you have previously requested additional information about a research study examining testing tools on tablet computers that we are conducting on behalf of National Center for Education Statistics (NCES), part of the U.S. Department of Education.

As we previously mentioned, we are currently recruiting students in grades 4, 8, or 12 for a research study supporting the development of new test questions for the National Assessment of Educational Progress (NAEP). We are conducting research sessions to try out new test questions using interactive computer software. Students will be asked to complete tasks on a tablet computer. The research sessions will explore how students work through the tasks on a tablet computer to ensure that the tasks are clear and understandable to a broad range of students. NCES simply wants to know how students would do the tasks if they were on an actual test, and would like to hear students’ thoughts on how to improve the tasks and make them easier to understand.

Each student who participates will receive a $25 gift card (from a major credit card company). The student’s parent or guardian, who brings their child to and from the session site, will also receive a $25 gift card.

I have included a short description of this research study in the hope that you might pass it along to anyone who may be interested. Upon request, I can also send you our Frequently Asked Questions (FAQs).

Sincerely,

EurekaFacts, LLC

ATTACHMENT: Study Fact Sheet <Appendix V>

Appendix M: EurekaFacts Recruitment Email/Letter to Parents/Guardians of Student Participants (Below Age 18)

<DATE>

IF SENT AS EMAIL - Subject: Research Study Opportunity

Dear <NAME>,

We are looking for students in grades 4, 8, and 12 to help us test new computer-delivered assessment tools and questions. These questions will be part of the National Assessment of Educational Progress (NAEP) delivered to students nationwide. NAEP serves as the central means by which the United States measures student proficiency in core academic domains, such as reading, writing, math, and science.

These voluntary research sessions will last no more than 75 minutes. Students will be asked to interact with a variety of tools and item types on a tablet computer, then provide feedback about their experiences with completed tasks. Both students and their parent/guardian providing transportation to and from the survey will be thanked for their time and effort with a $25 gift card (from a major credit card company) upon completion of the survey.

Students will be informed that their participation is voluntary, and that their work will not be graded or shared with their schools.

Participating students will have a key role in helping us improve our tests, and can enhance the experience of thousands of students all over the U.S. who will take them. Your child’s responses would be used only for research purposes and will not be disclosed or used, in identifiable form, for any other purpose except as required by law (Education Sciences Reform Act of 2002, 20 U.S.C §9573).

I have included additional information about this research study. If your child is interested in participating in this research study, please call <CONTACT PHONE> or email <CONTACT EMAIL>. We look forward to communicating with you soon!

Thank you,

<NAME>

EurekaFacts, LLC

ATTACHMENT: Study Fact Sheet <Appendix V>

Appendix N: EurekaFacts Confirmation of Interview (Email/Letter) to Parent/Guardian of Student Participant (Below Age 18)

<DATE>

IF SENT AS EMAIL - Subject: Confirmation of Your Child’s NAEP Research Study

Dear <NAME>,

Thank you for allowing your child to participate in a research study for the National Assessment of Educational Progress (NAEP). Before giving new tests to all students, the National Center for Education Statistics (NCES), part of the U.S. Department of Education, needs to receive feedback on new tools and item types. Your child’s responses will be used only for research purposes and will not be disclosed or used, in identifiable form, for any other purpose except as required by law (Education Sciences Reform Act of 2002, 20 U.S.C §9573).

Your child has been scheduled for a research interview at <TIME> on <DAY, DATE>. The interview will last approximately 75 minutes. Students will be asked to interact with a variety of tools and item types on a tablet computer, then provide feedback about their experiences with completed tasks. Your child will receive a $25 gift card (from a major credit card company) as a thank you for participating. In addition, if you bring your child to and from the research site, you will receive a $25 gift card.

The session will take place at:

We have attached a Consent Form to this reminder. Please sign and date it, and return the form granting consent for your child to participate in the NAEP research when your child comes in for the session. Without your consent, your child will not be able to participate in the session.

Thank you for allowing your child to volunteer in this important research for NAEP. Our researcher is looking forward to meeting with your child. If you have any questions, please contact me.

Thank you,

<NAME>

EurekaFacts, LLC

ATTACHMENT: Consent Form <Appendix T>

Appendix O: EurekaFacts Confirmation of Interview (Email/Letter) to Student Participant (Age 18 or Above)

<DATE>

IF SENT AS EMAIL - Subject: Confirmation of Your Participation in a NAEP Research Study

Dear <NAME>,

Thank you for agreeing to participate in this voluntary research study to provide feedback on questions for the National Assessment of Educational Progress (NAEP). Before giving new tests to all students, the National Center for Education Statistics (NCES), part of the U.S. Department of Education, needs to receive feedback on new tools and item types. All the information you provide will be used only for research purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (Education Sciences Reform Act of 2002, 20 U.S.C §9573).

You have been scheduled for a session at <TIME> on <DAY, DATE>. Your will receive a $25 gift card (from a major credit card company) as a thank you for participating. In addition, your parent or legal guardian will receive a $25 gift card for the time involved and to help offset the travel/transportation costs of taking you to and from the research site.

The session will take place at:

We have attached a Consent Form to this confirmation. Please sign, date, and return the form when you come to the interview, agreeing to participate in the voluntary NAEP research.

We look forward to having you participate in this very important research study. If you have any questions, please contact me.

Thank you,

<NAME>

EurekaFacts, LLC

ATTACHMENT: Consent Form <Appendix U>

Appendix P: EurekaFacts Confirmation (Email/Letter) to School/Organization Where Sessions Will Be Conducted

<DATE>

IF SENT AS EMAIL - Subject: Confirmation of NAEP Research Study

Dear <NAME>,

Thank you for assisting EurekaFacts in carrying out research studies for the National Assessment of Educational Progress (NAEP). Students’ responses will be used only for research purposes and will not be disclosed or used, in identifiable form, for any other purpose except as required by law (Education Sciences Reform Act of 2002, 20 U.S.C §9573).

Sessions have been scheduled at <SCHOOL/ORGANIZATION> from <TIME to TIME> on <DAY, DATE>. The sessions will each last approximately 75 minutes. To thank them for their time, participating students will receive a $25 gift card (from a major credit card company).

[Include the following information if applicable: If a parent/ guardian transports the student to and from the research site, the parent/guardian will also receive a $25 gift card.]

Thank you for assisting us in this important research for NAEP. If you have any questions, please contact me.

Thank you,

<NAME>

EurekaFacts, LLC

Appendix Q: EurekaFacts Thank You Email/Letter to Parents or Legal Guardians of Student Participants (Below Age 18)

<DATE>

IF SENT AS EMAIL - Subject: Thank you for your child’s participation in the NAEP Research Study

Dear <NAME>,

On behalf of the National Center for Education Statistics (NCES), part of the U.S. Department of Education, EurekaFacts would like to thank you for allowing your child[ren] to participate in our research study. Thanks to your family’s participation, we are better able to develop the best possible test for the National Assessment of Educational Progress (NAEP).

Your child[ren]’s assistance helped us better understand how students approach the test questions and tools. The students’ open, candid, and insightful comments were very helpful.

Again, thank you for your assistance.

Sincerely,

<NAME>

<TITLE>

EurekaFacts, LLC

Appendix R: EurekaFacts Thank You Email/Letter to Student Participants Age 18 or Above

<DATE>

IF SENT AS EMAIL - Subject: Thank you for your participation in the NAEP Research Study

Dear <NAME>,

On behalf of the National Center for Education Statistics (NCES), part of the U.S. Department of Education, EurekaFacts would like to thank you for your participation in our research study. Thanks to your participation, we are better able to develop the best possible test for the National Assessment of Educational Progress (NAEP).

Your assistance helped us better understand how students approach both the test questions and tools. Your open, candid, and insightful comments were very helpful.

Again, thank you for your assistance.

Sincerely,

<NAME>

<TITLE>

EurekaFacts, LLC

Appendix S: EurekaFacts Thank You Email/Letter to School/Organization/Association of Student Participants

<DATE>

IF SENT AS EMAIL - Subject: Thank you for your participation in the NAEP Research Study

Dear <NAME>,

On behalf of the National Center for Education Statistics, part of the U.S. Department of Education, thank you for your assistance with recruitment of students for the National Assessment of Educational Progress (NAEP) research study. Your students’ assistance helped us better understand how students approach the test questions and tools. Your students’ open, candid, and insightful comments were very helpful. The assistance provided by your students, combined with input from other students, were useful and informative. We greatly appreciate you having taken the time to support our important research effort.

Again, thank you for your assistance.

Sincerely,

EurekaFacts, LLC

Appendix T: EurekaFacts Consent Form for Parents of Student Participants (Below Age 18)

PARENTAL CONSENT DOCUMENT

FOR PARTICIPATION IN RESEARCH

EurekaFacts, LLC

Title of Research Study: Item Types and Tools Usability Study

Thank you for your interest in participating in the research project described below. This consent form explains the research study. Please read it carefully, and feel free to ask questions about anything you do not understand. If you do not have questions now, you may ask them if they occur to you later.

What is this study about?

The National Center for Education Statistics (NCES), part of the U.S. Department of Education, is conducting a study with students currently in grades 4, 8, and 12 on new tools and item types for the National Assessment of Educational Progress (NAEP). NCES needs to know whether these new tools and item types are intuitive and easy to use. Students’ input will help NCES improve the test so that students such as your child will understand it.

Where will the research interviews take place?

The interviews will take place at <NAME OF INTERVIEW LOCATION>, located at <LOCATION>, near <NEARBY LANDMARK OR METRO STATION>.

What will happen during the research interview?

Students will be asked to complete several tasks regarding new tools and item types. The research sessions will explore how students work through the tasks on a tablet computer to ensure that the tasks are clear and understandable to a broad range of students. NCES simply wants to know how students would do the tasks if they were on an actual test, and would like to hear students’ thoughts to improve the tasks and make them easier to understand.

How long will the research interview last and when will it take place?

The interview will last up to 75 minutes and will require only one visit.

Will you keep information confidential?

Your child will be assigned a unique student identifier (ID), and at no time will his/her name be linked to any of his/her answers. Your child’s responses will be used only for research purposes and will not be disclosed or used, in identifiable form, for any other purpose except as required by law (Education Sciences Reform Act of 2002, 20 U.S.C §9573). The screens of the tablets will be recorded, along with the audio from the session. No cameras will be used, and no images or video of your child or their surroundings will be captured.

Will the results go to my child’s school?

Your child’s answers will not be disclosed to his/her school, and will not be linked to any personally identifiable information such as the school’s name.

What are the possible risks of being in the study?

There are no known risks to participating in this study.

Who can I contact with questions or for further information?

<CONTACT NAME> is carrying out this study at EurekaFacts on behalf of NCES. If you have any questions, please call him/her at <CONTACT PHONE> or by email <CONTACT EMAIL>.

Can participation end early?

Your participation in this study is voluntary. Your child has the right to refuse to answer particular questions. You may elect to withdraw from this study at any time.

By signing below, you agree that your child, _______________________________, may participate in this study. Your child will receive a $25 gift card (from a major credit card company) at the conclusion of the session. In addition, if you bring your child to and from the research site, you will also receive a $25 gift card.

Your signature below will indicate that your questions have been answered satisfactorily, and that you have read and understood the information provided above.

Print Name: _________________________________________________________________________

Signature: ___________________________________________________________________________

Date:_______________________________________________________________________________

School that student attends: ____________________________________________________________

Appendix U: EurekaFacts Consent Form for Adult Student Participants (Age 18 and Above)

PARTICIPATION CONSENT DOCUMENT

FOR PARTICIPATION IN RESEARCH

EurekaFacts, LLC

Title of Research Study: Item Types and Tools Usability Study

What is this study about?

Where will the research interviews take place?

The interviews will take place at <NAME OF INTERVIEW LOCATION>, located at <LOCATION>, near <NEARBY LANDMARK OR METRO STATION>.

What will happen during the research interview?

How long will the research interview last and when will it take place?

The interview will last up to 75 minutes and will require only one visit.

Will you keep information confidential?

You will be assigned a unique student identifier (ID), and at no time will your name be linked to any of your answers. Your responses will be used only for research purposes and will not be disclosed or used, in identifiable form, for any other purpose except as required by law (Education Sciences Reform Act of 2002, 20 U.S.C §9573). The screens of the tablets will be recorded, along with the audio from the session. No cameras will be used, and no images or video of you or your surroundings will be captured.

Will the results go to my school?

Your answers will not be disclosed to your school, and will not be linked to any personally identifiable information such as your school’s name.

What are the possible risks of being in the study?

There are no known risks to participating in this study.

Who can I contact with questions or for further information?

<CONTACT NAME> is carrying out this study at EurekaFacts on behalf of NCES. If you have any questions, please call him/her at <CONTACT PHONE> or by email <CONTACT EMAIL>.

Can participation end early?

Your participation in this study is voluntary. You have the right to refuse to answer particular questions. You may elect to withdraw from this study at any time.

By signing below, you agree that you will participate in this study. You will receive a $25 gift card (from a major credit card company) at the conclusion of the session. In addition, if your parent/guardian brings you to and from the research site, he or she will also receive a $25 gift card.

Your signature below will indicate that your questions have been answered satisfactorily, and that you have read and understood the information provided above.

Print Name: _________________________________________________________________________

Signature: ___________________________________________________________________________

Date:_______________________________________________________________________________

School that you attend: ________________________________________________________________

Appendix V: EurekaFacts Fact Sheet related to the Usability Study

Item Types and Tools Usability Study

Fact Sheet

1. What is this study about?

2. Where will the research interviews take place?

The interviews will take place at <NAME OF LOCATION>, located at <LOCATION>, near <NEARBY LANDMARK OR METRO STATION>.

3. What will happen during the interviews?

Students will be asked to complete several tasks regarding new tools and item types. The research sessions will explore how students work through the tasks on a tablet computer to ensure that the tasks are clear and understandable to a broad range of students. NCES simply wants to know how students would do the tasks if they were on an actual test, and would like to hear students’ thoughts on how to improve the tasks and make them easier to understand.

4. How long will the Interview last and when it will take place?

The interview session will last about 75 minutes and can be scheduled at your convenience. Each interview requires only one visit.

5. Whose consent do you require for participation?

For students below age 18: Once we have received informed consent from the parent/guardian for their eligible student to participate, we will then also ask the student for his or her consent to participate. Participation in the study is completely voluntary. The parent/guardian or student may withdraw consent for the student’s participation in the study at any time. The student also has the right to refuse to answer particular questions. Students age 18 or above can sign the consent form directly. Again, participation in the study is voluntary, and students have the right to withdraw intent to participate and/or refuse to answer particular questions.

6. Will you keep information confidential?

All of the participant’s responses will be used only for research purposes and will not be disclosed or used, in identifiable form, for any other purpose except as required by law [Education Sciences Reform Act of 2002 (ESRA 2002) 20 U.S.C., § 9573]. Each participant will be assigned a unique student identifier (ID), and at no time will his/her name be linked to any of his/her answers.

7. Will the results go to the participant’s school?

Participants’ responses will not be disclosed to his/her school, and will not be linked to any personally identifiable information such as the school’s name.

8. Are there rewards or incentives for participating?

Yes, the student will receive a $25 gift card (from a major credit card company) at the end of his or her interview. If the student’s parent or guardian brings the student to and from the interview site, he or she will also receive a $25 gift card.

9. What are the possible risks of being in the study?

There are no known risks to participating in this study.

10. Whom can I contact with questions or for further information?

If you have any questions, please call [CONTACT NAME] from EurekaFacts at [CONTACT PHONE] or [CONTACT EMAIL].

Appendix W: EurekaFacts Sample Recruitment Flyer for the Usability Study

PARTICIPANTS NEEDED!

EurekaFacts, a local research company in Rockville, MD, is conducting a study for the National Center for Educational Statistics (NCES), a part of the U.S. Department of Education.

We are looking for students in either grades 4, 8, or 12 to help us evaluate new tools and items for the National Assessment of Educational Progress (NAEP), commonly known as “The Nation’s Report Card”.

We will provide a $25 gift card to eligible students who take part in the study, and an additional $25 gift card to the parent/guardian who brings the student to and from the study location. Participation takes no more than 75 minutes.

This study will run from <DATE OR SEASON> through <DATE OR SEASON>. Students can arrange a time to participate that fits with their schedule.

Interested? Call <CONTACT PHONE> or email <CONTACT EMAIL>, or head to our website at <WEBSITE>, for more information.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Title	Appendices (A-R)
Author	mgisbert
File Modified	0000-00-00
File Created	2021-01-27

Appendices - Recruitment and Communication Materials

Appendices - NAEP DBA Usability Study 2015-16.docx

NCES Cognitive, Pilot, and Field Test Studies System