National Assessment of Educational Progress (NAEP) Digitally Based Assessments (DBA) Usability Study 2017-18

Good morning/afternoon/evening. My name is <CONTACT NAME> and I am contacting you from EurekaFacts, a local research company. We are carrying out an important study for the National Center for Education Statistics (NCES), part of the U.S. Department of Education. We are recruiting students who are currently in grades 4, 8, and 12 to participate in research sessions supporting the development of new tools, tutorials, and item types for the National Assessment of Educational Progress (NAEP). The research sessions will explore how students work through the tasks on a tablet computer to ensure that the tasks are clear and understandable to a broad range of students.

The voluntary research sessions will last no more than 75 minutes. Students will be asked to complete tasks on a tablet and answer questions about how easy or hard it is to accomplish assessment functions, answer questions in different formats, and understand what the system expects them to do next. Before working on the tablets, students will be asked about their familiarity and frequency regarding usage of laptops, smart phones, and tablets.

Student responses will be used only for research purposes and will not be disclosed or used, in identifiable form, for any other purpose except as required by law.

[INTERVIEWER note: If asked what law is being referenced, please provide its name: Education Sciences Reform Act of 2002 (ESRA 2002) 20 U.S.C., § 9573.]

We would like your help in distributing flyers to students, and we will follow up directly with anyone who contacts us. For each participating student, a $25 gift card (from a major credit card company) will be issued in appreciation of their participation. In addition, if a parent or legal guardian brings the student to and from the site, he or she will also receive a $25 thank you gift card.

These sessions are an important stage of the NAEP test development process. The comments of participating students from your school will help NCES improve the NAEP test.

Would your school be interested in participating in or helping us to recruit students for this research study?

If NO: Thank you for your time. Have a good day/evening. [Interviewer: End call]

If YES: Great, thank you very much. Would you be able to send flyers to parents of potential student participants?

If NO: Thank you for your time. Have a good day/evening. [Interviewer: End call]

If YES: Would you be able to make a room at your school available where EurekaFacts could conduct some 75-minute research sessions?

If YES: That’s great, we will contact you to schedule some sessions at your school after we have some students signed up. [Interviewer: go to Closing]

If NO: Thank you for your time. Have a good day/evening. [Interviewer: End call]

(Closing)

Thank you for agreeing to disseminate information about this important research study. We will send you email with informational flyers.

Is <EMAIL ADDRESS> the best email to reach you? OR What is the best email address to reach you?

We may need to reach you by telephone. Is the number I called the best number to use?

Do you have any questions at this time?

[INTERVIEWER: Answer any questions.]

If you do have any questions, please call <PHONE NUMBER>.

Thank you again for agreeing to be part of this important research project. Have a wonderful day/evening.

Appendix B: EurekaFacts Recruitment Phone Script for Parents/Guardians of Participants (Below Age 18)

Script when calling parents from previously compiled list:

Good <MORNING/AFTERNOON/EVENING>. May I please speak to <NAME> about <STUDENT’S NAME> session for the National Assessment of Educational Progress (NAEP) usability study for tools and item types study?

My name is <CALLER NAME>, and I am calling from EurekaFacts, a local research company. We are carrying out a study on behalf of the U.S. Department of Education’s National Center for Education Statistics, also known as NCES.

We are recruiting students currently in grades 4, 8, or 12 to participate in a research study supporting the development of new tools, tutorials, and item types for the National Assessment of Educational Progress or NAEP. [If parent asks what this is: This national assessment is administered to students in elementary, middle, and high schools throughout the United States.]

Are there any students at grades 4, 8, or 12 in your household?

No: (Thank you, have a good day/evening.)

Yes: (Great! Let me give you some additional information about the study.)

The research session will take place at <NAME OF SESSION SITE AND LOCATION>. It can be scheduled at your convenience and will last about 75 minutes. During the session, your child will be asked to complete several tasks on a tablet computer using a variety of tools and interactions. NCES simply wants to know how students would do the tasks if they were on an actual test, how intuitive and easy to understand the tasks are, and to hear students’ thoughts on how to improve the tasks and make them easier to understand.

If selected to participate, your child will receive a $25 gift card (from a major credit card company) as thanks for his or her time. In addition, if you bring your child to and from the research site, you will also receive a $25 gift card.

These research sessions are an important stage of the NAEP development process. By sharing their feedback with us, students can directly guide the future of this nationwide assessment by helping NCES evaluate the test and instructions to create the best test possible.

Do you believe that your child would be interested in participating?

No: (Thank you anyway, have a good day/evening.)

Yes: (Great! Thank you so much. I have a few brief questions to ask about you and your child. It should take only a few minutes.

[Skip to Screener.]

Script when calling a parent/guardian who responded to email or letter

Good morning/afternoon/evening. My name is <NAME>. I’m calling from EurekaFacts on behalf of the National Center for Education Statistics, part of the U.S. Department of Education.

Thank you for contacting us about the possibility of your child participating in a research session to help assess testing tools for the National Assessment of Educational Progress, or NAEP. First, I have a few brief screening questions to ask about you and your child. It should only take a few minutes.

Screener

[INTERVIEWER: Record all responses.]

What is your child’s name?

What grade is your child currently in (OR, IF INTERVIEW OCCURS IN THE SUMMER, What grade will your child be in this fall)? [INTERVIEWER: Do not read list]

 Grade 4, 8, or 12 – Good.

 Other  (Unfortunately, we need only students currently in grade 4, grade 8, or grade 12 for this study. Thank you for your time. Have a good day/evening.)

What school does your child attend? [INTERVIEWER: Note if homeschooled]

____________________________________________________________

What school district does he or she attend? [INTERVIEWER: Note if parochial, charter, homeschooled] __________________________________________________

What is your child’s race or ethnicity? [INTERVIEWER: Read list only if parent doesn’t immediately reply. “Is he/she…?”]

Ethnicity:  Hispanic or Latino  Not Hispanic or Latino

Race (one or more of the categories below):

 American Indian or Alaska Native

 Asian

 Black or African American

 Hispanic or Latino

 Native Hawaiian or other Pacific Islander

 White or Caucasian

[If the respondent indicates they prefer not to disclose this information, the interviewer should say “That’s fine” and move on to the next question.]

Just to be sure we're working with a broad range of students for this study, does your child participate in the National School Lunch program?

 Yes

 No

 Refused to disclose

[If the respondent indicates they prefer not to disclose this information, the interviewer should say “That’s fine” and move on to the next question.]

Would you consider the area you live in to be …

 Urban

 Suburban

 Rural

Ok. Thank you and what is the zip code of the area that you live in?

______________

Thank you. We would like to speak with your child to confirm if (he/she) is interested in participating in the research session. It will take only a couple of minutes and then I will ask to speak with you again for another moment.

Is your child available to speak with us now or should we call (him/her) back at another time?

 Yes, the child is available for screening questions  [Skip to script for students]

 No, please call back.

Is the phone number you gave me earlier the best number to call?

Other phone numbers: ____________________________

Is there a good time for me to call? When would be best for you?

[INTERVIEWER: If child needs to receive a call back, record name, phone number, date/time]

Parent/Guardian Name ___________________________________________________________

Child Name ____________________________________________________________________

Child Phone ____________________________________________________________________

Date ____________________

Time ____________________

SCRIPT FOR STUDENTS

Good morning/afternoon/evening. My name is <NAME> and I’m calling from EurekaFacts, a local research company. We are doing an important study for the National Center for Education Statistics, part of the U.S. Department of Education.

We have spoken to your parent and we are pleased that you may be interested in helping us develop testing tools for a nationwide student assessment, the National Assessment of Educational Progress, or NAEP.

Before I tell you more about what you would be asked to do, I need to ask you one question:

What grade are you currently in (OR, IF INTERVIEW OCCURS IN THE SUMMER, What grade will you be in this fall)? [INTERVIEWER: Do not read list]

 Grade 4, 8, or 12 – Good.

 Other  (Unfortunately, we need only students currently in grade 4, grade 8, or grade 12 for this study. Thank you for your time. Have a good day/evening.)

Thank you. For the research session, you will be asked to visit <LOCATION> and complete several tasks on a tablet computer using a variety of tools and interactions. Your responses will not be graded.

Your input will help evaluate the test that will be given to students throughout the nation. To thank you for participating, you will receive a $25 gift card (from a major credit card company). If your parent or guardian brings you to and from <LOCATION>, they will also receive a $25 gift card. The session will last about 75 minutes. Your participation is totally voluntary. If you participate, no one but the study’s researchers will see your responses and all other session materials, and we will not use your name to identify you.

Would you like to participate?

 Yes

 No

If NO: Thank you for your time. I will need to let your parent know that you would not like to participate. Is he or she available?

TO PARENT: Your child has decided not to participate, but I would like to thank you very much for taking the time to speak with us about this opportunity.

If YES: Thank you for agreeing to participate in this important research study. I will need to speak to your parent to talk about the date/time of the session. Is your parent available?

 Yes

 No

[INTERVIEWER: If calling parent back, record name, phone number, date/time]

Parent/Guardian Name ______________________________________________

Child Name ________________________________________________________

Parent Phone ______________________________________________________

Date ______________________________________________________________

Time ______________________________________________________________

IF PARENT AVAILABLE AND CURRENTLY SCHEDULING INTERVIEWS

Thank you again for allowing your child to participate in the NAEP research sessions. We will be holding our current sessions from <DATE> to <DATE>. What would be the best date/time to schedule the session?

Date ____________________

Time ____________________

Thank you. We will follow up with a confirmation email/letter and a consent form. What is your email or postal address for us to send these to?

____________________________________________________________________________________

IF PARENT AVAILABLE AND NOT CURRENTLY SCHEDULING SESSIONS

Thank you again for allowing your child to participate in the NAEP research sessions. We are in between research sessions, and will need to call you back when we are actively scheduling the sessions to determine the date and time that will work best. What are your scheduling preferences? [Select all that apply]

 Morning

 Afternoon

 After school

 Weekends

Thank you so much for talking with us today. We look forward to communicating with you soon.

Appendix C: EurekaFacts Recruitment Phone Script for Students Age 18 or Above Who Responded to Email, Advertisement, or Invitation

May I please speak to <STUDENT’S NAME>? Good <MORNING/AFTERNOON/EVENING>. My name is <NAME>. I’m calling from EurekaFacts on behalf of the U.S. Department of Education’s National Center for Educational Statistics, also known as NCES. Thank you for contacting us about the possibility of participating in a research session to help test new testing tools for the National Assessment of Educational Progress, also known as the NAEP assessment. First, I have a few brief screening questions to ask you about yourself. It should take only a few minutes of your time.

Before proceeding, please confirm that you are still interested in the possibility of participating in this research interview?

Yes: [INTERVIEWER: Proceed to “Screener.”]

No: (Thank you for your time. Have a good <day/evening>.)

Screener:

[INTERVIEWER: Record all responses]

What is your name?

What grade are you currently in (OR, IF INTERVIEW OCCURS IN THE SUMMER, What grade will you be in this fall)? (If the student indicates that he/she is in any grade other than 12, thank the student and end call: Unfortunately we are interviewing only students currently in grades 4, 8, or 12 for this study. Thank you for your time. Have a good <day/evening>.)

What school do you attend? [INTERVIEWER: Note if homeschooled]

____________________________________________________________

What school district do you attend? [INTERVIEWER: Note if parochial, charter, homeschooled] __________________________________________________

What is your race or ethnicity? [INTERVIEWER: Read list only if respondent doesn’t immediately reply. “Are you…?”]

Ethnicity:  Hispanic or Latino  Not Hispanic or Latino

Race (one or more of the categories below):

American Indian or Alaska Native
Asian
Black or African American
Hispanic or Latino
Native Hawaiian or other Pacific Islander
White or Caucasian

[If the respondent indicates they prefer not to disclose this information, the interviewer should say “That’s fine” and move on to the next question.]

Just to be sure we're working with a broad range of students for this study, do you participate in the National School Lunch program?

Yes
No
Refused to disclose

[If the respondent indicates they prefer not to disclose this information, the interviewer should say “That’s fine” and move on to the next question.]

Would you consider the area you live in to be …

Urban
Suburban
Rural

Ok. Thank you and what is the zip code of the area that you live in?

________________________

Would you like to participate?

Yes
No  (THANK AND END CALL)

If NO: Thank you for your time, and have a wonderful <day/evening>.

If YES:

Thank you for agreeing to participate in this important research study. For the research session, you will be asked to visit <LOCATION> and complete several tasks on a tablet computer using a variety of tools and interactions. Your responses will not be graded.

Your input will help us evaluate the test that will be given to students throughout the nation. To thank you for participating, you will receive a $25 gift card (from a major credit card company). If your parent or legal guardian brings you to and from our office, he or she will also receive a $25 gift card. The session will last about 75 minutes. Your participation is totally voluntary. If you participate, no one but the study’s researchers will see your responses and all other session materials, and we will not use your name to identify you.

Please note that before your interview starts, you will be asked to sign a consent form to confirm that you have agreed to participate in the interview. You will not be able to participate until the form has been signed.

IF CURRENTLY SCHEDULING INTERVIEWS

Thank you again for agreeing to participate in the NAEP research sessions. We will be holding our current sessions from <DATE> to <DATE>. What would be the best date/time to schedule the session?

Date ____________________

Time ____________________

IF NOT CURRENTLY SCHEDULING SESSIONS

Thank you again for agreeing to participate in the NAEP research sessions. We are in between research sessions, and will need to call you back when we are actively scheduling the sessions to determine the date and time that will work best. What are your scheduling preferences? [Select all that apply]

 Morning

 Afternoon

 After school

 Weekends

Thank you so much for talking with us today. We look forward to communicating with you soon.

Appendix D: EurekaFacts Recruitment Email/Letter to Schools of Potential Participants

<DATE>

IF SENT AS EMAIL - Subject: Research Study Opportunity

Hello,

I am contacting you from EurekaFacts, a local research company. We are carrying out an important study for the National Center for Education Statistics (NCES), part of the U.S. Department of Education. We are contacting you to ask for your help in spreading the word about this exciting research opportunity.

We are looking for students in grades 4, 8, and 12 to complete several tasks on a tablet computer using a variety of tools and interactions. These tasks will be part of the National Assessment of Educational Progress (NAEP) delivered to students nationwide. It is vital for us to have student feedback to develop intuitive tools and item types.

The voluntary research sessions will last no more than 75 minutes. Students will be asked to interact with a variety of tools and item types on a tablet computer, then provide feedback about their experiences with completed tasks. If selected to participate, the student will receive a $25 gift card (from a major credit card company) from us. The student’s parent or guardian will also receive a $25 gift card to thank him or her for bringing their child to and from the interview site.

Students will have a key role in helping us improve our tests, and can enhance the experience of thousands of students all over the U.S. who will take them. Student responses will be used only for research purposes and will not be disclosed or used, in identifiable form, for any other purpose except as required by law, under the [Education Sciences Reform Act of 2002 (ESRA 2002) 20 U.S.C., § 9573].

If you are willing to help us recruit students for this study, or would like additional information about it, please contact me, and we will follow up as soon as possible. I have included a fact sheet with additional information about this research study.

We appreciate you considering this request, and I invite you to please contact me with any questions regarding this project. I look forward to communicating with you.

Thank you,

<NAME>

EurekaFacts, LLC

ATTACHMENT: Study Fact Sheet <Appendix Q>

Appendix E: EurekaFacts Recruitment Email/Letter to Parent-Teacher-Student Associations/Youth Organizations

<DATE>

IF SENT AS EMAIL - Subject: Research Study Opportunity

Hello,

I am contacting you from EurekaFacts, a local research company. We are carrying out an important study for the National Center for Education Statistics (NCES), part of the U.S. Department of Education. We would like to ask for your help in spreading the word about this exciting research opportunity.

The voluntary research sessions will last no more than 75 minutes. Students will be asked to interact with a variety of tools and item types on a tablet computer, then provide feedback about their experiences with completed tasks. If selected to participate, the student will receive a $25 gift card (from a major credit card company) from us. The student’s parent or guardian, who brings their child to and from the interview site, will also receive a $25 gift card.

Participating students will have a key role in helping us improve our tests, and can enhance the experience of thousands of students all over the U.S. who will take them. Student responses will be used only for research purposes and will not be disclosed or used, in identifiable form, for any other purpose except as required by law (Education Sciences Reform Act of 2002, 20 U.S.C §9573).

I have included a short description of this research study in the hope that you might pass it along to anyone who may be interested.

We appreciate you considering this request, and I invite you to please contact me with any questions regarding this project. I look forward to communicating with you.

Thank you,

EurekaFacts, LLC

ATTACHMENT: Study Fact Sheet <Appendix Q>

Appendix F: EurekaFacts Recruitment Email/Letter to Adults who have Requested Information on Research Study Opportunities

<DATE>

IF SENT AS EMAIL - Subject: Education Research Opportunity for Students in Grades 4, 8, or 12

Hello,

I am contacting you from EurekaFacts, a local research company in Rockville, MD. You are receiving this email because you have previously given us your address in order to receive information regarding upcoming research opportunities.

In case you are the parent or legal guardian of a child who is currently in grades 4, 8, or 12, or know anyone who is, we’d like to inform you that we are currently carrying out a research study on behalf of the National Center for Education Statistics (NCES), part of the U.S. Department of Education. This study supports the development of new testing tools and item types for the National Assessment of Educational Progress (NAEP).

We are conducting research sessions to try out new approaches to test questions using interactive computer software. Students will be asked to complete tasks on a tablet computer. The research sessions will explore how students work through the tasks on a tablet computer to ensure that the tasks are clear and understandable to a broad range of students. NCES simply wants to know how students would do the tasks if they were on an actual test, and would like to hear students’ thoughts to improve the tasks and make them easier to understand.

Each student who participates will receive a $25 gift card (from a major credit card company). The student’s parent or guardian will also receive a $25 gift card for bringing the student to and from the session site.

Students’ responses will not be graded, or shared with their school or anyone other than the researchers. These research sessions are an important stage of the NAEP development process. The students can directly guide the future of this nationwide assessment by helping NCES create the best test possible.

If a student chooses to participate, his or her responses will be used only for research purposes and will not be disclosed or used, in identifiable form, for any other purpose except as required by law [Education Sciences Reform Act of 2002, 20 U.S.C §9573].

I have included a short description of this research study in the hope that you might pass it along to anyone who may be interested.

Thank you for your consideration and assistance regarding this effort. Please do not hesitate to contact me with any questions you may have regarding our firm or this project. We look forward to communicating with you.

Sincerely,

EurekaFacts, LLC

ATTACHMENT: Study Fact Sheet <Appendix Q>

Appendix G: EurekaFacts Recruitment Email/Letter to Organizations that have Requested Additional Information About Usability Studies

<DATE>

IF SENT AS EMAIL - Subject: Education Research Opportunity for Students in Grades 4, 8, or 12

Hello,

I am contacting you from EurekaFacts, a local research company in Rockville, MD. You are receiving this email because you have previously requested additional information about a research study examining testing tools on tablet computers that we are conducting on behalf of National Center for Education Statistics (NCES), part of the U.S. Department of Education.

As we previously mentioned, we are currently recruiting students in grades 4, 8, or 12 for a research study supporting the development of new test questions for the National Assessment of Educational Progress (NAEP). We are conducting research sessions to try out new test questions using interactive computer software. Students will be asked to complete tasks on a tablet computer. The research sessions will explore how students work through the tasks on a tablet computer to ensure that the tasks are clear and understandable to a broad range of students. NCES simply wants to know how students would do the tasks if they were on an actual test, and would like to hear students’ thoughts on how to improve the tasks and make them easier to understand.

Each student who participates will receive a $25 gift card (from a major credit card company). The student’s parent or guardian, who brings their child to and from the session site, will also receive a $25 gift card.

I have included a short description of this research study in the hope that you might pass it along to anyone who may be interested. Upon request, I can also send you our Frequently Asked Questions (FAQs).

Sincerely,

EurekaFacts, LLC

ATTACHMENT: Study Fact Sheet <Appendix Q>

Appendix H: EurekaFacts Recruitment Email/Letter to Parents/Guardians of Student Participants (Below Age 18)

<DATE>

IF SENT AS EMAIL - Subject: Research Study Opportunity

Dear <NAME>,

We are looking for students in grades 4, 8, and 12 to help us test new tablet-delivered assessment tools and questions. These questions will be part of the National Assessment of Educational Progress (NAEP) delivered to students nationwide. NAEP serves as the central means by which the United States measures student proficiency in core academic domains, such as reading, writing, math, and science.

These voluntary research sessions will last no more than 75 minutes. Students will be asked to interact with a variety of tools and item types on a tablet computer, then provide feedback about their experiences with completed tasks. Both students and their parent/guardian providing transportation to and from the survey will be thanked for their time and effort with a $25 gift card (from a major credit card company) upon completion of the survey.

Students will be informed that their participation is voluntary, and that their work will not be graded or shared with their schools.

Participating students will have a key role in helping us improve our tests, and can enhance the experience of thousands of students all over the U.S. who will take them. Your child’s responses would be used only for research purposes and will not be disclosed or used, in identifiable form, for any other purpose except as required by law [Education Sciences Reform Act of 2002, 20 U.S.C §9573].

I have included additional information about this research study. If your child is interested in participating in this research study, please call <CONTACT PHONE> or email <CONTACT EMAIL>. We look forward to communicating with you soon!

Thank you,

<NAME>

EurekaFacts, LLC

ATTACHMENT: Study Fact Sheet <Appendix Q>

Appendix I: EurekaFacts Confirmation of Interview (Email/Letter) to Parent/Guardian of Student Participant (Below Age 18)

<DATE>

IF SENT AS EMAIL - Subject: Confirmation of Your Child’s NAEP Research Study

Dear <NAME>,

Thank you for allowing your child to participate in a research study for the National Assessment of Educational Progress (NAEP). Before giving new tests to all students, the National Center for Education Statistics (NCES), part of the U.S. Department of Education, needs to receive feedback on new tools and item types. Your child’s responses will be used only for research purposes and will not be disclosed or used, in identifiable form, for any other purpose except as required by law [Education Sciences Reform Act of 2002, 20 U.S.C §9573].

Your child has been scheduled for a research interview at <TIME> on <DAY, DATE>. The interview will last approximately 75 minutes. Students will be asked to interact with a variety of tools and item types on a tablet computer, then provide feedback about their experiences with completed tasks. Your child will receive a $25 gift card (from a major credit card company) as a thank you for participating. In addition, if you bring your child to and from the research site, you will receive a $25 gift card.

The session will take place at:

We have attached a Consent Form to this reminder. Please sign and date it, and return the form granting consent for your child to participate in the NAEP research when your child comes in for the session. Without your consent, your child will not be able to participate in the session.

Thank you for allowing your child to volunteer in this important research for NAEP. Our researcher is looking forward to meeting with your child. If you have any questions, please contact me.

Thank you,

<NAME>

EurekaFacts, LLC

ATTACHMENT: Consent Form <Appendix O>

Appendix J: EurekaFacts Confirmation of Interview (Email/Letter) to Student Participant (Age 18 or Above)

<DATE>

IF SENT AS EMAIL - Subject: Confirmation of Your Participation in a NAEP Research Study

Dear <NAME>,

Thank you for agreeing to participate in this voluntary research study to provide feedback on questions for the National Assessment of Educational Progress (NAEP). Before giving new tests to all students, the National Center for Education Statistics (NCES), part of the U.S. Department of Education, needs to receive feedback on new tools and item types. All the information you provide will be used only for research purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (Education Sciences Reform Act of 2002, 20 U.S.C §9573).

You have been scheduled for a session at <TIME> on <DAY, DATE>. Your will receive a $25 gift card (from a major credit card company) as a thank you for participating. In addition, your parent or legal guardian will receive a $25 gift card for the time involved and to help offset the travel/transportation costs of taking you to and from the research site.

The session will take place at:

We have attached a Consent Form to this confirmation. Please sign, date, and return the form when you come to the interview, agreeing to participate in the voluntary NAEP research.

We look forward to having you participate in this very important research study. If you have any questions, please contact me.

Thank you,

<NAME>

EurekaFacts, LLC

ATTACHMENT: Consent Form <Appendix P>

Appendix K: EurekaFacts Confirmation (Email/Letter) to School/Organization Where Sessions Will Be Conducted

<DATE>

IF SENT AS EMAIL - Subject: Confirmation of NAEP Research Study

Dear <NAME>,

Thank you for assisting EurekaFacts in carrying out research studies for the National Assessment of Educational Progress (NAEP). Students’ responses will be used only for research purposes and will not be disclosed or used, in identifiable form, for any other purpose except as required by law (Education Sciences Reform Act of 2002, 20 U.S.C §9573).

Sessions have been scheduled at <SCHOOL/ORGANIZATION> from <TIME to TIME> on <DAY, DATE>. The sessions will each last approximately 75 minutes. To thank them for their time, participating students will receive a $25 gift card (from a major credit card company).

[Include the following information if applicable: If a parent/ guardian transports the student to and from the research site, the parent/guardian will also receive a $25 gift card.]

Thank you for assisting us in this important research for NAEP. If you have any questions, please contact me.

Thank you,

<NAME>

EurekaFacts, LLC

Appendix L: EurekaFacts Thank You Email/Letter to Parents or Legal Guardians of Student Participants (Below Age 18)

<DATE>

IF SENT AS EMAIL - Subject: Thank you for your child’s participation in the NAEP Research Study

Dear <NAME>,

On behalf of the National Center for Education Statistics (NCES), part of the U.S. Department of Education, EurekaFacts would like to thank you for allowing your child[ren] to participate in our research study. Thanks to your family’s participation, we are better able to develop the best possible test for the National Assessment of Educational Progress (NAEP).

Your child[ren]’s assistance helped us better understand how students approach the test questions and tools. The students’ open, candid, and insightful comments were very helpful.

Again, thank you for your assistance.

Sincerely,

<NAME>

<TITLE>

EurekaFacts, LLC

Appendix M: EurekaFacts Thank You Email/Letter to Student Participants Age 18 or Above

<DATE>

IF SENT AS EMAIL - Subject: Thank you for your participation in the NAEP Research Study

Dear <NAME>,

On behalf of the National Center for Education Statistics (NCES), part of the U.S. Department of Education, EurekaFacts would like to thank you for your participation in our research study. Thanks to your participation, we are better able to develop the best possible test for the National Assessment of Educational Progress (NAEP).

Your assistance helped us better understand how students approach both the test questions and tools. Your open, candid, and insightful comments were very helpful.

Again, thank you for your assistance.

Sincerely,

<NAME>

<TITLE>

EurekaFacts, LLC

Appendix N: EurekaFacts Thank You Email/Letter to School/Organization/Association of Student Participants

<DATE>

IF SENT AS EMAIL - Subject: Thank you for your participation in the NAEP Research Study

Dear <NAME>,

On behalf of the National Center for Education Statistics, part of the U.S. Department of Education, thank you for your assistance with recruitment of students for the National Assessment of Educational Progress (NAEP) research study. Your students’ assistance helped us better understand how students approach the test questions and tools. Your students’ open, candid, and insightful comments were very helpful. The assistance provided by your students, combined with input from other students, were useful and informative. We greatly appreciate you having taken the time to support our important research effort.

Again, thank you for your assistance.

Sincerely,

EurekaFacts, LLC

Appendix O: EurekaFacts Consent Form for Parents of Student Participants (Below Age 18)

PARENTAL CONSENT DOCUMENT

FOR PARTICIPATION IN RESEARCH

EurekaFacts, LLC

Title of Research Study: Item Types and Tools Usability Study

Thank you for your interest in participating in the research project described below. This consent form explains the research study. Please read it carefully, and feel free to ask questions about anything you do not understand. If you do not have questions now, you may ask them if they occur to you later.

What is this study about?

The National Center for Education Statistics (NCES), part of the U.S. Department of Education, is conducting a study with students currently in grades 4, 8, and 12 on new tools, tutorials, and item types for the National Assessment of Educational Progress (NAEP). NCES needs to know whether these new tools, tutorials, and item types are intuitive and easy to use. Students’ input will help NCES improve the test so that students such as your child will understand it.

Where will the research interviews take place?

The interviews will take place at <NAME OF INTERVIEW LOCATION>, located at <LOCATION>, near <NEARBY LANDMARK OR METRO STATION>.

What will happen during the research interview?

Students will be asked to complete several tasks regarding new tools and item types. The research sessions will explore how students work through the tasks on a tablet computer to ensure that the tasks are clear and understandable to a broad range of students. NCES simply wants to know how students would do the tasks if they were on an actual test, and would like to hear students’ thoughts to improve the tasks and make them easier to understand.

How long will the research interview last and when will it take place?

The interview will last up to 75 minutes and will require only one visit.

Will you keep information confidential?

Your child will be assigned a unique student identifier (ID), and at no time will his/her name be linked to any of his/her answers. Your child’s responses will be used only for research purposes and will not be disclosed or used, in identifiable form, for any other purpose except as required by law (Education Sciences Reform Act of 2002, 20 U.S.C §9573). The screens of the tablets will be recorded, along with the audio from the session. No cameras will be used, and no images or video of your child or their surroundings will be captured.

Will the results go to my child’s school?

Your child’s answers will not be disclosed to his/her school, and will not be linked to any personally identifiable information such as the school’s name.

What are the possible risks of being in the study?

There are no known risks to participating in this study.

Who can I contact with questions or for further information?

<CONTACT NAME> is carrying out this study at EurekaFacts on behalf of NCES. If you have any questions, please call him/her at <CONTACT PHONE> or by email <CONTACT EMAIL>.

Can participation end early?

Your participation in this study is voluntary. Your child has the right to refuse to answer particular questions. You may elect to withdraw from this study at any time.

By signing below, you agree that your child, _______________________________, may participate in this study. Your child will receive a $25 gift card (from a major credit card company) at the conclusion of the session. In addition, if you bring your child to and from the research site, you will also receive a $25 gift card.

Your signature below will indicate that your questions have been answered satisfactorily, and that you have read and understood the information provided above.

Print Name: _________________________________________________________________________

Signature: ___________________________________________________________________________

Date:_______________________________________________________________________________

School that student attends: ____________________________________________________________

Appendix P: EurekaFacts Consent Form for Adult Student Participants (Age 18 and Above)

PARTICIPATION CONSENT DOCUMENT

FOR PARTICIPATION IN RESEARCH

EurekaFacts, LLC

Title of Research Study: Item Types and Tools Usability Study

What is this study about?

Where will the research interviews take place?

The interviews will take place at <NAME OF INTERVIEW LOCATION>, located at <LOCATION>, near <NEARBY LANDMARK OR METRO STATION>.

What will happen during the research interview?

How long will the research interview last and when will it take place?

The interview will last up to 75 minutes and will require only one visit.

Will you keep information confidential?

You will be assigned a unique student identifier (ID), and at no time will your name be linked to any of your answers. Your responses will be used only for research purposes and will not be disclosed or used, in identifiable form, for any other purpose except as required by law (Education Sciences Reform Act of 2002, 20 U.S.C §9573). The screens of the tablets will be recorded, along with the audio from the session. No cameras will be used, and no images or video of you or your surroundings will be captured.

Will the results go to my school?

Your answers will not be disclosed to your school, and will not be linked to any personally identifiable information such as your school’s name.

What are the possible risks of being in the study?

There are no known risks to participating in this study.

Who can I contact with questions or for further information?

<CONTACT NAME> is carrying out this study at EurekaFacts on behalf of NCES. If you have any questions, please call him/her at <CONTACT PHONE> or by email <CONTACT EMAIL>.

Can participation end early?

Your participation in this study is voluntary. You have the right to refuse to answer particular questions. You may elect to withdraw from this study at any time.

By signing below, you agree that you will participate in this study. You will receive a $25 gift card (from a major credit card company) at the conclusion of the session. In addition, if your parent/guardian brings you to and from the research site, he or she will also receive a $25 gift card.

Your signature below will indicate that your questions have been answered satisfactorily, and that you have read and understood the information provided above.

Print Name: _________________________________________________________________________

Signature: ___________________________________________________________________________

Date:_______________________________________________________________________________

School that you attend: ________________________________________________________________

Appendix Q: EurekaFacts Fact Sheet related to the Usability Study

Item Types and Tools Usability Study

Fact Sheet

1. What is this study about?

2. Where will the research interviews take place?

The interviews will take place at <NAME OF LOCATION>, located at <LOCATION>, near <NEARBY LANDMARK OR METRO STATION>.

3. What will happen during the interviews?

Students will be asked to complete several tasks regarding new tools and item types. The research sessions will explore how students work through the tasks on a tablet computer to ensure that the tasks are clear and understandable to a broad range of students. NCES simply wants to know how students would do the tasks if they were on an actual test, and would like to hear students’ thoughts on how to improve the tasks and make them easier to understand.

4. How long will the Interview last and when it will take place?

The interview session will last about 75 minutes and can be scheduled at your convenience. Each interview requires only one visit.

5. Whose consent do you require for participation?

For students below age 18: Once we have received informed consent from the parent/guardian for their eligible student to participate, we will then also ask the student for his or her consent to participate. Participation in the study is completely voluntary. The parent/guardian or student may withdraw consent for the student’s participation in the study at any time. The student also has the right to refuse to answer particular questions. Students age 18 or above can sign the consent form directly. Again, participation in the study is voluntary, and students have the right to withdraw intent to participate and/or refuse to answer particular questions.

6. Will you keep information confidential?

All of the participant’s responses will be used only for research purposes and will not be disclosed or used, in identifiable form, for any other purpose except as required by law [Education Sciences Reform Act of 2002 (ESRA 2002) 20 U.S.C., § 9573]. Each participant will be assigned a unique student identifier (ID), and at no time will his/her name be linked to any of his/her answers.

7. Will the results go to the participant’s school?

Participants’ responses will not be disclosed to his/her school, and will not be linked to any personally identifiable information such as the school’s name.

8. Are there rewards or incentives for participating?

Yes, the student will receive a $25 gift card (from a major credit card company) at the end of his or her interview. If the student’s parent or guardian brings the student to and from the interview site, he or she will also receive a $25 gift card.

9. What are the possible risks of being in the study?

There are no known risks to participating in this study.

10. Whom can I contact with questions or for further information?

If you have any questions, please call [CONTACT NAME] from EurekaFacts at [CONTACT PHONE] or [CONTACT EMAIL].

Appendix R: EurekaFacts Sample Recruitment Flyer for the Usability Study

PARTICIPANTS NEEDED!

EurekaFacts, a local research company in Rockville, MD, is conducting a study for the National Center for Educational Statistics (NCES), a part of the U.S. Department of Education.

We are looking for students in either grades 4, 8, or 12 to help us evaluate new tools and items for the National Assessment of Educational Progress (NAEP), commonly known as “The Nation’s Report Card”.

We will provide a $25 gift card to eligible students who take part in the study, and an additional $25 gift card to the parent/guardian who brings the student to and from the study location. Participation takes no more than 75 minutes.

This study will run from <DATE OR SEASON> through <DATE OR SEASON>. Students can arrange a time to participate that fits with their schedule.

Interested? Call <CONTACT PHONE> or email <CONTACT EMAIL>, or head to our website at <WEBSITE>, for more information.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Title	Appendices (A-R)
Author	mgisbert
File Modified	0000-00-00
File Created	2021-01-22

National Assessment of Educational Progress (NAEP) Digitally Based Assessments (DBA) Usability Study 2017-18

NCES Cognitive, Pilot, and Field Test Studies System

Appendices - NAEP DBA Usability Studies 2017-18