NAEP Survey Assessments Innovation Lab (SAIL) Test Assembly Experimental Study

National Center for Education Statistics

National Assessment of Educational Progress

National Assessment of Educational Progress (NAEP)

Survey Assessments Innovations Lab (SAIL)

Test Assembly Experimental Study

Appendices

Communication Materials

OMB# 1850-0803 v.272

September 2020

Table of Contents

Appendix A: Recruitment Letter to Parents or Legal Guardians of Student Participants (Under Age 18) 3

Appendix B: Recruitment Email to Parents or Legal Guardians of Student Participants (Under Age 18) 4

Appendix C: Recruitment Web-Surveyor Intake Form—Link Provided in Emails to Parents/Legal Guardians Interested in Participating 5

Appendix D: Consent Form for Parent/Legal Guardian of Student Participants (Under Age 18) 7

Appendix E: Follow-Up Email to Confirm Participation 10

Appendix F: Recruitment Email/Letter to Organizations 11

Appendix G: Recruitment Flyer 12

Appendix A: Recruitment Letter to Parents or Legal Guardians of Student Participants (Under Age 18)

Insert Date Here 

 

Dear Parent or Guardian, 

The National Center for Education Statistics (NCES) is conducting research to investigate how the design of test impacts students’ test taking motivation, and have contracted with researchers at Educational Testing Service (ETS) to design and administer a project called Simple Solutions for Increasing Student Motivation – Test Assembly. We are inviting 8th grade students to participate in the study between October 15, 2020 to January 31, 2021. For this project, students will complete the activities online at home after school or on the weekend. Students will be compensated for their participation with a $20 virtual gift card approximately 2 weeks after the completion of the study. If your child decides not to participate in the study or withdraws without completing the study, they will not receive the $20 virtual gift card.  

 
Your child will spend a maximum of 50 minutes independently completing a two-part math assessment and a survey about their overall experience. The math assessment will include a variety of item types (e.g., multiple-choice, grid, graphing) and will include topics that are typically learned in 8^th grade. Students will have a maximum of 20-minutes to complete each math assessment. Participation is voluntary, and a student may withdraw from the study at any time without penalty. We will also ask you to complete background information about your child (e.g., demographics, English language proficiency). 

 

Names or other identifying information will not be used and will remain private. We follow strict rules to make sure that others do not know names of students who have participated; all information obtained from the students will be safely protected¹. Data will be securely stored on encrypted ETS servers that only members of the research team will have access to.  

 

The study will be held between October 15, 2020 and January 31, 2021.  

 

If you are interested in participating, please complete the attached parent consent form and student profile form, and return to us as soon as possible. Please feel free to email me or the project manager, Christine Luce, at [email protected] with any questions.  

 

We hope that your child will choose to participate in this study. Thank you for your time and support of our research.  

 

Sincerely, 

 

Blair Lehman 
Research Scientist 

Educational Testing Service

660 Rosedale Road  l  Princeton, NJ 08541  l  Email: [email protected]  l  Phone: 847-942-3789 

Appendix B: Recruitment Email to Parents or Legal Guardians of Student Participants (Under Age 18)

[Subject: Paid Education Research Opportunity]

Insert Date Here 

 

Dear Parent or Guardian, 

The National Center for Education Statistics (NCES) is conducting research to investigate how the design of test impacts students’ test taking motivation, and have contracted with researchers at Educational Testing Service (ETS) to design and administer a project called Simple Solutions for Increasing Student Motivation – Test Assembly. We are inviting 8th grade students to participate in the study between October 15, 2020 to January 31, 2021. For this project, students will complete the activities online at home after school or on the weekend. Students will be compensated for their participation with a $20 virtual gift card approximately 2 weeks after the completion of the study.  If your child decides not to participate in the study or withdraws without completing the study, they will not receive the $20 virtual gift card.  

 
Your child will spend a maximum of 50 minutes independently completing a two-part math assessment and a survey about their overall experience. The math assessment will include a variety of item types (e.g., multiple-choice, grid, graphing) and will include topics that are typically learned in 8^th grade. Students will have a maximum of 20-minutes to complete each math assessment. Participation is voluntary, and a student may withdraw from the study at any time without penalty. We will also ask you to complete background information about your child (e.g., demographics, English language proficiency). 

 

Names or other identifying information will not be used and will remain private. We follow strict rules to make sure that others do not know names of students who have participated; all information obtained from the students will be safely protected¹. Data will be securely stored on encrypted ETS servers that only members of the research team will have access to.  

 

The study will be held between October 15, 2020 and January 31, 2021. 

 

If you are interested in participating, please complete the online form and [link to web-surveyor intake form] consent as soon as possible. Please feel free to email me or the project manager, Christine Luce, at [email protected] with any questions.  

 

We hope that your child will choose to participate in this study. Thank you for your time and support of our research.  

 

Sincerely, 

 

Blair Lehman 
Research Scientist 

Educational Testing Service

660 Rosedale Road  l  Princeton, NJ 08541  l  Email: [email protected]  l  Phone: 847-942-3789 

Appendix C: Recruitment Web-Surveyor Intake Form—Link Provided in Emails to Parents/Legal Guardians Interested in Participating

Information collected by ETS researchers will not be reported to anyone, including your child’s school or teacher.  

 

Student Name (Last, First): _________________________________________________________________ 

Parent Name (Last, First):__________________________________________________________________ 

Parent contact information:  

Phone: _____________________________________________ 

Email: _______________________________________________ 

Name of School your child attends: ________________________________________________________ 

Location of School (City, State): ____________________________________________________________ 

Type of School (e.g. public, private, charter, vocational):___________________________________ 

Grade your child will be entering in September 2020: ____________ 

Please type an X to select a response for each question below.

What is your child’s gender? 

Male: __________ Female:__________ Other: ___________ Prefer not to answer:_________ 

What is your child’s ethnicity?

Hispanic or Latino: ____

Not Hispanic or Latino: ____

What is your child’s race? Please select all that apply.

American Indian or Alaskan Native: _____ 

Asian or Asian American: _____ 

Black or African American: _____ 

Native Hawaiian or other Pacific Islander: _____ 

White: _____ 

Prefer not to answer: _____ 

What is your child’s ELL classification? 

Current ELL: _____ 

Former ELL (reclassified as proficient): _____ 

Initially proficient at school entry (native English speaker or fluent in English upon school entry: _____  

Does your child have Title I Accommodations (yes, no)? _____ 

Does your child participate in free or reduced lunch program (yes, no)? _____ 

What is the name of your child’s current math course? _____ 

What is your child’s current math grade? _____

 

Does the device your child will use to complete this study have an internet connection? _____ 

 

What type of device will your child use to complete this study? 

Laptop: _____ 

Desktop: _____ 

Tablet: _____ 

Appendix D: Consent Form for Parent/Legal Guardian of Student Participants (Under Age 18)

 

PARENT CONSENT FORM 

This consent form contains important information to help you and your child decide whether to participate in a research study. Please read this consent form carefully.  

 

Researchers are available to explain this study to you in more detail. Please feel free to ask questions about anything that is not clear at any time.  

 

• Being in a research study is voluntary – it is your and your child’s choice. 

• If your child chooses to join, your child can still drop out/stop at any time. 

 

Have your child join this research study only if you and your child understand: 

 

• Why this research study is being conducted. 

• What activities will be conducted during the research study. 

• Any possible benefits to your child. 

• The possible risks to your child. 

• Other options your child could choose instead of being in this research study. 

• Whether your child will be compensated for participating, and, if so, how much. 

• What to do if you/your child have problems with or questions about this research study. 

• Your child’s legal rights as a research study volunteer. 

 

“You” refers to the parent/guardian of the child being asked to participate. 

 

Your child is being asked to take part in a research study conducted by Dr. Blair Lehman.  

 

1. WHY IS THIS RESEARCH STUDY BEING DONE? 

The purpose of this research is to investigate how the design of tests impact students’ test taking motivation.  

 

2. WHAT IS MY CHILD BEING ASKED TO DO?  

As part of a research study conducted by Educational Testing Service (ETS), your child will independently complete a two-part math assessment and a survey about their overall experience. The math assessment will include a variety of item types (e.g. multiple-choice, grid, graphing) and will include topics that are typically learned in 8^th grade. Students will have a maximum of 20-minutes to complete each math assessment block. Your child’s time commitment for participating in this research study is a maximum of 50 minutes online after school or on the weekend. 

 

All information obtained from the students will be private¹. Data will be securely stored on encrypted ETS servers that only members of the research team will have access to. We follow strict rules to make sure that others do not know names of students who have participated. 

 

Your child’s participation in this research is voluntary and you have the opportunity to withdraw your child from the project at any time without penalty. We hope you will allow your child to participate in this research project by completing the consent form below.  

 

2. ARE THERE BENEFITS TO BEING IN THIS RESEARCH STUDY? 

We do not anticipate that you and your child will benefit directly from this research study. Although you may not receive any benefit, in the future others may benefit because of what the researchers learn from this study about how test design impacts student motivation.  

 

4. WHAT ARE THE RISKS? 

There are minimal risks or discomforts associated with the completion of this research study. These discomforts and risks might include a potential breach of privacy, however, the researchers will guard against any disclosure of personally identifying information.  

 

5. HOW WILL MY CHILD’S DATA BE USED? 

Any data collected will be used only for research and training purposes. Your child will not be named in any results. Data collected as a part of this study will be kept private¹, and will be stored on secure, password-protected computers and servers.  

 

No personally identifiable information will be used in any research report or presentation unless you provide your express consent.  Research findings may be shared at conferences and in publications, however, no one will be individually identified.  

 

Finally, only anonymized survey data and activity logs will be uploaded to an ETS data repository and will be used indefinitely for future, unspecified secondary purposes.  

 

6. WHAT OTHER OPTIONS ARE THERE? 

You may choose not to allow your child to participate in this research study.  

 

7. WILL MY CHILD RECEIVE A PAYMENT? 

Your child will receive a $20 virtual gift card as compensation for participating and completing this research study. If your child decides not to participate in the study or withdraws without completing the study, he or she will not receive the $20 virtual gift card.

 

8. WHO CAN I CALL IF I HAVE QUESTIONS? 

If you or your child have any questions or concerns about this research study, or if your child has any problems that occur from taking part in this research study, you or your child may call the researcher Dr. Blair Lehman at 847-942-3789 or at [email protected]. 

 

If you have any questions about your child’s rights as a research participant or if you believe your child has suffered an injury as a result of taking part in the research study, you or your child may contact the ETS institutional review board administrator at [email protected].

8. WHAT ARE MY CHILD’S RIGHTS AND WHAT ELSE SHOULD I KNOW AS THE PARENT OF A RESEARCH STUDY VOLUNTEER? 

Your child’s participation in this research study is completely voluntary, and your child has the right to withdraw at any time. Your child may choose not to take part in this research study at all, and your child also has the right to refuse to answer any questions that make them uncomfortable. You and your child also have the right to know specifically how your child’s data will be used.  

 

10. STATEMENT OF CONSENT 

You have read this consent form and have been able to ask questions and state any concerns. The researcher has responded to your questions and concerns, if any. You believe you understand the research and the potential benefits and risks that are involved. You also understand that the research content [survey, task, etc.] to which your child will have access are considered the secure and proprietary property of ETS. 

 

In signing this form, you acknowledge you confirmed your child’s willingness to participate.   

 

Please check one: 

__ Yes, I allow my child to participate in this research study. 

__ No, I do not allow my child to participate in this research study. 

 

 

_________________________________________ _________ 

Parent/Guardian Signature (type name here) Date 

 

_________________________________________ 

Printed Parent/Guardian Name   

 

_________________________________________ 

Child’s Name   

 

Please check here if you would like to be contacted regarding other ETS study opportunities.  I consent for an ETS researcher to contact me regarding other study opportunities at ETS.  

(Do not check this box if you do not wish to be contacted.) 

Appendix E: Follow-Up Email to Confirm Participation

[Subject: Paid Education Research Opportunity]

Insert Date Here

Dear <Parent or Legal Guardian’s Name>,

 

Thank you for agreeing for <Child’s Name> to participate in the Test Assembly Study.

 

<Child’s name> can access the study materials at <Study link> and has been scheduled to complete the study between <Start date> and <End date>. The study will take a maximum of 50 minutes to complete. Please confirm that <Child’s name> will be able to complete the study during this time period.

 

If your child is no longer available during this time period, we will work with you to find an alternative time period.

If you have any questions or if <Child’s name> encounters any issues during completion of the study, please feel free to contact me and I will help to address any issues.

  

Sincerely,

Blair Lehman
Research Scientist

Educational Testing Service

660 Rosedale Road l Princeton, NJ 08541 l Email: [email protected] l Phone: 847-942-3789

Appendix F: Recruitment Email/Letter to Organizations

[Subject: Paid Education Research Opportunity for Students in 8^th grade]

Insert Date Here 

 

Dear Educational Colleague,

The National Center for Education Statistics (NCES) is conducting research to investigate how the design of test impacts students’ test taking motivation, and have contracted with researchers at Educational Testing Service (ETS) to design and administer a project called Simple Solutions for Increasing Student Motivation – Test Assembly. We are inviting 8th grade students to participate in the study between October 15, 2020 to January 31, 2021. For this project, students will complete the activities online at home after school or on the weekend. Students will be compensated for their participation with a $20 virtual gift card approximately 2 weeks after the completion of the study. If your child decides not to participate in the study or withdraws without completing the study, they will not receive the $20 virtual gift card.

Students will spend a maximum of 50 minutes completing a two-part math assessment and a survey about their overall experience. The math assessment will include a variety of item types (e.g., multiple-choice, grid, graphing) and will include topics that are typically learned in 8^th grade. Each student will have a maximum of 20-minutes to complete each math assessment block. Each student will complete the two-part math assessment and survey independently. Participation is voluntary, and a student may withdraw from the study at any time without penalty. We will also ask parents and legal guardians to complete background information about their child (e.g., demographics, English language proficiency).

Names or other identifying information will not be used and will remain private. We follow strict rules to make sure that others do not know names of students who have participated; all information obtained from the students will be safely protected⁰. Data will be securely stored on encrypted ETS servers that only members of the research team will have access to.

The study will be held between October 15, 2020 and January 31, 2021. If you are interested in distributing information about this opportunity to parents and legal guardians of children at <organization name>, please let us know and we can provide the information to be sent to parents and legal guardians to sign up for the study. Please feel free to email me or the project manager, Christine Luce, at [email protected] with any questions.

Thank you for your time and support of our research.

  

 

Sincerely, 

 

Blair Lehman 
Research Scientist 

Educational Testing Service

660 Rosedale Road  l  Princeton, NJ 08541  l  Email: [email protected]  l  Phone: 847-942-3789 

Appendix G: Recruitment Flyer

ETS Seeks 8^th Grade Students for Math Study – Students who participate in this study will receive a $20 virtual gift card for 50 minutes of participation. The study can be completed online at home and students will be asked to complete a math assessment and survey questions. The study will take place from October 15^th to January 31^st. If interested, please complete the survey [link to Parent/Legal Guardian Recruitment email text, Web-Surveyor Intake Form, and Informed Consent].

Note that there is a word limit for posting study advertisements on the ETS internal website and we plan to only have one recruitment flyer. However, parents and legal guardians will be able to access the full recruitment email text, web-surveyor intake form, and informed consent through the link provided. In addition, parents and legal guardians will be able to contact researchers with any questions that they have after reviewing the materials. Thus, while this flyer is brief, parents and legal guardians will have access to all information before deciding whether or not their child will participate.

1 All of the information provided by participants may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151). By law, every National Center for Education Statistics (NCES) employee as well as every NCES agent, such as contractors and NAEP coordinators, has taken an oath and is subject to a jail term of up to 5 years, a fine of $250,000, or both if he or she willfully discloses ANY identifiable information about participants. Electronic submission of participant’s information will be monitored for viruses, malware, and other threats by Federal employees and contractors in accordance with the Cybersecurity Enhancement Act of 2015. The collected information will be combined across respondents to produce statistical reports.

1 All of the information provided by participants may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151). By law, every National Center for Education Statistics (NCES) employee as well as every NCES agent, such as contractors and NAEP coordinators, has taken an oath and is subject to a jail term of up to 5 years, a fine of $250,000, or both if he or she willfully discloses ANY identifiable information about participants. Electronic submission of participant’s information will be monitored for viruses, malware, and other threats by Federal employees and contractors in accordance with the Cybersecurity Enhancement Act of 2015. The collected information will be combined across respondents to produce statistical reports.

1 All of the information provided by participants may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151). By law, every National Center for Education Statistics (NCES) employee as well as every NCES agent, such as contractors and NAEP coordinators, has taken an oath and is subject to a jail term of up to 5 years, a fine of $250,000, or both if he or she willfully discloses ANY identifiable information about participants. Electronic submission of participant’s information will be monitored for viruses, malware, and other threats by Federal employees and contractors in accordance with the Cybersecurity Enhancement Act of 2015. The collected information will be combined across respondents to produce statistical reports.

1 All of the information provided by participants may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151). By law, every National Center for Education Statistics (NCES) employee as well as every NCES agent, such as contractors and NAEP coordinators, has taken an oath and is subject to a jail term of up to 5 years, a fine of $250,000, or both if he or she willfully discloses ANY identifiable information about participants. Electronic submission of participant’s information will be monitored for viruses, malware, and other threats by Federal employees and contractors in accordance with the Cybersecurity Enhancement Act of 2015. The collected information will be combined across respondents to produce statistical reports.

0 All of the information provided by participants may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151). By law, every National Center for Education Statistics (NCES) employee as well as every NCES agent, such as contractors and NAEP coordinators, has taken an oath and is subject to a jail term of up to 5 years, a fine of $250,000, or both if he or she willfully discloses ANY identifiable information about participants. Electronic submission of participant’s information will be monitored for viruses, malware, and other threats by Federal employees and contractors in accordance with the Cybersecurity Enhancement Act of 2015. The collected information will be combined across respondents to produce statistical reports.