Program for International Student Assessment (PISA) 2021 Field Test Pretest

I am pleased to notify you that <Insert Name of School or Schools> in your district have been selected to participate in the Pretest for the 2021 administration of the Program for International Student Assessment (PISA). As the largest international education study in the world, PISA measures the mathematics, reading, and science literacy of students nearing the end of compulsory education. The United States and more than 80 other countries and education systems will participate in the next round of data collection. PISA results are used by researchers and policymakers to chart national progress against international standards and the educational progress of our global economic peers and competitors. The purpose of the pretest is to help ensure that the PISA assessment and field operations will be effective for the field test in the spring of 2020.

In preparation for the 2021 main study, PISA will administer a pretest in winter 2020. The purpose of the pretest is to help ensure that the PISA assessment and field operations will be effective for the field test in the spring of 2020. <School District> will play an important role by participating, and I know that we can count on your full support to make this a meaningful experience for your schools and students.

The PISA 2021 Pretest will be administered in <Insert month>. PISA representatives will work with your school(s) to conduct the assessment on a date that is convenient for them. Schools participating in the PISA pretest will receive $250, and each school’s PISA school coordinator (a school staff person designated to liaise with PISA staff) will receive $200 as a thank you for the coordinator’s time and effort. Each student who participates will be eligible to receive $25 and a volunteer service certificate of 4 hours from the U.S. Department of Education.

More information about PISA can be found at http://nces.ed.gov/surveys/pisa.. Name, our NAEP State Coordinator, will contact your staff with additional information.

I know that I can count on you to help accomplish our goal of 100 percent participation. Thank you for your time and support.

Sincerely,

Name of Chief State School Officer

CC: State Testing Director

District Testing Director

NAEP State Coordinator

The National Center for Education Statistics (NCES) is authorized to conduct the Program for International Student Assessment (PISA) by the Education Sciences Reform Act of 2002 (ESRA 2002, 20 U.S.C. §9543), and to collect students’ education records from educational agencies or institutions for the purpose of evaluating federally supported education programs under the Family Educational Rights and Privacy Act (FERPA, 34 CFR §§ 99.31(a)(3)(iii) and 99.35). The data are being collected for NCES by Westat, a U.S.-based research organization. All of the information you provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151).

Shape2

Notification Letter from NAEP State Coordinator to Principal

[Date]

[Title] [Name First] [Name Last], [Title/Department]

[School District]

[Address 1]

[City], [State] [Zip code]

Dear [Title] [Name Last]:

I am pleased to notify you that your school has been selected to participate in the Pretest for the 2021 administration of the Program for International Student Assessment (PISA). As the largest international education study in the world, PISA measures the reading, mathematics, and science literacy of students nearing the end of compulsory education. The United States and more than 80 other countries and education systems will participate in the next round. PISA results are used by researchers and policymakers to chart national progress against international standards and the educational progress of our global economic peers and competitors. The purpose of the pretest is to help ensure that the PISA assessment and field operations will be effective for the field test in the spring of 2020.

PISA representatives will provide significant support to your school, bring all materials and equipment, including tablets, and administer the assessment. The PISA pretest assessment will take around three hours to complete. Up to 65 students will be selecte to participate in the prestest and the PISA representative will work with the school coordinator at your schools to select the students.

In <insert month>, I will send you an assessment date and detailed Pretest information. If the date presents a conflict for your school, we will work together to identify an alternate. I will also ask you to identify a school coordinator.

More information about PISA can be found at http://nces.ed.gov/surveys/pisa.. If you have questions, please contact me at telephone number or via email at email address.

Our chief state school officer, name, and your district superintendent, name, support PISA and look forward to your school’s participation. We know that we can count on you to help reach our goal of 100 percent participation.

Sincerely,

NAEP State Coordinator

CC: District Test Director

Shape3

PISA 2021 Field Test Brochure –Front Cover

What is PISA?

The Program for International Student Assessment (PISA) is an international assessment of students nearing the end of compulsory education that measures how well students apply their knowledge and skills in mathematics, reading, science, and financial literacy to problems set in real-life contexts. Conducted every 3 years since 2000, the next round of PISA in 2021 will include students from more than 80 countries and education systems around the world, including the United States. PISA is coordinated by the Organization for Economic Cooperation and Development (OECD) and conducted in the United States by the National Center for Education Statistics (NCES) within the U.S. Department of Education.

Why is PISA important and what are the benefits of participating?

PISA provides a unique opportunity for the United States to understand how well students nearing the end of compulsory education are prepared to enter the workforce or post-secondary education in comparison to their peers from more than 80 countries. Through participating in PISA, schools, and students contribute to ongoing efforts to improve education in our country. PISA helps identify U.S. students’ strengths and weaknesses in the domains of mathematics, reading, science, and financial literacy and offers insights into successful policies and practices in other countries.

Each school and student that participates in PISA plays a vital role in representing other schools and students that are similar. It is essential that sampled schools contribute to this important measurement of educational progress to ensure the U.S. sample reflects the knowledge and skills of all students across the country.

As a token of appreciation for participating, schools will receive $250, the school coordinator will receive $200, students who participate will each receive $25 as well as 4 hours of volunteer service time.

What is the purpose of the PISA Field Test in spring 2020?

PISA conducts a field test (a small-scale, trial run of the assessment) in every participating country to determine if any of the test questions are biased because of national, social, or cultural differences and whether field operations worked as planned. Analyses of student responses are conducted to check for evidence of unanticipated differences in student performance across countries that could indicate a linguistic or conceptual translation problem. Test items found to be problematic are revised to mitigate identified issues or dropped from the assessment altogether. In spring 2020, the United States will participate in the PISA 2021 field test. Assessments will be conducted using computers. The field test data will provide valuable information to ensure that the PISA 2021 assessment is unbiased and fair to U.S. students.

What will schools and students be asked to do in the PISA field test?

The PISA 2021 field test has three primary components: (1) a computer-based student assessment, (2) a computer-based student questionnaire, and (3) an online school questionnaire.

Up to 60 students in each school will be selected to participate in the PISA assessment, which takes about 3.5 hours, including instructions, breaks, and a questionnaire. Students will answer questions in various combinations of mathematics, reading, science, and financial literacy.

The principal of each school will be asked to appoint a staff member to act as the PISA school coordinator. The school coordinator will work with Westat staff to coordinate the assessment and submit a student list for sampling. The principal of each school will also be asked to complete a 45-minute online questionnaire about school and student body characteristics and policies.

Shape4

PISA 2021 Field Test Brochure – (Continued Inside Cover)

How is PISA developed?

PISA is developed through an international collaborative process involving input from U.S. and international experts in science, reading, mathematics, financial literacy, and educational measurement. The PISA assessment materials are thoroughly reviewed by within-country experts to make sure the materials are appropriate for each country’s students. Finally, the final assessment materials are endorsed by all participating countries.

What do PISA questions look like?

PISA includes a mix of item types: some items require students to select from among possible responses, while others require students solve problems and provide written answers. Examples of PISA assessment questions are available at http://nces.ed.gov/surveys/pisa/educators.asp and https://www.oecd.org/pisa/pisaproducts/pisa-test-questions.htm.

Which countries are participating in PISA 2021?

Where do I find more information?

Visit the PISA website at http://nces.ed.gov/surveys/pisa.

For additional questions about PISA 2021, contact the PISA U.S. home office at 1-888-638-2597 or email [email protected].

Shape5

PISA 2021 Field Test FAQ for school administrators –Front Page

Program for International Student Assessment

FREQUENTLY ASKED QUESTIONS

PISA 2021 Field Test (Spring 2020)

Information for Schools

PISA is an international assessment of students nearing the end of compulsory education that measures how well students apply their knowledge and skills to solve problems related to mathematics, reading, and science, and financial literacy in real-life contexts. In the PISA 2021 field test, students will be assessed in mathematics, reading, science, financial literacy, or some combination of these subjects. The PISA 2021 field test will be administered in the spring of 2020.

What does participating in PISA entail?

Each school is asked to designate a school coordinator to work with Westat and to submit a list of all students born in 2004. Up to 60 students in each school will be sampled to participate in the PISA assessment, which takes about 3.5 hours of total time, including instructions, breaks, and a questionnaire. The principal of each school will also be asked to complete a 45-minute online questionnaire about school and student body characteristics and policies.

What is the field test for?

Field tests are a critical part of the development of test questions for international assessments like PISA. Field tests (small-scale, trial runs of an assessment) allow assessment developers to try out new questions to determine their performance when administered to students around the world. The results of the U.S. field test will help assessment developers to eliminate wording or topics that would put U.S. students at a disadvantage relative to students in other countries. The field test also ensures that data collection procedures are done efficiently and correctly.

Why should my school and students participate?

Your school was randomly selected so that the overall U.S. field test sample would be representative of the overall U.S. school population. Field-testing on a diverse sample of students from a variety of schools, locations, and backgrounds ensures that the assessment wording and concepts are not regionally, culturally, or socially biased. The participation of selected schools in the United States is vital to ensuring an accurate representation of the overall population of students nearing the end of compulsory education across the country—including U.S. public and private schools. Although the assessment is voluntary, we rely on school and student participation to ensure the results are complete and accurate. We cannot do that without the support of schools like yours.

What are the benefits to participating?

PISA informs national discussions about education as well as international competitiveness. By participating, students ensure fairness in the questions that will be used in the main study in 2021. Field-testing the assessment questions will identify wording and concepts that could disadvantage U.S. students in the main study. Students’ efforts in the field test ensure that the achievement of our nation’s 15-year-olds is accurately and fairly reported.

As a token of appreciation for participating, schools will receive $250, the school coordinator will receive $200, and students who participate will each receive $25 as well as 4 hours of volunteer service time.

Is it possible that my school will be selected for the main study as well as the field test?

It is extremely unlikely that a school selected for the field test will be selected for the main study.

Will all of our students be asked to participate?

Probably not. In each school, all students born in 2004 and enrolled in grade 7 or higher will have an equal chance of selection. Up to 60 students in each school will be selected to participate. Only in very small schools will the school sample likely include all eligible students. In addition, some students with disabilities or limited English proficiency may be offered a limited set of accommodations to enable their participation; otherwise, they may be excused from the assessment.

Shape6

PISA 2021 Field Test FAQ for school administrators – Back Page

Who conducts the PISA assessment?

The National Center for Education Statistics (NCES), within the U.S. Department of Education, conducts this study as authorized by the Education Sciences Reform Act of 2002 (ESRA 2002, 20 U.S.C. §9543) and with the approval of the U.S. Office of Management and Budget under OMB# 1850-0755. The assessment process will be undertaken by trained staff from Westat, a research organization under contract to NCES. In compliance with standard protocols, Westat field staff undergoes FBI background checks.

How is the school questionnaire administered?

The school questionnaire is administered online from a secure website. The school questionnaire, which looks at school characteristics such as enrollment and school practices, takes about 45 minutes to complete.

Do teachers or other school staff need to help administer the assessment?

No. Westat field staff will visit the school on the day of the assessment, bringing with them all the materials required, and will handle the entire administration of the assessment. Field staff will also bring all of the necessary computer equipment. There is no requirement for school resources such as school computers or access to the internet. All that is required is adequate space to set up the equipment and assess the students.

When will the PISA field test be conducted?

The PISA field test will be conducted between March 16 and April 24, 2020. Westat will work with schools to identify an assessment date convenient for the school in that time period.

How long does PISA take?

Students will take a computer-based assessment. This approximately 3-hour session includes time for instructions, the assessment, breaks, and a questionnaire that students complete about themselves. The assessment location will be used for about 5 hours total, including setup and breakdown by Westat staff. All assessment activities will take place in one day.

What will happen with the collected data?

The data collected for PISA will be used to report on students’ knowledge and skills as group descriptions at the national level. All of the information provided by school staff and students may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151). The U.S. Office of Management and Budget has approved this data collection under OMB# 1850-0755. Reports of the findings from PISA will not identify participating districts, schools, students, or individual staff. Individual responses will be combined with those from other participants to produce summary statistics and reports.

Where can I find more information?

Visit the PISA website at http://nces.ed.gov/surveys/pisa. For additional questions about PISA 2021, contact the PISA U.S. home office at 1-888-638-2597 or email [email protected].

Shape7

PISA 2021 Field Test FAQ for Students

Program for International Student Assessment

FREQUENTLY ASKED QUESTIONS

PISA 2021 Field Test (Spring 2020)

Information for Students

What is PISA?

The Program for International Student Assessment (PISA) is an international assessment of students nearing the end of compulsory education that measures how well students apply their knowledge and skills in solving problems in reading, mathematics, science, and financial literacy. Conducted every three years, PISA presents problems that students are likely to encounter in the real world. This spring, students like you from more than 80 countries and education systems, including the United States, will participate in the PISA field test.

Why should I participate in PISA?

You are one of up to 60 students in your school who have been selected to participate in PISA. Participation in PISA is voluntary, but each student who participates plays a vital role in ensuring the U.S. results are representative of all types of schools and education environments across the country—including public and private schools. Plus, if you participate in PISA you will…

• Receive a certificate for 4-hours of volunteer service from the U.S. Department of Education;

• Represent students like you across the country; and

• Make an important contribution to this global measurement of educational progress by representing other students

just like you across the United States.

Only 3,000 students will have the unique opportunity to participate in PISA across the United States. How often do you get the chance to represent your country? Participating in PISA is a national service. You can have an impact on the bigger picture of education in the United States and around the world.

What subjects are assessed in PISA?

Each student participating in PISA will be assessed in various combinations of the following subjects: reading, mathematics, science, and financial literacy.

How long does PISA take?

The PISA assessment takes approximately 3.5 hours to complete. This includes time for instructions, the assessment, breaks, and a survey about you.

How did my school get selected to take PISA?

The schools that participate in PISA in the United States are randomly selected from a list of all schools in the country that enroll age-eligible students. This is done to ensure that U.S. participants accurately represent the entire population of students nearing the end of compulsory education and not just particular types of schools or groups of students.

How did I get selected to take PISA?

The only criteria for being eligible to be selected for PISA are being born in 2004 and being enrolled in grade 7 or higher. From a list of all eligible students provided by your school, up to 60 students were randomly selected to participate. Every eligible student enrolled in a PISA selected school has an equal chance of being selected. Students in other countries are selected in the same way to make sure each country is fairly represented and no country is advantaged or disadvantaged because of the types of schools or groups of students selected.

What types of questions will I see on PISA?

PISA questions assess the knowledge and skills students have learned, both in and out of school, based on situations you may encounter in real life. Some PISA questions require that you select from a set of provided answers; other PISA questions require that you write out your response. PISA is administered on computer and many of the items are interactive, where you manipulate different pieces of information. To try PISA items on your own, please visit http://www.oecd.org/pisa/test/.

What is done with the information you collect from me?

PISA is not designed to produce individual test scores and your individual performance is not shared with your teachers, school, or district in any way. Student responses are combined with other student responses and are only used for statistical purposes. The data collected for PISA will be used to report on students’ knowledge and skills as group descriptions at the national level. All of the information you provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151). The U.S. Office of Management and Budget has approved this data collection under OMB# 1850-0755.

Where can I find more information?

Visit the PISA website at http://nces.ed.gov/surveys/pisa. For additional questions about PISA, contact the PISA U.S. home office at 1-888-638-2597 or email [email protected].

Sample Notification Letter – PISA 2021 Field Test

SCHOOL LETTERHEAD

Dear Parent or Guardian,

This letter is to inform you about an important international study of student learning being conducted in our school this spring. This study is called the Program for International Student Assessment, or PISA. PISA provides important information for internationally benchmarking performance in reading, mathematics, and science of students in the United States nearing the end of compulsory education against their peers in countries around the world.

Each time the study is conducted, new test questions need to be developed and tested to ensure that those questions accurately measure the knowledge and skills of students. In spring 2020, new questions will be tested in approximately 50 schools in the United States tro ensure that PISA is valid and fair to all U.S. students. In preparation for PISA 2021 field test, PISA will administer a pretest in winter 2020. The purpose of the pretest is to help ensure that the PISA assessment and field operations will be effective for the field test in 2020.

Our school has accepted an invitation from the National Center for Education Statistics (NCES) within the U.S. Department of Education to participate in the PISA pretest. A select few of our students born in 2004, along with your teenager, will take part in this study. The enclosed summary sheet provides some background on PISA, explains what is involved for each student selected to participate in the study, and gives a contact phone number and email address where you can find answers to any questions you might have.

To have an accurate picture of what U.S. students can do, it is critically important that each student selected takes part in the study. The PISA assessment is taken on a computer. Also, students will be asked to complete a questionnaire about themselves. I urge you to support this effort by encouraging your teenager to take part; however, participation in this study is entirely voluntary. Previous experience suggests that students enjoy taking part, and all participating students will receive $25 and a certificate from the U.S. Department of Education for 4 hours of volunteer service.

The information provided by students will not be shared with the school. NCES is authorized to conduct PISA by the Education Sciences Reform Act of 2002 (ESRA 2002, 20 U.S.C. §9543). All of the information provided by your teenager may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151). Students and schools are never identified in any reports. All reported statistics refer to the United States as a whole. The U.S. Office of Management and Budget has approved this data collection under OMB# 1850-0755.

Thank you for taking the time to learn about this important study.

Sincerely,

Enclosures: Facts for Parents about PISA

The National Center for Education Statistics (NCES) is authorized to conduct the Program for International Student Assessment (PISA) by the Education Sciences Reform Act of 2002 (ESRA 2002, 20 U.S.C. §9543), and to collect students’ education records from educational agencies or institutions for the purpose of evaluating federally supported education programs under the Family Educational Rights and Privacy Act (FERPA, 34 CFR §§ 99.31(a)(3)(iii) and 99.35). The data are being collected for NCES by Westat, a U.S.-based research organization. All of the information [you / your child] provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151). The U.S. Office of Management and Budget has approved this data collection under OMB# 1850-0755.

FAQ for Parents About the PISA Pretest

Facts for Parents

About the PISA Pretest

In Januay of this year, your teenager’s school will be taking part in the Program for International Student Assessment (PISA) Pretest.. Your teenager was among those students selected to take part in the study.

What is PISA?

PISA is the world’s largest international assessment that measures student learning in reading, mathematics, and science. More than 80 countries and education systems representing nearly 90 percent of the world’s economy participate in PISA, which is coordinated through the Organization for Economic Cooperation and Development (OECD). The assessment occurs every 3 years (2000, 2003, 2006, 2009, 2012, 2015, 2018, and 2021) and provides information about how students in the U.S. compare in achievement with students in other countries. The National Center for Education Statistics (NCES), within the U.S. Department of Education, conducts PISA in the United States as authorized by the Education Sciences Reform Act of 2002 (ESRA 2002, 20 U.S.C. §9543).

What is the purpose of the PISAPretest?

The PISA Pretest is testing questions in electronic formats on tablet computers in preparation for the 2021 international study. This pretest will allow test developers to help ensure that the PISA assessment and field operations will be effective for the field test in spring of 2020.

What is involved?

PISA staff will visit the school and administer the assessment. The assessment will take approximately 3.5 hours; it includes time for instructions, the assessment, breaks, and a questionnaire that students complete about themselves.

What are the benefits?

The nation as a whole benefits from PISA by having a greater understanding of how the knowledge and skills of U.S. students compare with their peers in other countries. The country benefits from the pretest by ensuring that the questions used are fair and appropriate for U.S. students. Schools that participate in PISA will receive $250, and each student who participates will receive $25 plus a certificate from the U.S. Department of Education for 4 hours of volunteer service.

Who administers PISA and what is done wih the information collected from my teenager?

The entire assessment is administered by trained staff from Westat, a research organization under contract to the U.S. Department of Education’s National Center for Education Statistics.

PISA is not designed to produce individual scores and your teenager’s individual performance is not shared with teachers, the school, or the district in any way. Student responses are combined with other student responses and are only used for statistical purposes. The data collected for the PISA field test will be used to evaluate the functioning, validity and fairness of the items for the main study, to be conducted in 2021. All of the information provided by your teenager may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151). The U.S. Office of Management and Budget has approved this data collection under OMB# 1850-0755.

Where can I find out more about PISA?

More information about PISA is available at the PISA website at http://www.nces.ed.gov/surveys/pisa. If you have specific questions you can call PISA staff at 1-888-638-2597 or email us at [email protected].

The National Center for Education Statistics (NCES) is authorized to conduct the Program for International Student Assessment (PISA) by the Education Sciences Reform Act of 2002 (ESRA 2002, 20 U.S.C. §9543), and to collect students’ education records from educational agencies or institutions for the purpose of evaluating federally supported education programs under the Family Educational Rights and Privacy Act (FERPA, 34 CFR §§ 99.31(a)(3)(iii) and 99.35). The data are being collected for NCES by Westat, a U.S.-based research organization. All of the information [you / your child] provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151). The U.S. Office of Management and Budget has approved this data collection under OMB# 1850-0755.

Appendix B: Simulated School Environment Recruitment Materials

Recruitment E-mail/Letter to Youth Organizations

IF SENT AS E-MAIL - Subject: Education Research Opportunity for 15 year old Students for PISA

<DATE>

Dear <NAME>,

We are administering an important study on behalf of the U.S. Department of Education’s National Center for Education Statistics (NCES). Because your organization plays an active role in the community, we need your help.

We are looking for 15-year-old students to test new questions that will be used for the Program for International Student Assessment, also known as PISA, a computer-based international assessment of reading, mathematics, and science literacy administered periodically to 15-year-olds in schools throughout the United States and in more than 80 other countries. Student participation is vital in helping us develop the best way to deliver the assessment.

The voluntary research sessions will last for about 3.5 hours. During group sessions, students will be asked to answer questions using a tablet computer as if it were an actual assessment. Their feedback helps us find and fix problems before other students take the assessment nationwide. Because we care about student privacy, we will assign each student a unique student identifier. Student names will not be linked to their answers.¹

Each student will receive a $60 gift card from a major credit card company at the end of the session as a thank you for participation. In addition, the student’s parent or legal guardian will receive a $60 gift card from a major credit card company as a thank you for bringing the student to and from the testing site.

We would like to ask for your assistance in letting parents/legal guardians and students know how they can help shape this international assessment to be conducted in the United States. I have included a flyer with a brief description of our study in the hopes that you can post it for everyone to see. I have also included a Frequently Asked Questions (FAQs) page with additional information about the study.

We appreciate your consideration of this request and invite you to please contact us with any questions regarding this project. We look forward to speaking with you soon!

ATTACHED: Flyer^² and FAQs

Sincerely,

EurekaFacts <CONTACT PHONE>

EurekaFacts LLC Recruitment Phone Script for Community Centers and Organizations

Hello. May I please speak to <NAME OF CONTACT AT COMMUNITY CENTER>?

Good morning/afternoon/evening. My name is <CONTACT NAME> and I am contacting you from EurekaFacts, a local research company, and we need your help. We are carrying out an important study for the U.S. Department of Education’s National Center for Education Statistics (NCES). Because your organization plays an active role in the community, we are asking for your help in spreading the word about this exciting research opportunity.

We are recruiting 15- year-old students to help us evaluate computer-based questions that are part of an assessment called the Program for International Student Assessment, also known as PISA, which is delivered to students nationwide and in more than 80 other countries. [If community center or organization asks what PISA is: This international assessment is a computer-based assessment of reading, mathematics, and science literacy administered periodically to 15-year-olds in schools throughout the United States and in other countries. PISA is designed to gauge how well the students can apply their knowledge and skills to problems set in real-life contexts.]

Students play a key role in helping us improve our assessments. In fact, their participation enhances the experience of thousands of students all over the United States who will take the assessment in the spring. Student participation is vital in helping us fix any issues and develop the best way to deliver the assessment. The study will not retain any personally identifiable information. [Recruiter: If asked about confidentiality, please read the statement in the footnote.]³Each student who participates will receive a $60 gift card from a major credit card company at the end of the session as a thank you for participating. In addition, the student’s parent or guardian will also receive a $60 gift card from a major credit card company as a thank you for bringing their child to and from the testing site.

We would like your help in distributing and posting informational flyers to let parents and students know how they can be involved in this important study. We will follow up directly with those who contact us. We will hold sessions at <LOCATION>, <ADDRESS>.

Would your <community center or organization> be interested in advertising and/or helping us to recruit students for this study?

If NO: Thank you for your time. Have a wonderful day/evening. [END CALL]

If YES: Good, thank you very much. To ensure that we have a broad mix of students to participate in the research session, I have a few brief screening questions to ask you. It should take only a few minutes.

Screener

[Recruiter: Determine recruitment needs in advance. Look in public documents and the website to find what grades the community center or organization includes, as well as the size, socio-economic status (SES), and racial/ethnic makeup of the student population. Record all responses to Screener questions. End Screener at whatever point the staff member gives a response that confirms that their group has already been sampled adequately. End Screener Script: “Based on the requirements of this study, we are not able to include your students in the study at this time. Thank you for your time. Have a good day/evening.” [Interviewer: End call]]

1. Do you have <a child or children> who <is or are> 15 years old and who might be available to participate in the study?

If NO: Thank you for your time. Have a wonderful day/evening. [END CALL]

If YES: Great.

2. Would you be able to distribute and post flyers regarding this study? [Recruiter: Interested parties would contact EurekaFacts directly. We would follow up with them by phone and ask them some screening questions to ensure that a diverse sample of students is included in this study.]

If NO: Thank you for your time. Have a good day/evening. [Recruiter: End call]

If YES: Great. Thank you for taking the time to answer the screening questions. Your <community center or organization> is generally eligible to participate in the study. EurekaFacts staff will follow up with interested parents/legal guardians by phone to make interview arrangements. [Recruiter go to Closing]

(Closing)

Thank you for agreeing to share the information about this important study. We will send you an email with the informational flyers attached.

We may need to reach you by telephone. Is the number I called the best number to use?

Do you have any questions at this time? [Recruiter: Answer any questions.]

If you have any (additional) questions, please call <PHONE NUMBER>.

Thank you again for agreeing to be part of this important research project.

Recruitment Internet/Newspaper Advertisement

Shape8

Shape9

Flyer/Information Brochure

Shape10

EurekaFacts invites 15 year old students to participate in an important study for the U.S. Department of Education’s National Center for Education Statistics (NCES). Sessions will take place on <DATE>. Students will get a $60 gift card and the parent or legal guardian will also get a $60 gift card as a thank you for bringing the student to and from the testing site.

E-mail/Letter Recruiting Parents or Legal Guardians of Student (under age 18) Participants

IF SENT AS E-MAIL - Subject: Paid Education Research Opportunity

Dear <NAME>:

We are reaching out to parents of teenage students just like you, because we need your help. Our company, EurekaFacts, is administering an important education study for the National Center for Education Statistics (NCES), which is part of the U.S. Department of Education.

NCES is looking for 15-year-old students to test new questions that will be used for the Program for International Student Assessment (PISA). Students’ participation and opinions will help us find and fix issues before other students take the assessments nationwide in the spring.

The research session for your child will take place at <LOCATION> and will last about 3 and a half hours. Your child will be in a classroom setting with a group of students. They will answer questions using a tablet computer as if it were an actual assessment.

Your child will receive a $60 gift card from a major credit card company at the end of the session as a thank you. The parent or legal guardian who brings the student to and from the site will also receive a $60 gift card from a major credit card company as a thank you for his/her time and effort.

Because we care about your child’s privacy, we will assign him/her a unique student identifier. Your child’s name will not be linked to his/her answers.^⁴

The group session will be <on DATE OR scheduled in DATE RANGE>. If your child is interested in participating in this voluntary study, please call <CONTACT PHONE> or e-mail <CONTACT E-MAIL>. We look forward to speaking with you soon!

Thank you,

<NAME>

EurekaFacts

EurekaFacts LLC Recruitment Phone Script for Parents/Guardians of Student Participants

(Script when calling parents/guardians from a previously compiled list)

Good morning/afternoon/evening. Is this the <NAME FROM CALL LIST> household? My name is <CALLER NAME>, and I am calling from EurekaFacts, a local research company. Our company is administering an important education study on behalf of the National Center for Education Statistics (NCES), which is part of the U.S. Department of Education, and we need your help.

We are recruiting 15-year-old students to help us test computer-based questions. If selected to participate, your child will receive a $60 gift card from a major credit card company at the end of the session as a thank you. The parent or legal guardian who brings them to and from the study site will also receive a $60 gift card from a major credit card company as a thank you for his/her time and effort.

Are there any students in your household who are 15 years old?

	SELECT ONE
Yes	1	(CONTINUE)

No	2	(THANK AND TERMINATE)

We are administering a study to explore how students work through these questions. Students’ participation and opinions will help us find and fix issues before other students take the assessment nationwide in the spring. This assessment is part of a study called the Program for International Student Assessment, also known as PISA. [If parent/guardians asks what PISA is: This international assessment is a computer-based assessment of reading, mathematics, and science literacy administered periodically to 15-year-olds in schools throughout the United States and in other countries. PISA is designed to gauge how well the students can apply their knowledge and skills to problems set in real-life contexts.]

The study session will take place at <NAME OF STUDY SITE AND LOCATION> <on DATE OR scheduled in DATE RANGE> and will last about 3.5 hours. Your child will be in a classroom setting with a group of students. He/she will answer questions using a tablet computer as if it were an actual assessment.

NCES is authorized to conduct this study under the Education Sciences Reform Act ⁵ and all of the information you provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law⁶ [Recruiter: If parent/legal guardian asks about the specific laws please provide the citation from the footnote.]

If your child qualifies, do you give your permission for us to invite your child to participate?

	SELECT ONE
Yes	1	(CONTINUE)

No	2	(THANK AND TERMINATE)

What is your child’s sex?  (READ THE QUESTION AND RESPONSE OPTIONS)

	SELECT ONE
Male	1
Female	2

Would you like me to send the consent form by e-mail or by U.S. Postal mail?

SELECT ONE
e-mail	(CONTINUE TO 4A)

postal mail	(CONTINUE TO 4B)

4A. What is your e-mail address?

RECORD PARENT’S/GUARDIAN’S EMAIL ADDRESS:

____________________________________________________________________________________

4B. What is your postal mail address?

RECORD PARENT’S/GUARDIAN’S MAILING ADDRESS AND PHONE NUMBER:

____________________________________________________________________________________

The next few questions are for classification purposes only.

Does your child participate in the National School Lunch program?

SELECT ONE

Yes

1

No

2

Prefer not to answer

3
Do you speak a language other than English at home?

	SELECT ONE
Yes	1	(CONTINUE TO 6A)
No	2
Prefer not to answer	3

6A. What is this language? (For example: Korean, Italian, Spanish, Vietnamese)

Is your child Hispanic or Latino?

SELECT ONE

Yes

1

No

2

Prefer not to answer

3
Which of the following categories best describes your child’s ethnic or racial background? (READ LIST)

	SELECT ALL THAT APPLY
American Indian or Alaska Native		1
Asian		2
Black or African American		3
Native Hawaiian or Other Pacific Islander		4
White		5
Prefer not to answer		6

We are administering study sessions in <Month> and <Month>. If your child chooses to participate, which month would work best for you to take [HIM/HER] to the study session? <<OR, WE ARE ADMINISTERING STUDY SESSIONS ON DATE. WILL YOUR CHILD BE ABLE TO PARTICIPATE ON THAT DATE>>

[CHECK SCHEDULE FOR RESPONDENT’S AVAILABLE TIMES.]

RECORD THE PREFERRED MONTH:

____________________________________________________________________________________

Now that you have agreed for your child to participate in the study if he/she is selected, I would like to be able to ask your child a few questions, during which you are welcome to stay on the line. Is he/she available to speak with me at this time? (If not, get a call back time and call back) What is your child’s name?

RECORD NAME:

_________________________________________________________________________________

Again, you are welcome to stay on the line, but please allow your child to speak for him/herself. Thank you.

EurekaFacts LLC Recruitment Phone Script for Student Participants

RECRUITER: Please determine how talkative and articulate the student is. It is critical that we speak with students who are comfortable expressing themselves with someone they are meeting for the first time. Please use your best judgment and recruit only students that are thoughtful and freely express themselves in a manner that is relevant to the topic being discussed.

All recruits MUST be articulate students who can answer open ended questions, and can provide support or explanation for their responses (e.g., “I like it because...,” “I don’t like it because...”)

DO NOT recruit any who have difficulty answering these questions, whose parent/guardian answers FOR them, who seem hesitant or shy speaking with you (as an adult they do not know), who answer “I don’t know,” or who CANNOT provide an explanation supporting their responses.

Hi, <STUDENT’S NAME>. My name is and I have been speaking with your parent/guardian for a few minutes.

As I explained to your parent/guardian, I work for a research company and we are administering an important study to help us test questions for an international student assessment, the Program for International Student Assessment, also known as PISA. For the study session, if you agree to participate, you would be asked to visit <LOCATION> and complete computer-based questions. Your responses would not be graded. The National Center for Education Statistics (or NCES) simply wants to know how you answer the questions.

You will be in a classroom setting with a group of other students your age. You will answer questions using a tablet computer as if it were an actual assessment. Your participation will help us find and fix problems with the assessment before other students take these assessments nationwide in the spring. The session will take about 3.5 hours and you will receive a $60 gift card from a major credit card company to thank you for participating in the research session. If a parent or legal guardian brings you to and from the session site, they will also receive a $60 gift card from a major credit card company to thank them for their time and effort.

As I said before, your responses would be used to improve the national assessment so that it works well for all students. We would not use your name to identify you in any of our records.

Do you think you would be interested in participating if selected?

	SELECT ONE
Yes	1	(CONTINUE)

No	2	(READ THE FOLLOWING)

If No: Thank you for your time. I will need to let your parent/guardian know that you would not like to participate. Is he or she available?

TO PARENT/GUARDIAN: Your child has decided not to participate, but I would like to thank you very much for taking the time to speak with us about this opportunity.

How old are you?

RECORD AGE

(MUST BE 15 years old OTHERWISE THANK AND TERMINATE)

What type of school do you currently attend? (READ LIST)

	SELECT ONE
Public school	1	(CONTINUE)
Private school	2	(CONTINUE)

Don’t know	3	(THANK AND TERMINATE)

You qualify to participate in a study session with a researcher in < LOCATION>. Your parent/legal guardian is available to bring you to the study on <DAY/DATE AND TIME>. The session will last about 3.5 hours. You will receive a $60 gift card from a major credit card company for participating. Your parent/guardian gave permission for you to attend if you are available. Will you be able to attend the session?

	SELECT ONE
Yes	1	(CONTINUE)

No	2	(THANK AND TERMINATE)

To attend the session and receive the $60 we must receive the parental consent form that we will [e-mail OR send by U.S. Postal mail] to your parent/guardian. Your parent/guardian can scan and e-mail the consent form to us, or you can bring it with you and give it to the study leader. Do you understand that you must have a completed and signed parental consent form to participate?

	SELECT ONE
Yes	1

No	2

Thank you. We look forward to seeing you at the research session.

[END OF SCREENER]

Frequently Asked Questions (FAQs)

Frequently Asked Questions

(From Consent Form)

What is this study about?

The National Center for Education Statistics (NCES), part of the U.S. Department of Education, is conducting a study with students who are 15 years old about questions that will be used for the Program for International Student Assessment (PISA). Students’ participation will help them find and fix issues before other students take the assessments nationwide in the spring.

Where will the research session take place?

The study session will take place at <NAME OF RESEARCH SESSION LOCATION>, located at <LOCATION>. There will be more than one adult present in the room.

What will happen during the research session?

Students are assessed in a group setting. At the start of a session, students will be given a brief introduction to the process and to the task(s) they will be asked to complete. Following this introduction, the student will be asked to complete a real-world assessment of the PISA system, allowing the systems to be tested in the manner that will be used in the international study to help identify system issues during the software development process.

How long will the research session last?

The session will last about 3.5 hours. The session requires only one visit to the site.

Will you keep information private and confidential?

The participating student will be assigned a unique student identifier (ID) and his/her name will not be linked to any of his/her answers.⁷

Will the results go to the participating student’s school?

The participating student’s answers will not be disclosed to their school and will not be linked to any personally identifiable information, such as the school’s name.

What are the possible risks of being in the study?

There are no known risks to participating in this study.

Whom can I contact with questions or for further information?

<CONTACT NAME> is coordinating this session at EurekaFacts on behalf of NCES. If you have any questions, please call <CONTACT PHONE> or e-mail <CONTACT E-MAIL>.

Is participation required?

Participation in this study is completely voluntary. The participating student has the right to refuse to answer particular questions. The student may elect to withdraw from this study at any time and can end early if he/she wishes.

Consent Form for Parent/Guardian of Student Participants

PARENTAL CONSENT FORM

FOR PARTICIPATION IN RESEARCH

Thank you for your interest in participating in the study described below. This consent form explains the study. Please read it carefully and feel free to ask questions about anything you do not understand. If you do not have questions now, you may ask them later as they occur to you.

What is this study about?

The National Center for Education Statistics (NCES), part of the U.S. Department of Education, is conducting a study with 15 year old students about questions that will be used for the Program for International Student Assessment (PISA). Students’ participation will help them find and fix issues before other students take the assessments nationwide in the spring.

Where will the study sessions take place?

The study sessions will take place at <NAME OF STUDY LOCATION>, located at <LOCATION>. There will be more than one adult present in the room.

What will happen during the research session?

Students are assessed in a group setting. At the start of a session, they are given a brief introduction to the process and to the task(s) he or she will be asked to complete. Following this introduction, your child will complete the task and discuss in a group setting with other students what he/she thought about while working through the task(s).

How long will the research session last?

The session will last about 3.5 hours. The session requires only one visit to the site.

Will you keep information private and confidential?

Your child will be assigned a unique student identifier (ID) and his/her name will not be linked to any of his/her answers.⁸

Will the results go to my child’s school?

Your child’s answers will not be disclosed to his/her school and will not be linked to any personally identifiable information, such as the school’s name.

What are the possible risks of being in the study?

There are no known risks to participating in this study.

Whom can I contact with questions or for further information?

<CONTACT NAME> is coordinating this session at EurekaFacts on behalf of NCES. If you have any questions, please call <CONTACT PHONE> or e-mail <CONTACT E-MAIL>.

Is participation required?

Participation in this study is completely voluntary. Your child has the right to refuse to answer particular questions. Your child may elect to withdraw from this study at any time and can end early if he/she wishes.

By signing below, you agree that your child, _______________________________, may participate in this study and that we may record your child’s conversation with the researcher. Your child will receive a $60 gift card from a major credit card company at the conclusion of the session.

In addition, if transportation to the site is involved, you will receive a $60 gift card from a major credit card company to thank you for bringing your child to and from the study session site.

Your signature below will indicate that your questions have been answered satisfactorily, and that you have read and understood the information provided above.

Print Name: ___________________________________________________________________________

Signature: ____________________________________________________Date: __________________

E-mail address: _________________________________________________________________________

Day of Research Session Reminder E-mail

Dear <NAME>:

Thank you for agreeing to help us try out new computer-based questions from the Program for International Student Assessment (PISA).

The session will last about 3.5 hours. We will begin promptly at <TIME> and it is important that you arrive 15 minutes early to check in. If you are late, we cannot guarantee that your child will be able to participate and receive the $60 gift card from a major credit card company.

This is a friendly reminder that your child has been scheduled to participate in a study at the following location and date/time:

LOCATION: Location Address

DATE: Day of Week, Month Day, Year

TIME: XX:XX AM/PM

GIFT CARD: Your child will receive a $60 gift card to thank him/her for their participation. If you bring your child to and from the research session, you will also receive a $60 gift card from a major credit card company as a thanks for your time.

PLEASE BRING THE FOLLOWING ITEMS TO THE RESEARCH SESSION:

Parent/guardian valid picture ID – this is required for participation.
Signed parent/guardian consent form (attached). If needed, we have copies available to complete when you arrive.
If your child uses eyeglasses for reading, computer work, or watching TV, they should wear them during the study session.

Thank you for volunteering in this very important research for NCES. We look forward to seeing you. If you have any questions about the study, please contact <EurekaFacts CONTACT NAME> at <CONTACT INFORMATION>.

Sincerely,

<NAME>

EurekaFacts Confirmation of Research Session (Email/Letter) to Student Participant

<DATE>

IF SENT AS E-MAIL - Subject: Confirmation of Your Participation in our Study

Dear <NAME>:

Thank you for agreeing to help us try out new computer-based questions from the Program for International Student Assessment (PISA). Your comments, along with the comments of other students, will help us identify and address issues prior to the operational use of the system. You will not be graded on your work.

Your participation is totally voluntary. All of your answers may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151).

The research session will last about 3.5 hours. We will begin promptly at <TIME> and it is important that you arrive 15 minutes early to check in. If you are late, we cannot guarantee that you will be able to participate and receive the $60 gift card from a major credit card company.

LOCATION: <ADDRESS>

DATE: <DATE>

TIME: <TIME>

GIFT CARD: You will receive $60 to thank you for your participation. If your parent or legal guardian brings you to and from the research session, he or she will receive $60 to thank them for bringing you.

PLEASE BRING THE FOLLOWING ITEMS TO THE STUDY SESSION:

Signed <parent/guardian> consent form, if not previously provided by e-mail.
If you use eyeglasses for reading, computer work, or watching TV, you should wear them during the study session.

If for any reason you cannot fulfill your commitment, DO NOT invite anyone else to take your place. Instead, please call us at <PHONE NUMBER> so we can find a replacement for you.

Sincerely,

<NAME>

EurekaFacts Confirmation of Participation (Email/Letter) to Parent/Guardian of Student Participant

<DATE>

IF SENT AS E-MAIL - Subject: Confirmation of Your Child’s Participation in Study

Dear <NAME>,

Thank you for agreeing to allow your child to participate in this important study about the Program for International Student Assessment (PISA). Your <son OR daughter> is scheduled for <his OR her> session on <DAY AND DATE> at <NAME OF STUDY LOCATION>, located at <LOCATION>, near <NEARBY LANDMARK OR METRO STATION>.

Your child will be assigned a unique student identifier (ID) and his/her name will not be linked to any of his/her answers.⁹

We will begin promptly at <TIME> and it is important that you arrive 15 minutes early to check in. If you are late, we cannot guarantee that your child will be able to participate and that you and your child will receive $60. The session will last about 3.5 hours.

Please read and sign the attached consent form. Your child can either bring the signed form to the session or you can scan it and e-mail it back. Without a signed form, your child cannot participate in the study and receive the $60.

PLEASE BRING THE FOLLOWING ITEMS TO THE RESEARCH SESSION:

Your valid picture ID is required for participation.
Signed parent/guardian consent form, if not previously provided by e-mail (attached).
If your child uses eyeglasses for reading, computer work, or watching TV, he/she should wear them during the study session.

If for any reason you cannot fulfill your child’s commitment, DO NOT invite anyone else to take your child’s place. Instead, please call us at <PHONE NUMBER> so we can find a replacement for participating in the study.

Thank you for allowing your child to volunteer in this very important research for NCES. We look forward to seeing you. If you have any questions about the study, please contact <EurekaFacts CONTACT NAME> at <CONTACT INFORMATION>.

Sincerely,

<NAME>

Attachment: Consent Form for Parent/Guardian of Student Participants <Appendix>

Thank You E-mail/Letter to Parent/Guardian of Student Participants

<DATE>

IF SENT AS E-MAIL - Subject: Thank you for your child’s participation in the PISA Study

Dear <Name>,

On behalf of the National Center for Education Statistics (NCES), part of the U.S. Department of Education, thank you for allowing <CHILD’S NAME> to participate in the study for the Program for International Student Assessment (PISA). <CHILD’S NAME>’s input helped us identify and address issues prior to the operational use of the system. The assistance <he/she> provided, combined with input from other students, was incredibly useful and informative. We greatly appreciate you taking the time to bring <CHILD’S NAME> to and from the study location to participate.

For more information on PISA, please visit the NCES PISA website at: https://nces.ed.gov/surveys/PISA/.

Again, thank you for your assistance.

Sincerely,

<NAME>

EurekaFacts

1 NCES is authorized to conduct this study under the Education Sciences Reform Act of 2002 (ESRA 2002, 20 U.S.C. §9543). All of the information you provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151).

2 Note: Text for the flyer/brochure can be found below.

3 NCES is authorized to conduct this study under the Education Sciences Reform Act of 2002 (ESRA 2002, 20 U.S.C. §9543). All of the information you provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151).

4 NCES is authorized to conduct this study under the Education Sciences Reform Act of 2002 (ESRA 2002, 20 U.S.C. §9543). All of the information you provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151).

5 The Education Sciences Reform Act of 2002 (ESRA 2002, 20 U.S.C. §9543).

6 20 U.S.C. §9573 and 6 U.S.C. §151.

7 NCES is authorized to conduct this study under the Education Sciences Reform Act of 2002 (ESRA 2002, 20 U.S.C. §9543). All of the information you provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151).

8 NCES is authorized to conduct this study under the Education Sciences Reform Act of 2002 (ESRA 2002, 20 U.S.C. §9543). All of the information you provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151).

9 NCES is authorized to conduct this study under the Education Sciences Reform Act of 2002 (ESRA 2002, 20 U.S.C. §9543). All of the information you provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151).

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
Author	Stancavage, Fran
File Modified	0000-00-00
File Created	2021-01-15

Program for International Student Assessment (PISA) 2021 Field Test Pretest

NCES System Clearance for Cognitive, Pilot, and Field Test Studies 2019-2022

Appendices PISA 2021 FT Pretest