Changes Memo

_MEMORANDUM NATIONAL CENTER FOR EDUCATION STATISTICS

Institute of Education Sciences

United States Department of Education

Date: March 1, 2022

To: Robert Sivinski, OMB

Through: Carrie Clarady, OMB Liaison, NCES

From: Samantha Burg, NCES

Re: Program for International Student Assessment 2022 (PISA 2022) Main Study Change Memo (OMB# 1850-0755 v.27)

The Program for International Student Assessments (PISA) is an international assessment of 15-year-olds, which focuses on assessing students’ reading, mathematics, and science literacy. PISA was first administered in 2000 and is typically conducted every three years. The United States has participated in all of the previous cycles and planned to participate in 2021 in order to track trends and to compare the performance of U.S. students with that of students in other education systems. PISA is sponsored by the Organization for Economic Cooperation and Development (OECD). In the United States, PISA is conducted by the National Center for Education Statistics (NCES), within the U.S. Department of Education. In each administration of PISA, one of the subject areas (reading, mathematics, or science literacy) is the major domain and has the broadest content coverage, while the other two subjects are the minor domains. PISA emphasizes functional skills that students have acquired as they near the end of mandatory schooling (aged 15 years), and students’ knowledge and skills gained both in and out of school environments. The next administration of PISA will focus on mathematics literacy as the major domain. Reading and science literacy will also be assessed as minor domains, with additional assessment of financial literacy. In addition to the cognitive assessments described above, PISA 2022 will include questionnaires administered to school principals and assessed students. To prepare for the main study, PISA countries will conduct a field test in the spring of the year previous, primarily to evaluate newly developed assessment and questionnaire items but also to test the assessment operations.

The request to conduct PISA 2022 main study recruitment and field test was approved in February 2021 (OMB# 1850-0755 v.25) and included the plan to submit a change request to OMB in early spring 2022 with updates to the respondent burden estimates for the main study data collection, the final main study procedures and instruments for data collection in the fall of 2022. This change request:

1. Adds the inclusion of the Global Crisis Module

2. Changes the allotted time for the student and school questionnaires

3. Updates the burden hours and cost amounts

4. Updates the statistical methodology for the main study

5. Includes changes to various months and dates referenced for the main study

Changes to all documents are described below in more detail. Specific illustrated changes are marked in red; additions are in simple red font, while deletions are in red strikethrough. This request provides the final plans, burden and materials for the PISA 2022 main study data collection.

Based on feedback and experience from the PISA 2022 field trial, NCES is requesting OMB approval for the changes to aspects of the PISA 2022 main study descriptions in the Supporting Statements Parts A and B and to the main study recruitment materials in Appendix A2 as detailed below.

1. Specific material changes to Part A

Section A.2: Inclusion of the Global Crisis Module (p.5-6)

Global Crisis Module. The impact of the COVID-19 pandemic on schools and students is of great interest internationally. A module of pandemic-related items, called the Global Crisis Module, was developed for the field test for both the school and student questionnaires and will be administered in the main study. This module has 3 distinct purposes:

• capture key contextual variables to describe students’ learning experiences during times of school building closures as well as preparedness for future global crises;

• focus on educational opportunities as well as students’ subjective views on distance learning; and

• avoid potentially sensitive questions (e.g., about personal health issues due to COVID-19).

The school and student questionnaires, including these newly developed items, are presented in appendix C.

Section A.12: Changes in time allotted for questionnaires (p.13)

For data collection, in 50 schools field test and 256 for the national sample: (a) a 53-minute school administrator questionnaire for the field test and a 45-minute school administrator questionnaire for the national sample¹, (b) a 3-minute review of consent forms by parents, (c) a 36 35-minute computer-based core background student questionnaire, (e) a 15 10-minute ICT student questionnaire, and (f) a 15 10-minute set of FL student questionnaire items.

Section A.12 (Table A-1): Changes in hours of burden estimates (p. 14)

Table A-1. Burden estimates for PISA 2022 main study

MAIN STUDY —Based on core + international options
US national sample
Recruitment and Pre-Assessment Activity
School Administrator	297	.84	249	249	90	374
Special Handling Districts IRB Staff Approval (US sample)	31	1.00	31	31	120	62
Special Handling Districts IRB Panel Approval (US sample)	187	1.00	187	187	60	187
School Coordinator	249	.85	212	212	240	848
School Administrator
Questionnaire	212	1.00	212	212	45	159
Parent
Student Participation Consent	10,418	1.00	10,418	10,418	3	521
Student
Directions	10,418	0.80	8,334	8,334	10	1,389
Assessment	10,418	0.80	8,334	8,334	120	16668
Student questionnaire (main questionnaire)	10,418	0.80	8,334	8,334	36 35	5000 4,862
Student questionnaire (FL questionnaire)	10,418	0.80	8,334	8,334	15 10	2084 1,389
Student questionnaire (ICT questionnaire)	10,418	0.80	8,334	8,334	15 10	2084 1,389
State samples (up to 3 states)
Recruitment and Pre-Assessment Activity
School Administrator (US states)	162	1.00	162	162	90	243
School Coordinator (US states)	162	1.00	162	162	240	648
School Administrator
Questionnaire (US states)	162	1.00	162	162	45	122
Parent
Student Participation Consent	8,424	1.00	8,424	8,424	3	421
Student
US states (includes up to 3)
Directions	8,424	0.83	6992	6992	10	1,165
Assessment	8,424	0.83	6992	6992	120	13,984
Student questionnaire (main questionnaire)	8,424	0.83	6992	6992	36 35	4,195 4079
Student questionnaire (FL questionnaire)	8,424	0.83	6992	6992	15 10	1,748 1165
Student questionnaire (ICT questionnaire)	8,424	0.83	6992	6992	15 10	1,748 1165
Total School Recruitment Burden - Main Study			1,003	1,003		2,362
Total Burden Requested in this Submission (v.26) (v.27)			6,748 35,171	11,728 66,197		5,691 18,916
NOTE: OMB Clearance Requested: Total Burden includes all burden associated with conducting the PISA 2022 Field Test and the recruitment and preassessment activities for the PISA 2022 Main Study. The estimated PISA 2022 Main Study burden is conservatively high because the PISA 2022 Main Study may include up to 3 states, however the burden is held consistent with national sample schools because of potential variability between states. The total student burden does not include time for cognitive assessment and its associated instructions, because assessment is not subject to PRA. *Special note: For the national main study sample, we expect to draw an initial sample of 297 schools. Taking into account closed, merged, and ineligible schools (assuming a similar rate to 2018, around 16% of sampled schools), as well as the historical school-level response rate, we anticipate interacting with/recruiting about 249 of these schools, of which, we estimate, 212 will participate in the PISA 2022 main study. To obtain the required number of students, we will ask to sample up to 52 students in each school. However, some of the smaller schools will not have 52 students available. We estimate: 212 schools x 52 students sampled x 0.945 (a factor used to account for variations in student population sizes across the schools) = 10,418 starting student sample size that we will work to recruit. Based on the 2018 student assessment rates (~80%), we estimate that, in the end, we will assess about 8,334students (10,418x 0.80), which will assure that we meet the minimum required 7,950 assessed students.

Changes in estimated hourly rates and burden hours (p. 14)

The estimated hourly rates for secondary noninstructional staff/coordinators, principals/education administrators, and parents ($22.10 $22.70, $48.24 $49.52, $25.72 $27.07respectively) are based on Bureau of Labor Statistics (BLS) May 2019 National Occupational and Employment Wage Estimates.² The federal minimum wage of $7.25 is used as the hourly rate for students. For the PISA 2022 field test and recruitment for the main study, for the national and state samples, a total of 5,716 18,916 burden hours are anticipated, resulting in an estimated burden time cost to respondents of approximately $121,746 $227,408.

Section A-16: Changes in projected dates for publishing General Audience Report (p. 15-16)

General Audience Report. Approximately one year after data collection, in December 2022 2023, to correspond with the international data release, NCES will publish the U.S. National First Look Report with information on the status of reading, mathematics, and science education among students in the U.S. in comparison to their international peers, written for a non-specialist, general U.S. audience. This report will present the results of analyses in a clear and non-technical way, conveying how U.S. students compare to their international peers, and what factors, if any, may be associated with the U.S. results.

The results for financial literacy will be released internationally in 2023 2024, with NCES releasing a Data Point report to present the results of financial literacy for the U.S. Accompanying both the First Look report and the Data Point, will be a set of tables and figures released on the NCES website to provide additional details.

Following the release of the national report, additional data will be made available to secondary users in the form of the International Data Explorer (IDE), an online tool on the NCES website, and a U.S. public-release dataset.

2. Specific material changes to Part B

B.1 Respondent Universe: Changes in dates for main study and recruitment (p.1)

PISA 2021 2022 assesses students nearing the “end of their compulsory school experience” and, as all prior administrations of PISA, is conducted in the United States by the National Center for Education Statistics (NCES) within the U.S. Department of Education. For international comparability, the target population is defined as students who are 15 years old, in grades 7 or higher. A range of exact birthdates is specified by the international coordinating committee based on the months in which the data will be collected. However, students must be between the ages of 15 years and 3 completed months and 16 years and 2 completed months at the beginning of the testing period. In the U.S., the universe for the selection of schools is all types of schools in the 50 states and the District of Columbia. Within sampled schools, students will be selected for participation by drawing a random sample among the 15-year-old students.

B.2 Procedures for the Collection of Information

This section presents information on the PISA international standards and description for school and student sampling, recruitment, and data collection procedures for the PISA 2022 main study. Gaining schools’ and students’ cooperation in voluntary research is increasingly challenging and employing effective strategies for gaining the cooperation of schools is central to the data collection effort. PISA 2022 main study states, districts, and schools will be recruited beginning in October 2021 March 2022 and data collection will be conducted from September October-November 2022.

B.2a Statistical Methodology

General Changes:

• Omitted references to the Field Test

• Updated dates and timelines to better reflect the Main Study

Sampling Frame of Schools

The population of schools for PISA 2022 is defined as all schools containing any 15-year-olds in grades 7 through 12. As in previous PISA cycles, the school sampling frame for the PISA 2022 main study sample will be was developed from the most up to date NCES Common Core of Data (CCD) and Private Schools Survey (PSS) datasets. For the PISA 2022 Field Test, we will use the school sample selected for the initial Field Test that was planned for Spring 2020, but suspended due to the spread of the pandemic. That sample used the sampling frame prepared for the National Assessment of Educational Progress (NAEP) 2019, which uses the 2017-2018 CCD and the 201702018 PSS school data. We will minimize overlap, to the degree possible, with NAEP and the Progress in International Reading Literacy Study (PIRLS), which will be collecting data in schools at the same time. For the PISA 2022 Main Study, we used the schools sampling frame prepared for the National Assessment of Educational Progress (NAEP 2019, which uses the 2018-2019 CCD and the 2017-2018 PSS school data. The 2022 National Assessment of Educational Progress (NAEP 2022) does not include a grade 12 school sample. Therefore, overlap with the PISA 2022 school sample is not expected and the use of overlap control is not necessary.

The grade structure of the school is a key stratification variable designed to reduce sampling error, but this is especially so in PISA because data analyses have shown that achievement is highly related to grade. Other stratification variables may include public/private schools, region of the country, location (urban/suburban/town/rural, etc.), gender, state, enrollment of 15 year olds, and enrollment by race/ethnicity.

Field Test Sampling (section deleted)

Overview of Main Study

For the core computer-based assessment in mathematics, reading, and science, the international required minimum number of completed assessments is 6,300 students in 150 schools. An additional 1,650 assessed students are required for education systems assessing financial literacy for total of 7,950 assessed students. To achieve a larger number of students assessed than in 2018 and required for financial literacy, and to account for anticipated nonparticipation and student ineligibility, the number of students sampled within schools will again be 52 students, as it was in 2018, but with a larger school sample. Assuming the same response level as in PISA 2018, the initial target is a total sample of about 297 schools, with estimated 249 eligible, to yield about 212 participating schools (assuming a total 85% school participation rate). As allowed under the international sampling standards, to achieve the target final school response rate, we will use replacement schools to complete the sample.

Assuming a within-school student assessment rate of 80 percent (rates were 81 percent in 2000, 70 percent in 2003, 83 percent in 2006, 78 percent in 2009, 81 percent in 2012, 83 percent in 2015, and 78 percent in 2018), the original sample size of students within schools will be up to 52 students. In schools that do not have 52 PISA-eligible students, all eligible students will be sampled. Should any states participate in the 2022 assessment, each state would have a sample of 60 schools and 2,948 students to yield 2,447 assessed students (given an expected 83% assessment rate). As the main study plans for states and subnational jurisdictions are finalized, this information will be updated in the respondent burden table in the Supporting Statement Part A.

Field Test Instrumentation and Design (section deleted)

Cognitive Design: Significant changes in cognitive design between field test and main study. Assessment design reduced from 66 forms to 45 forms. Added results from field test (last paragraph below)

For the PISA 2022 field test, there are a total of 66 forms in the field test containing a combination of clusters in trend mathematics and science, trend mathematics and reading MSAT, trend mathematics and new mathematics, trend mathematics and financial literacy, financial literacy and reading, or only new mathematics. For the assessment, which will be administered in a 2-hour session, students will receive one form with a combination of clusters depending on the form.

Cognitive items to be administered in the field trial consist of the following subjects and number of clusters (groups of items/units):

Mathematics = 6 intact trend clusters and 12 new clusters

Reading = a reduced version of the 2018 reading MSAT;

Science = 6 intact trend clusters

Financial Literacy = 2 clusters assembled from new and trend items

For the PISA 2022 main study, the United States will use an assessment design with total of 45 forms containing a combination of clusters mathematics MSAT and science, mathematics MSAT and reading MSAT, reading MSAT and science, mathematics MSAT and financial literacy, and financial literacy and reading MSAT. For the assessment, administered in a 2-hour session, each student will receive one form with a combination of clusters depending on the form.

The results of the field test showed no significant difference between the percentage of students that reached the items and selected the correct answers in the FUO versus the VUO forms both for reading trend and new items. The pairs for both trend items and new items were observed to be very similar in response time and no unit order effect was found. Additionally, machine-based coding was introduced, showing a 20 percent reduction of human coding burden across items, domains, and participating countries. Based on this evidence, it was determined that the order in which units were presented did not significantly impact the estimation of item and person parameters and an MSAT design could be introduced for the first time into the main study.

Financial Literacy Design: Changes in years referenced and projected yield of students:

The U.S. is again participating in the optional financial literacy assessment in 2021 2022. The 2021 2022 assessment design is the same as the one used in PISA 2018, with an expanded student sample used to assess financial literacy in the same session that mathematics, science, and reading are assessed. Approximately 12 students will be were assigned financial literacy assessment items, while the remaining 48 40 students with receive receiving combinations of mathematics, reading and science items. Each student sampled for financial literacy will receive two clusters of math or reading and two clusters of financial literacy. The design for financial literacy is based on a yield of 384 1,650 assessed students. These students will be selected separately from those sampled for the core assessment but will be administered the assessment in the same session as the students taking the core assessment and will be included in Group 1 (see explanation above). The financial literacy instrument will contain contains 2 clusters with trend items from 2012, 2015, and 2018 as well as new interactive items.

Student questionnaires. In the 20221 cycle, the U.S. will administer three student questionnaires that will be completed in a single student questionnaire session following the assessment session: the core student questionnaire, financial literacy questionnaire and the information and communication technology (ICT) familiarity questionnaire. The content of these instruments is described below. Students access the questionnaires through the Student Delivery System (SDS), in the same way they access the assessment, approximately 150 minutes after the assessment session has ended. Unlike the school questionnaire, the student questionnaires are only administered from the SDS, and students only access the questionnaires in the questionnaire session. That is, student access to the questionnaires can only be accomplished during the in-school session; there is no access to the student questionnaire outside of school hours.

Student Core Questionnaire

The goal is for the student questionnaire to take approximately 356 minutes to complete in the field test and the main study. The PISA 2022 field test student questionnaire will implement a matrix sampling design. Using within-construct matrix sampling is an innovation to the questionnaire design in PISA 2022 that has not been used in previous PISA cycles where different respondents will receive different sets of items to reduce student burden while maintaining content coverage across relevant areas. This approach is available for PISA 2022 due to the limited time available e for the questionnaire and the large student sample size in large-scale assessments. This design is expected to be carried to the main study.

The approach being proposed for PISA 2022 will utilize an alternative matrix sampling design rotates questions within constructs instead of across constructs. In the PISA 2022 proposed within-construct matrix sampling design, every student will receive questions on all constructs but only answer a subset of all questions for each construct, thus resulting in a complete database in terms of construct-level indices. The PISA field test data for most of the scales analyzed with IRT was collected using the new within-construct matrix sampling design for questionnaires where each student answers a randomly chosen subset of five items for each construct from the pool of 8-10 items for the respective construct. The evaluation of the design confirmed that scales scores could be estimated for every student in the sample, regardless of which specific set of items the students answered. Based on this evaluation of the field test data, this design will be used only for constructs that:

• can be IRT scaled across country/economy-by-language groups without substantial misfit,

• meet minimal reliability criteria established in previous PISA cycles (i.e., >.6), will be reported as scaled indices (vs. reported on the item-level only), and

• are not a component of the PISA ESCS index.

This approach will be implemented for a select number of scales in the field test. A decision on the use of this design for the main study will be made based on an empirical evaluation of the PISA 2022 Field Test results. This approach is viable for PISA 2022 due to the limited time available for the questionnaire and the large student sample size in large-scale assessments.

Page 5 (Financial Literacy Questionnaire): Estimated time changed from 15 minutes to 10 minutes.

Page 5 (Information and Communication Technology (ICT) Familiarity questionnaire): Estimated time changed from 15 minutes to 10 minutes.

B.2b Respondent Recruitment

Pages 6-8: Revised to remove references to the Field Test and reflect the dates and timeline of the PISA 2022 Main Study.

B.2c Data Collection

Page 10: Student break changed from 15 minutes to 10 minutes.

3. Specific material changes to Appendix A

The School Contact Materials in Appendix A underwent only primarily general changes, including:

• Deleted all references to the Field Test

• Deleted all references to the Teacher Questionnaire

• Dates changed from “September and October 2021” to “October and November 2022”

• School year changed from “2021-2022” to “2022-2023”

• Sample size of students (per school) changed from 60 to 52

• Birth date range changed from “between July 1, 2005 and June 30, 2006” to “between July 1, 2006 and June 30, 2007”

• Added a link to the newly available results of PISA 2018 to demonstrate what the results of PISA 2022 are likely to look like

See below for more specific changes to Appendix A.

PISA 2022 School Coordinator’s Handbook (p. 44)

Exhibit 2.1. Timing of Assessment

Activity	Student Time	Room Availability Time
Student Questionnaire	66 60 minutes (approximately)	66 60 minutes (approximately)

MyPISA project website for School Coordinators and Schools (p. 83)

The following screenshots show the content for the PISA 2022 Main Study and show the web content of MyPISA.us for schools and school coordinators with text for each screen shown below the screen. The systems for the PISA 2022 main study are not yet active, but will not change other than references to the current round of PISA and updates to the specifics of the main study, such as number of students to be sampled and removal of teacher information pertaining to teacher participation in the questionnaire. MyPISA.us is described in Part B of the main study package.

Updated screenshots and, if necessary, any updated text will be provided in a change request by May 2022. The text displayed below each screenshot in this section is the text as it will appear on MyPISA.us for PISA 2022.

4. Specific material changes to Appendix B – no changes

5. Specific material changes to Appendix C

Two types of changes were made to the field test versions of PISA 2022 questionnaires in preparing them for the main study: (1) wording or structure of some questions was revised and (2) items were deleted. All changes are listed in the table below. The field test questionnaire item numbers correspond to those in the OMB approved Appendix C Field Test Questionnaires and are provided below to facilitate cross referencing the noted changes.

School Questionnaire

1. Items with revised wording or structure

2022 Field Test		2022 Main Study
Item #	Field Test Item	Item#	Main Study Item
SC002	As of February 1, 2021, what was the total school enrolment (number of students)?	SC002	As of February 1, 2022, what was the total school enrolment (number of students)?
SC064	During 2019-2020 academic year, what proportion of students' parents participated in the following school-related activities?	SC064	During 2021-2022, what proportion of students' parents participated in the following school-related activities?
SC192	During the 2019-2020 school year, how often has the school staff done each of the following?	SC192	During the 2021-2022 school year, how often has the school staff done each of the following?
SC032	During the 2019-2020 school year, have any of the following methods been used to monitor the practice of teachers at your school?	SC032	During the 2021-2022 school year, have any of the following methods been used to monitor the practice of teachers at your school?
SC173	During the 2019-2020 school year, how often has the school staff done each of the following?	SC173	During the 2021-2022 school year, how often has the school staff done each of the following?

2. Deleted school questionnaire items from field test

SC159	Does your school host visiting teachers from other countries?
SC179	Approximately what percentage of students in your school repeated a grade at the following grade levels last academic year?
SC164	In the last full academic year, what proportion of students in your school’s final grade left school without a diploma or alternative credential (e.g., a GED)?
SC203	Who has the main responsibility for the following at your school?
SC201 – Q02	During the last 12 months, how often did you or other members of the school management team engage in the following activities? Observing instruction in the classroom
SC201 – Q08	During the last 12 months, how often did you or other members of the school management team engage in the following activities? Reviewing school administrative procedures and reports
SC201 – Q09	During the last 12 months, how often did you or other members of the school management team engage in the following activities? Resolving problems with the lesson timetable in this school
SC201 – Q10	During the last 12 months, how often did you or other members of the school management team engage in the following activities? Collaborating with principals from other schools on difficult work tasks
SC191	Is there state or district legislation on including students’ parents or guardians in school activities?
SC190- Q03	Do the following statements apply to your school? Students' cell phones are collected when entering the school or before the start of class.
SC190 – Q04	Do the following statements apply to your school? The school has a written statement specifically about the use of digital devices for pedagogical purposes.
SC199	Thinking about the last internal evaluation/self-evaluation in your school: Did the following occur?
SC200	Thinking about the last external evaluation in your school: Did the following occur?
SC027- Q01	Which of the following types of in-house professional development exist at your school? The teachers in our school cooperate by exchanging ideas or materials when teaching specific units or series of lessons.
SC183-Q1	Which of the following types of in-house professional development exist for staff who teach mathematics at your school? The mathematics teachers in our school cooperate by exchanging ideas or materials when teaching specific units or series of lessons.
SC172-Q01	To what extent is each of the following behaviours a problem in your school? Cheating
SC172-Q08	To what extent is each of the following behaviours a problem in your school? Physical injury caused by students to teachers or non-teaching staff
SC186	Are the following subjects required or optional for students in grade 10?
SC150	Does your school offer any of the following options to students in <national modal grade for 15-year-olds> whose heritage language is not English
SC188-Q01	To what extent is your school's mathematics program structured according to the following resources? District curriculum standards or curriculum guides
SC188-Q02	To what extent is your school's mathematics programme structured according to the following resources? State curriculum standards or curriculum guides
SC188-Q05	To what extent is your school's mathematics programme structured according to the following resources? Results from districts assessments
SC188-Q06	To what extent is your school's mathematics programme structured according to the following resources? Results from state assessments
SC196	In your school, are mathematics standardized tests and/or teacher-developed tests of students in grade 10 used for any of the following purposes?
SC158	Is there any formal curriculum for the following topics in grade 10?
SC217-Q09	During the time when your school building was closed to students because of COVID-19, were teachers in your school asked to do the following things? Keep track of students’ attendance in distance learning activities
SC209-Q01	With respect to the PISA test in your school, have you done any of the following? I have shared information about the PISA test administration with the staff at my school.
SC209-Q02	With respect to the PISA test in your school, have you done any of the following? I have shared information about the PISA test administration with the students at my school.
SC209-Q03	With respect to the PISA test in your school, have you done any of the following? I have shared information about the PISA test administration with the parents or guardians of students at my school.
SC179	Approximately what percentage of students in your school repeated a grade at the following grade levels last academic year?
SC191	Is there state or district legislation on including students' parents or guardians in school activities?

Student Questionnaire

1. Items with revised wording or structure

2022 Field Test		2022Main Study
Item #	Field Test Item	Item#	Main Study Item
Intro1	In this questionnaire you will find questions about the following topics: - You, your family, and your home - How you think about your life - Your school - Your school schedule and learning time - Language learning in school - Your view on reading	Intro1	Note: The bullet point order has changed in MS In this questionnaire you will find questions about the following topics: - You, your family, and your home - Language learning in school - Your view on reading - How you think about your life - Your school - Your school schedule and learning time
ST003	On what date were you born?	ST003	On what date were you born?
01	1998	01	2000
02	1999	02	2001
03	2000	03	2002
04	2001	04	2003

2. Deleted student questionnaire items from field test

	2022 Field Test
ST038-Q1	During the past 12 months, how often have you had the following experiences in school? I got called names by other students.
ST227	Which of the following statements best describes your living situation?
ST251 –Q5	How many of these items are there at your home? Washers
ST252	How many digital devices with screens are there in your home?
ST257	How many of the following types of books are in your home?
ST228	Who usually lives at home with you?
ST229	Who usually lives at your homes with you?
ST231	Based on this definition, how many parents and/or guardians do you have?
ST232	Which of the following qualifications does your parent or guardian have?
ST233	Can your parent or guardian do the following?
ST242	Does your parent or guardian have a paid job?
ST332	The following two questions concern your parent or guardian's job:
ST243	Which of the following job categories best describes your parent or guardian's main job?
ST234	Which of the following qualifications do your parents or guardians have?
ST235	Can your parents or guardians do the following?
ST244	How many of your parents or guardians have a paid job?
ST248	The following questions concern your parents' or guardians' jobs.
ST245	Which of the following job categories best describe your parents' or guardians' main job(s)?
ST236	Think about the three parents or guardians who have spent the most time raising and taking care of you. Which of the following qualifications do they have?
ST237	Think about the three parents or guardians who have spent the most time raising and taking care of you. Can your parents or guardians do the following?
ST246	How many of your parents or guardians have a paid job?
ST249	Think about the three parents or guardians who have spent the most time raising and taking care of you. The following questions concern your parents’ or guardians’ jobs.
ST247	Think about the three parents or guardians who have spent the most time raising and taking care of you. Which of the following job categories best describe each of your parents' or guardians' main job?
ST238	Does your mother have a paid job?
ST239	Which of the following job categories best describes your mother's main job?
ST240	Does your father have a paid job?
ST241	Which of the following job categories best describes your father's main job?
ST269	This school year, how often have you felt the following ways while at school?
ST328	Which of the following job categories best describes the kind of job you expect to have when you are about 30 years old?
ST333	How many languages in total do you speak with people at home?
ST281	This school year, how often did your teacher do the following things in your mathematics lessons?
ST287	This school year, how often did your teacher do the following things in your mathematics lessons?
ST282	This school year, how often did your teacher do the following things in your mathematics lessons?
ST286	This school year, how often did your teacher do the following things in your mathematics lessons?
ST284	This school year, how often did your teacher do the following things in your mathematics lessons?
ST288	This school year, how often did your teacher do the following things in your mathematics lessons?
ST277	How often have you encountered this type of mathematics problem in the following settings?
ST279	How often have you encountered this type of mathematics problem in the following settings?
ST278	How many times have you encountered this type of mathematics problem in the following settings?
ST280	How many times have you encountered this type of mathematics problem in the following settings?
ST289-Q01	Thinking about mathematical concepts: how familiar are you with the following terms? Divisors
ST289-Q03	Thinking about mathematical concepts: how familiar are you with the following terms? Subjunctive Scaling
ST289-Q11	Thinking about mathematical concepts: how familiar are you with the following terms? Declarative Fraction
ST289-Q12	Thinking about mathematical concepts: how familiar are you with the following terms? Radicals
ST271	This school year, how often have you felt the following ways during your mathematics lessons?
ST263-Q01	To what extent do you agree or disagree with the following statements? With enough effort, anyone can increase their intelligence
ST263-Q03	To what extent do you agree or disagree with the following statements? With enough effort everyone can get good grades in mathematics.
ST263-Q05	To what extent do you agree or disagree with the following statements? With enough effort everyone can get good grades in English/Language Arts.
ST264	How much do you think each of the following can be changed?
ST274	How often do you use digital resources (e.g., computer, tablet, software programs, websites) for the following in your mathematics lessons or when completing homework?
ST297	This school year, which types of additional mathematics instruction or lessons do you participate in?
ST298	Why do you participate in additional mathematics instruction or lessons in this school year?
ST299	Have you studied any of the following topics during the additional mathematics instruction or lessons this school year?
ST325	To what extent do you agree or disagree with the following statements?

Student Financial Literacy Questionnaire

1. Items with revised wording or structure

2022 Field Test		2018 Main Study
Item #	Field Test Item	Item#	Main Study Item
	NA

2. Deleted items from field test

	2022 Field Test
FL164-Q17	Thinking about school lessons in the last 12 months, have you heard of, or learnt about, the following terms? Credit default swap
FL164-Q18	Thinking about school lessons in the last 12 months, have you heard of, or learnt about, the following terms? Call Option
FL171-Q06	In the last 12 months, how often have you done the following things? Made a payment using a non-bank card (e.g., a prepaid card to pay for school meals or transportation)
FL171-Q13	In the last 12 months, how often have you done the following things? Paid to download or use a mobile app
FL169-Q09	To what extent do you agree with the following statements? Pleasure and fun sometimes keep me from getting work done
FL169-Q12	To what extent do you agree with the following statements? It is harder to know how much I spent when I pay by cash than when I pay online.
FL172-Q02	To what extent do you agree or disagree with the following statements? I am comfortable telling my friends when I cannot afford to buy something.

ICT Questionnaire

1. Items with revised wording or structure

2. Deleted items from field test

	2022 Field Test
IC183-Q06	To what extent are you able to do the following tasks when using digital resources? Edit digital photos or other digital images
IC183-Q11	To what extent are you able to do the following tasks when using digital resources? Change the settings of a device or app to improve the way it operates

1 After the field test is complete and the data are evaluated, the collection of questions used during the field test, old and new, will be evaluated. The national sample questionnaire burden will not exceed 45 minutes.

2