MEMORANDUM OMB # 1850-0803 v. 221
DATE: November 21, 2017
TO: Kashka Kubzdela
National Center for Education Statistics
FROM: Lydia Malley
National Center for Education Statistics
SUBJECT: International Computer and Information Literacy Study (ICILS 2018) Pretest - Updated [revision to a previously approved package (1850-0803 v.207)]
The International Computer and Information Literacy Study (ICILS) is a computer-based international assessment of eighth-grade students’ computer and information literacy (CIL) skills. ICILS was first administered internationally in 2013 in 21 education systems and will be administered again in 2018. The United States will participate for the first time in the 2018 administration. U.S. participation in this study will provide data on students’ skills and experience using technology to investigate, create, and communicate, and will provide a comparison of U.S. student performance and technology access and use with those of the international peers. This study will also allow the U.S. to begin monitoring the progress of its students compared to that of other nations and to provide data on factors that may influence student computer and information literacy skills. The data collected through ICILS will provide valuable information with which to understand the nature and extent of the “digital divide” and has the potential to inform understanding of the relationship between technology skills and experience and student performance in other core subject areas. The request to conduct a real-world test of the IEA ICILS assessment systems with students, allowing the systems to be tested in the manner that will be used in the main study to help identify system issues during the software development process was approved in September 2017 (OMB# 1850-0803 v.207). This request is to replace the originally included field test version of the student questionnaire with the most current draft version of the U.S. adapted ICILS 2018 main study questionnaire, which includes the final international version of this questionnaire (the U.S. adaptations are currently in review by the IEA).
Since OMB approval of this study, we were required to delay the start of the pretesting from October to December 2017 due to delays in the IEA timeline for providing a pilot version of the IEA ICILS assessment system. We will receive the version of the IEA ICILS assessment system by December 2017. It will include the main study version of the student questionnaire instead of the field test version because the IEA will have programmed the questionnaire updates into the system for this delivery.
To reflect this change and the adjustments necessary to the study procedures and materials, we revised the following parts of Volume 1 and Volume 2 (the original, approved text is shown in red font, and the revised text in blue font). These revisions do not change the approved estimated response burden nor the cost to federal government for this study.
Revisions to Volume 1
Cover page:
Updated version number and added “revised November 2017”
Section 2: Background and Study Rationale
Original text:
The pretesting will be conducted with students from grade 8 and will be held in a simulated classroom after two preliminary/draft versions of the IEA ICILS assessment systems are produced. Two pretesting events will take place during fall 2017, with event sessions lasting approximately 175 minutes each.
Revised Text:
The pretesting will be conducted with students from grade 8 and will be held in a simulated classroom after one preliminary/draft version of the IEA ICILS assessment system is produced. Two pretesting events will take place during winter 2017-2018, with event sessions lasting approximately 175 minutes each.
Section 3: Recruitment and Data Collection:
Original Text:
Shortly after each of the two preliminary/draft IEA ICILS assessment versions are released, a pretesting event with students will be held in a simulated classroom. (…)
As part of the assessment administration in all event sessions, students will take a set of survey questions. The maximum time for the student questionnaire component is 30 minutes (included in the 175-minute time estimation for each session). Volume 2 includes the student survey items that were administered in the field test.2 As the items for the 2018 ICILS are finalized, the final questions will be included in the ICILS assessment system for pretesting. As such, session one will use the field test versions of the questionnaires and session two is likely to use the final questionnaires. If, as anticipated, questionnaire items for session two will differ from those that will be used in session one (as included here), we will submit to OMB for review a new clearance request under OMB# 1850-0803 (please note that the estimated required response burden for session two is included in this request; if additional burden will be needed at the time of session two request submission, we will explicitly request the additional burden, otherwise we will not ask for new burden amount).
Revised Text:
Shortly after the preliminary/draft IEA ICILS assessment version is released, two pretesting events with students will be held in a simulated classroom. (…)
As part of the assessment administration in all event sessions, students will take a set of survey questions. The maximum time for the student questionnaire component is 30 minutes (included in the 175-minute time estimation for each session). Volume 2 includes the student survey items that will be administered in both sessions of this pretest, and which are from the ICILS 2018 main study questionnaire. The international version of the 2018 ICILS student survey items has been finalized and their U.S. adaptations are in review by the IEA. If any changes are made to these items based on IEA’s review, we will submit a revised version of this request to OMB with the final item versions. No additional burden will be added.
Deleted Footnotes:
Draft content may be used in the earlier versions.
2 The final items will consist of those selected for ICILS 2018 administration (to be submitted for OMB review in fall 2017: OMB #1850-0929 v.6-7). Volume 2 provides the questionnaires approved as part of the 1850-0929 v.1-5 field test submittal.
Original Text:
The schedule for this study, including all activities, will begin in September 2017 through December 2017 as detailed in Table 3.
Table 3: Project Schedule
Activity |
Dates |
Recruitment, pretesting, data collection, analysis and final report |
September–December 2017 |
Revised Table of Total Hourly Burden
The schedule for this study, including all activities, will begin in September 2017 through January 2018 as detailed in Table 3.
Table 3: Project Schedule
Activity |
Dates |
Recruitment, pretesting, data collection, analysis and final report |
September 2017–January 2018 |
Revisions to Volume 2
Cover page:
Updated version number and added “revised November 2017”
Section 2: Background and Study Rationale (page 3)
Original text:
Students will take a set of survey questionnaire items during the ICILS pretest assessment session. For the first pretesting session the items previously administered as part of the ICILS field test will be used, and are included below. For the second pretesting session we anticipate using the ICILS 2018 items that will be finalized for the main study. An update to this request will be submitted to OMB when the final questionnaires are ready and prior to the second pretesting session.
Revised Text:
Students will take a set of survey questionnaire items during the ICILS pretest assessment session. For both sessions we will use the ICILS 2018 items that are finalized for the main study. Included here is the student questionnaire that will be used. In case any changes are made to the U.S. adaptations while the questionnaire is in review by the IEA, an update to this request will be submitted to OMB prior to these items being pretested.
Student Questionnaire (pages 4-25)
The field test version of the student questionnaire that was included in the original submission for this pretest (OMB# 1850-0803 v.207) was replaced with the draft version of the U.S. adapted ICILS 2018 main study questionnaire, which includes the final international version of this questionnaire (the U.S. adaptations are currently in review by the IEA).
Appendices
Cover page:
Updated version number and added “revised November 2017”
Section 2: Background and Study Rationale (page 3)
On pages 6 and 11, references to the timing of the pretesting sessions have been changed from “Sessions will be in October and December” to “Sessions will be in December and January”.
Page
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | O'Connell, James M |
| File Modified | 0000-00-00 |
| File Created | 2021-01-21 |