Intro pages

National Center for Education Statistics

National Assessment of Educational Progress

Volume II

Questionnaires

National Assessment of Educational Progress (NAEP)

2019 eNAEP Pretesting

OMB# 1850-0803 v.233

June 2018

Please note that the eNAEP system that will be used in this study displays the Confidential Information Protection and Statistical Efficiency Act of 2002 (CIPSEA) Title V, Subtitle A, Public Law 107-347 confidentiality pledge citation (along with a citation of the Cybersecurity Enhancement Act of 2015: 6 U.S.C. §151) because the eNAEP system is used for the main NAEP data collection, and the confidentiality of the operational NAEP assessment is governed by CIPSEA and 6 U.S.C. §151. However, confidentiality of the pretesting study described in this submission is governed by the Education Sciences Reform Act of 2002 (ESRA 2002, 20 U.S.C. §9573) and 6 U.S.C. §151. All participants that will use the eNAEP system for this study will be provided the correct citation on their consent forms and login cards (see below).

Paperwork Burden Statement (to appear on the back of student log-in cards)

Appropriate sections of the Paperwork Reduction Act and the NCES authorization and confidentiality statement indicated below are included in the consent forms and letters, and will be included on the eNAEP login cards.

Paperwork Burden Statement, OMB Information

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB control number. The valid OMB control number for this voluntary information collection is 1850-0803. The time required to complete this information collection is estimated to be an average of 100 minutes, including the time to review instructions and participate in the information collection. If you have any comments concerning the accuracy of the time estimate, suggestions for improving this collection, or any other concerns, please write to: National Assessment of Educational Progress (NAEP), National Center for Education Statistics, Potomac Center Plaza, 550 12^th St., SW, 4th floor, Washington, DC 20202.

Authorization and Confidentiality Assurance

National Center for Education Statistics (NCES) is authorized to conduct NAEP by the National Assessment of Educational Progress Authorization Act (20 U.S.C. §9622). All of the information you provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151).

OMB No. 1850-0803 Approval Expires 07/31/2019

Sample Student Login Card - front:

eNAEP Login Screen

Student Debriefing Questions

After all students have finished the assessment in each session, field staff will conduct a debriefing activity with the students. Field staff will ask the debriefing questions out loud to the group of students. The questions may be paraphrased, and additional follow-up questions, such as those indicated in the bulleted lists below, may be asked. Students will respond by raising their hands and by calling out responses, as appropriate. Field staff will use their discretion about whether some or all of the questions will be asked, depending on the amount of prompting the students require, and on the amount of time remaining. The debriefing session should not exceed 10 minutes. Field staff will write down the students’ responses to the questions.

1. What did you like the most about the assessment?

[Follow-up questions may include:]

• Can you explain that more?

• What did you like the least about the assessment?

• What was missing from the assessment?

• If you had to take a test, would you rather take it on a tablet, or on paper? Why?

• What do you like most about taking a test on a tablet? What do you like the least about taking a test on a tablet?

2. Did you see anything you thought was strange while taking the assessment?

[Follow-up questions may include:]

• Can you explain that more?

• Did questions or images seem slow to load at all to you?

• Was the system slow to move from one page/question to the next at all?

• Did you notice any “Aw, snap” errors?

• Were you able to make these errors occur when you wanted to? How?

3. Did you find the system easy to use and/or easy to learn to use?

[Follow-up questions may include:]

• How did you like the tutorial?

• Did you use the Help screens? Were the Help screens useful?

• What did you think of the tools available? Did you use them?

• Which tool or tools did you like the most? Why?

• Did you have any difficulty using any of the tools? What did you find difficult?

• How did you find the stylus worked with the assessment?

• Did you have any difficulty picking an answer using the available controls? If so, what made it difficult?

4. Is there anything you think would make the assessment better? If so, what?

5. Is there anything else you want to tell us about kids taking assessments on tablets?

Questionnaires

As part of the assessment administration, students will take a set of survey questionnaire items during their session. These items were previously administered as part of other NAEP assessments or testing. Given the final items for 2019 NAEP administrations have not yet been determined, we have included a library of previously administered components from which the 2019 items will be selected. The table below indicates the assessment components that form this library. Following the table are the components of survey questionnaire items. Not all of the items presented will be administered in 2019. The number of items administered to each student will be appropriate to the 15 minutes allocated. As the items are finalized throughout the development process, the final subset will be included in the eNAEP system for pretesting. As such, the earlier builds may include different items from the library than the final build.

Abbreviation Key

CAFS: Computer Access and Familiarity Study

NIES: National Indian Education Study

Questionnaire Components
Grade/Subject	Part	Survey Questionnaire
Grade 4 Core	II-a	2018 Pilot Grade 4 Core
	II-b	2018 Cognitive Interviews Grade 4 Core
Grade 8 Core	II-c	2018 Operational Grade 8 Core
	II-d	2018 Cognitive Interviews Grade 8 Core
Grade 12 Core	II-e	2018 Pilot Grade 12 Core
	II-f	2018 Cognitive Interviews Grade 12 Core
Grade 4 Reading	II-g	2017 Operational Grade 4 Reading
	II-h	2018 Cognitive Interviews Grade 4 Reading
Grade 8 Reading	II-i	2017 Operational Grade 8 Reading
	II-j	2018 Cognitive Interviews Grade 8 Reading
Grade 12 Reading	II-k	2018 Pilot Grade 12 Reading
Grade 4 Mathematics	II-l	2017 Operational Grade 4 Mathematics
	II-m	2018 Cognitive Interviews Grade 4 Mathematics
Grade 8 Mathematics	II-n	2017 Operational Grade 8 Mathematics
	II-o	2018 Cognitive Interviews Grade 8 Mathematics
Grade 12 Mathematics	II-p	2018 Pilot Grade 12 Mathematics
Grade 4 Science	II-q	2018 Pilot Grade 4 Science
Grade 8 Science	II-r	2018 Pilot Grade 8 Science
Grade 12 Science	II-s	2018 Pilot Grade 12 Science
Grade 4 NIES	II-t	2015 Operational Grade 4 NIES
Grade 8 NIES	II-u	2015 Operational Grade 8 NIES
Grade 4 CAFS	II-v	2017 Operational Grade 4 CAFS
Grade 8 CAFS	II-w	2017 Operational Grade 8 CAFS
Grade 12 CAFS	II-x	2015 Operational Grade 12 CAFS
Grade 4 and 8 SES-SQ	II-y	2018 Cognitive Interviews Grade 4 & 8 Household Composition