NAEP Updated Memo

NAEP SAIL VirtualWorld for ELA 2014-17 Updated Memo.docx

NCES Cognitive, Pilot, and Field Test Studies System

NAEP Updated Memo

OMB: 1850-0803

Document [docx]
Download: docx | pdf

MEMORANDUM OMB # 1850-0803 v. 170

DATE: September 9, 2016

TO: Kashka Kubzdela

National Center for Education Statistics


FROM: Patricia Etienne

National Center for Education Statistics


SUBJECT: National Assessment of Educational Progress (NAEP) Survey Assessments Innovations Lab (SAIL) Pretesting Activities: Virtual World for English Language Arts Assessment – Updated [Revision to a previously approved package (1850-0803 v.106)]



The SAIL Pretesting Activities: Virtual Worlds for English Language Arts Assessment was approved on July 17, 2014 (OMB #1850-0803, v. 106). After completing the playtesting phase of pretesting it has been determined that extended time is needed for the cognitive interview sessions to allow enough time for the think aloud process to take place.

Originally the cognitive interviews were intended to be completed during a 90 minute session. The revised plan increases the cognitive interview session time to 120 minutes. The additional 30 minutes of testing time with student cognitive interview participants is being requested to allow additional time to implement the think-aloud methodology during the cognitive interview sessions (i.e., time for training students on the method and additional time to complete the task while intermittently thinking aloud). In playtesting sessions (without the think aloud method), students have spent on average 75-80 minutes just completing the task, which led us to estimate that more time will be needed for the cognitive interviews. Given that we are increasing the estimated time to complete a cognitive interview by over 33%, we are also increasing the per student incentive for cognitive interviews from $25 to $30 to attract students to participate in a 2 hour session outside of school.

Finally, we are also adjusting the overall schedule for all activities approved for this study to accommodate unanticipated delays in the development of the ELA Virtual World tasks.

Volumes I and II, and Appendices A-F have been revised to reflect these changes. Following are the original and revised burden and cost tables, as well as the revised schedule.

Original Estimates of Hourly Burden for SAIL Pretesting Activities: Virtual Worlds ELA

Estimate of Hourly Burden for Cognitive Interviews

Respondent

Hours per respondent

Number of respondents

Total burden hours

Student Recruitment via School Administrators and Staff and Club and or Community Center Administrators

Initial contact with staff: e-mail, flyer distribution, and planning

0.33

9

3

Parent or Legal Guardian

Flyer review

0.08

180

15

Completion of online screening form or phone screening

0.15

30*

5

Consent form completion and return

0.13

15**

2

Confirmation to parent via email or letter

0.05

30*

2

Recruitment Totals


189

27

Student

Virtual World for ELA

1.5

15

23

Interview Totals


15

23

Total Burden

279 responses

204

50

* Subset of initial contact group ** Subset of students with completed screening forms



Combined Burden for SAIL Virtual World Research Activities


Number of respondents

Number of responses

Burden Hours

Pretest Activity




Total Play Testing

340

465

81

Total Cognitive Interviews

204

279

50

Total Tryouts

421

423

359

Overall Totals

965

1,167

490



Revised Estimates of Hourly Burden for SAIL Pretesting Activities: Virtual Worlds ELA

Estimate of Hourly Burden for Cognitive Interviews

Respondent

Hours per respondent

Number of respondents

Total burden hours

Student Recruitment via School Administrators and Staff and Club and or Community Center Administrators

Initial contact with staff: e-mail, flyer distribution, and planning

0.33

9

3

Parent or Legal Guardian

Flyer review

0.08

180

15

Completion of online screening form or phone screening

0.15

30*

5

Consent form completion and return

0.13

15**

2

Confirmation to parent via email or letter

0.05

30*

2

Recruitment Totals


189

27

Student

Virtual World for ELA

2

15

30

Interview Totals


15

30

Total Burden

279 responses

204

57

* Subset of initial contact group ** Subset of students with completed screening forms


Combined Burden for SAIL Virtual World Research Activities


Number of respondents

Number of responses

Burden Hours

Pretest Activity




Total Play Testing

340

465

81

Total Cognitive Interviews

204

279

57

Total Tryouts

421

423

359

Overall Totals

965

1,167

497


Costs to Federal Government:

Original Cost: $489,529

Revised Cost: $489,604

Revised Schedule Milestones for SAIL Pretesting Activities: Virtual Worlds ELA

High-Level Schedule of Milestones

Activity

Dates

Play-Testing

Nov 2014 – August 2016

Cognitive Labs

October 2016 – December 2016

Tryouts

December 2016 – May 2017



File Typeapplication/vnd.openxmlformats-officedocument.wordprocessingml.document
AuthorO'Connell, James M
File Modified0000-00-00
File Created2021-01-22

© 2024 OMB.report | Privacy Policy