Appendix E Appendix E NAEP 2019-2020 Feedback Forms

NATIONAL CENTER FOR EDUCATION STATISTICS

NATIONAL ASSESSMENT OF EDUCATIONAL PROGRESS

National Assessment of Education Progress (NAEP)

2019 and 2020

Appendix E

Assessment Feedback Forms

_{OMB# 1850-0928 v.11}

September 2018

No changes since v.10

Table of Contents

Appendix E1: NAEP 2019 School Coordinator Wrap-up Activities 2

Appendix E2: NAEP 2019 Telephone Quality Control Form 5

Appendix E1: NAEP 2019 School Coordinator Wrap-up Activities

Directions: Complete the following steps with the school coordinator before leaving the school on assessment day. Text shaded in grey are instructions to you. Do not read that text to the school coordinator. Estimated time to complete is 7 minutes.

	Note items to discuss with school coordinator:       Give the school coordinator a brief summary of how the assessment activities went in the school. Cover: Positive feedback Concerns or issues
	Step 2. Ask for feedback. Do you have any questions, concerns, or feedback about the assessment in the school, including preassessment activities? Record comments/issues raised by the school coordinator:

SCHOOL COORDINATOR WRAP-UP ACTIVITIES (CONTINUED)

Step 3. Schedule makeup session(s), if necessary.

If a makeup session is needed for DBA, P&P, or both, arrange time(s) and location(s) for the session(s).

Record comments/issues raised by the school coordinator

 

 

 

Step 4. Return NAEP storage envelope.

I have placed copies of all documents used in the assessment in the NAEP Storage Envelope.

IF GRADE 4 OR 8: NAEP would like you to retain the envelope until May 1 or the end of the school year in case there are questions about the assessment.

IF GRADE 12: NAEP would like you to retain the envelope until December 31, 2019.

You will receive an email in the spring as a reminder to destroy the contents of the envelope at that time.

• Give back the NAEP storage envelope.

Step 5. Give school certificate of appreciation.

Thank you for everything you have done to make this a successful assessment.

I have a Certificate of Appreciation I would like to give the school as a token of our appreciation for your participation.

• Give the school coordinator the school certificate of appreciation.

Record comments/issues raised by the school coordinator:

Step 6. Offer ancillaries.

Offer the used ancillaries from the session(s), as applicable.

Ancillaries the School Can Keep
Grade	Ancillary
4, 8, 12	HOT kit components as listed (refer to Conducting Science Sessions chapter for specific kit lists, if applicable)

Appendix E2: NAEP 2019 Telephone Quality Control Form

Introduction TO the School Coordinator

Hello, this is ______ from the National Assessment of Educational Progress. I work with the NAEP assessment coordinator for this area. A few days ago, our assessment team conducted (a/# _______) session(s) at your school. NAEP would like to thank you for all of your work in making the assessment at _________________ (SCHOOL NAME) possible. As part of our efforts to improve the performance of our assessment teams, we routinely call a random sample of school coordinators to determine how well the assessment coordinator performed tasks related to the telephone review of the preassessment tasks you completed in MyNAEP. We are also interested in the performance of the NAEP team on assessment day. This call should only take 10 minutes.

Just as a reminder, the assessment team at your school was led by _________________________ (AC NAME). When responding to these questions, please feel free to speak frankly. Your comments will help improve the quality of the assessments.

Questions about the INITIAL CALL

First, I have a few questions about the initial call that ______________ (SV/AC NAME) conducted with you on _______________ (INITIAL CALL DATE). As a reminder, during that call you may have discussed MyNAEP registration, been introduced to brochures about using the site, walked through logging in, seen the Prepare for Assessment screens, and scheduled a preassessment review call.

1. Overall, how would you rate the NAEP representative’s handling of this initial call?

Would you say it was very good, satisfactory, or unsatisfactory?

VERY GOOD 1 (Q2)

SATISFACTORY 2 (Q2)

UNSATISFACTORY 3 (Q1a)

1a. IF UNSATISFACTORY: Why do you feel the NAEP representative’s handling of the initial call was unsatisfactory? (PROBE FOR SPECIFICS)

Questions about the Preassessment REVIEW CALL

Now I have a few questions about the preassessment review call that ______________ (SV/AC NAME) conducted with you on _______________ (PREASSESSMENT REVIEW CALL DATE). As a reminder, during this call you may have reviewed the list of students selected for NAEP, completed the SD or ELL student information, reviewed the procedures to notify parents, reviewed procedures for distributing and monitoring questionnaires, identified any newly enrolled students, planned assessment day logistics, and covered participation and other support activities.

2. Overall, how would you rate the NAEP representative’s handling of the preassessment review call?

Would you say it was very good, satisfactory, or unsatisfactory?

VERY GOOD 1 (Q3)

SATISFACTORY 2 (Q3)

UNSATISFACTORY 3 (Q2a)

2a. IF UNSATISFACTORY: Why do you feel the NAEP representative’s handling of the preassessment review call was unsatisfactory? (PROBE FOR SPECIFICS)

3. Overall, how would you rate the NAEP representative on leading the review and update of student demographic information on MyNAEP during your call?

Would you say the review process was very good, satisfactory, or unsatisfactory?

VERY GOOD 1 (Q4)

SATISFACTORY 2 (Q4)

UNSATISFACTORY 3 (Q3a)

3a. IF UNSATISFACTORY: Why do you feel the process of reviewing and updating the demographic information was unsatisfactory? (PROBE FOR SPECIFICS)

Questions about the Preassessment REVIEW CALL (CONTINUED)

4. Overall, how would you rate the NAEP representative on leading the review of accommodations for students with disabilities and English language learners? Would you say the process of reviewing accommodations was very good, satisfactory, or unsatisfactory?

VERY GOOD 1 (Q5)

SATISFACTORY 2 (Q5)

UNSATISFACTORY 3 (Q4a)

4a. IF UNSATISFACTORY: Why do you feel the review was unsatisfactory? (PROBE FOR SPECIFICS)

 

5. Do you have any other comments or suggestions about the NAEP representative’s review of the preassessment activities you conducted on MyNAEP?

Questions about the Assessment

Now I have a few questions about the assessment that was conducted at your school

on ___________________________ (ASSESSMENT DATE).

6. Did the team arrive on time?

YES 1 (Q7)

NO 2 (Q6a)

DON’T KNOW 8 (Q7)

6a. IF NO, RECORD DETAILS ABOUT THE LATE ARRIVAL.

(PROBE: Could you tell me more about that or why they were late?)

Questions about the Assessment (CONTINUED)

7. As far as you know, did (all of) the session(s) start on time?

YES 1 (Q8)

NO (SOME OR ALL WERE DELAYED) 2 (Q7a and Q7b)

DON’T KNOW 8 (Q8)

7a. IF NO, RECORD ANY DETAILS OFFERED ABOUT THE DELAY.

(PROBE: Could you tell me more about why the session(s) was/were delayed?)

7b. DELAY WAS CAUSED BY (CODE ONE):

TEAM 1

OTHER REASON 2

COULD NOT DETERMINE 8

8. Did you have the opportunity to observe (any of) the assessment session(s)?

YES 1 (Q8a)

NO 2 (Q9)

8a. IF YES: How would you rate the way the NAEP representative(s) handled the session(s) you observed? Would you say very well, adequately, or poorly?

VERY WELL 1 (Q9)

ADEQUATELY 2 (Q8b)

POORLY 3 (Q8b)

DON’T KNOW 8 (Q9)

8b. IF ADEQUATELY OR POORLY, RECORD ANY DETAILS MENTIONED.

(PROBE: Could you tell me more about that or how the NAEP representative handled the sessions?)

Questions about the Assessment (CONTINUED)

9. How would you rate the overall organization of the assessment team?

Would you say it was very good, adequate, or poor?

VERY GOOD 1 (Q10)

ADEQUATE 2 (Q9a)

POOR 3 (Q9a)

DON’T KNOW 8 (Q10)

9a. IF ADEQUATE OR POOR, RECORD ANY DETAILS MENTIONED.

(PROBE: Could you tell me more about the team’s organization?)

10. How would you rate the overall professionalism of the assessment team?

Would you say it was very good, adequate, or poor?

VERY GOOD 1 (Q11)

ADEQUATE 2 (Q10a)

POOR 3 (Q10a)

DON’T KNOW 8 (Q11)

10a. IF ADEQUATE OR POOR, RECORD ANY DETAILS MENTIONED.

(PROBE: Could you tell me more about the team’s professionalism?)

11. How would you rate the way the assessment team interacted with the other school staff?

Would you say the interaction was very good, adequate, or poor?

VERY GOOD 1 (Q12)

ADEQUATE 2 (Q11a)

POOR 3 (Q11a)

DON’T KNOW 8 (Q12)

11a. IF ADEQUATE OR POOR, RECORD ANY DETAILS MENTIONED.

(PROBE: Could you tell me more about the team’s interaction with other school staff?)

Questions about the Assessment (CONTINUED)

12. How would you rate the way the assessment team interacted with the students?

Would you say the interaction was very good, adequate, or poor?

VERY GOOD 1 (Q13)

ADEQUATE 2 (Q12a)

POOR 3 (Q12a)

DON’T KNOW 8 (Q13)

12a. IF ADEQUATE OR POOR, RECORD ANY DETAILS MENTIONED.

(PROBE: Could you tell me more about the team’s interaction with students?)

13. At the end of the session(s), did ________________ (AC NAME) give you the red NAEP Storage Envelope and tell you when the materials should be destroyed? (REVIEW IF NECESSARY).

YES 1

NO 2

DON’T KNOW…………………………………….. 8

14. Think about all of the contacts you have had with NAEP staff and the activities that took place during the preassessment review call/visit and on assessment day. Overall, how would you rate the NAEP team’s handling of the assessment at your school?

Would you say it was excellent, good, satisfactory, unsatisfactory, unacceptable, not enough information to rate, or don’t know?

EXCELLENT 1 (Q15)

GOOD 2 (Q15)

SATISFACTORY 3 (Q15)

UNSATISFACTORY 4 (Q14a)

UNACCEPTABLE 5 (Q14a)

NOT ENOUGH INFORMATION TO RATE 8 (Q15)

DON’T KNOW 8 (Q15)

14a. IF UNSATISFACTORY OR UNACCEPTABLE: Why do you feel the assessment team’s handling of the assessment was unsatisfactory?

(PROBE FOR SPECIFICS)

Questions about the Assessment (CONTINUED)

15. Is there anything (else) you would like to say about the assessment team or any of its members?

16. NAEP is conducted regularly, and we continue to refine the materials and procedures for future assessments. Do you have any suggestions as to how we can make it easier for schools to participate in the future?

These are all of the questions I have. On behalf of NAEP, I want to thank you again for all of your help.