Response to OMB Passback

December 2, 2013

MEMORANDUM

To: Shelly Martinez, Kashka Kubzdela

From: Gail Mulligan, Jill McCarroll

Subject: Responses to Passback for Spring Third-Grade National Data Collection, Fourth-Grade Recruitment, and Fifth-Grade Tracking (OMB# 1850-0750 v.15)

This memo provides responses to the OMB passback sent on 11/26 and 11/27/2013.

1.       The child questionnaire contains a note on the cover that says: “A decision on the exact laptop that will be used to administer the child questionnaire is still pending. Therefore, some of the instructions provided to the child about how to use the computer to take the survey, for example the instructions about how to adjust the volume in item CLQ006, may need to be revised to be appropriate for the laptops that are selected.”  Please update us on this decision and the extent to which the choose is consistent with the testing.

NCES: The laptops selected for the study are very similar to the ones used in the pilot test. However, rather than using a stylus to select responses, the new laptops allow for children to select their responses using touch screens that are responsive to input from the child’s finger. In the pilot test, we saw that the stylus input was not always responsive; e.g., children had to forcibly push down on the screen with the stylus. The new touchscreens are quite responsive to light finger touch and we believe will be an improvement relative to the pilot test equipment.

The volume buttons are located in the lower right of the screen.

Here are the revised directions for using the touch screen and the volume buttons:

CLQ005

QUESTION TEXT:

PLUG HEADPHONES INTO THE COMPUTER. POSITION LAPTOP IN FRONT OF THE CHILD.

SAY: Now you are going to use the computer to tell us a little more about yourself. The computer will read

some sentences and questions to you while you listen on the headphones [HOLD UP HEADPHONES].

SAY: Then you will pick the answer that best describes you by touching your answer on the computer screen

[TOUCH THE MIDDLE OF THE COMPUTER SCREEN WITH ONE FINGER]. If you don’t want to pick an answer,

you don’t have to.

PRESS 1 AND ENTER TO CONTINUE.

CLQ006

QUESTION TEXT:

POINT TO THE VOLUME CONTROL BUTTONS, ABOVE THE STICKER, ON THE BOTTOM RIGHT CORNER OF THE

COMPUTER SCREEN.

SAY: You can change the volume by pressing either button above the sticker.

SAY: To make the sound louder, press the right button above the sticker. [POINT TO THE RIGHT BUTTON

ABOVE STICKER.] To make the sound softer, press the left button above the sticker. [POINT TO THE LEFT

BUTTON ABOVE STICKER.] If you want to turn the sound off, just turn the volume all the way down.

SHOW THE CHILD HOW TO ADJUST THE SIZE OF THE HEADPHONES.

SAY: Once you put the headphones on, you will hear directions about what to do. Do you have any questions?

[PAUSE 23

SECONDS AND ANSWER ANY QUESTIONS.]

SAY: Okay, go ahead and put these headphones on now. [HAND THE CHILD THE HEADPHONES.] Do the

headphones fit okay? [ADJUST HEADPHONES IF NECESSARY.]

[TURN LAPTOP AROUND. ROTATE LAPTOP SCREEN TO FACE THE CHILD.]

PRESS 1 AND ENTER TO CONTINUE.

2.       Perhaps because these are only programming specs and not screen shots, we not see any child assent or confidentiality pledge language.  Please provide this information, ideally as a screen shot.

NCES: Confidentiality language appropriate for third-graders is included in the child questionnaire text that is read to the child by the audio-CASI program (see below). This audio text informs the child that all of his/her answers are private and that he/she can skip questions he/she does not want to answer. There is also text on this in the item that is read to the child by the assessor before the audio-CASI program is initiated (item CLQ005). Additionally, all assessors are trained that any child who indicates he or she does not want to participate in the assessments should not be assessed.

CLQ005 (read by the assessor)

QUESTION TEXT:

PLUG HEADPHONES INTO THE COMPUTER. POSITION LAPTOP IN FRONT OF THE CHILD.

SAY: Now you are going to use the computer to tell us a little more about yourself. The computer will read

some sentences and questions to you while you listen on the headphones [HOLD UP HEADPHONES].

SAY: Then you will pick the answer that best describes you by touching your answer on the computer screen

[TOUCH THE MIDDLE OF THE COMPUTER SCREEN WITH ONE FINGER]. If you don’t want to pick an answer,

you don’t have to.

PRESS 1 AND ENTER TO CONTINUE.

CLQ012 (read by the audio-CASI)

QUESTION TEXT:

PROGRAMMER INSTRUCTIONS:

ENTER TEXT

Hello. Before we begin, let’s make sure the volume on the headphones is right for you. This is how loud it will

be when the sentences are read to you. If it is too loud or too soft please change the volume now or ask for

help.

In a minute, sentences will appear on the screen, and they will be read to you along with a list of answers.

Think about each sentence and decide how well it describes you. Once you've decided, touch your answer on

the computer screen. You can touch your answer even if the computer is still reading to you.

For these questions, there are no right or wrong answers, and all your answers are private. No one will be

told the answers that you pick. And remember, be sure your answers tell me about YOU.

Please touch the "NEXT" button to move to the next screen.

TURN OFF CONFIRMATION SCREEN.

CLQ028 (read by the audio-CASI)

QUESTION TEXT:

PROGRAMMER INSTRUCTIONS:

ENTER TEXT

Great, thank you for practicing with me.

One last thing before we begin: If there is a question that you do not want to answer, just touch the “NEXT”

button to skip over it. When you touch the “NEXT” button without answering, the computer will ask you if

you want to skip the question, just to make sure you didn’t skip the question by accident.

Please touch the "NEXT" button to move to the next screen.

TURN OFF CONFIRMATION SCREEN.

3.       Please confirm that the confidentiality pledge to the child protects his/her data from release even to his/her parents.

NCES: Yes, this is correct. We do not provide any information to parents about their child’s performance on the assessments or that we have collected from other school staff. The only information provided to parents is the results of the child’s hearing evaluation.

4.       We appreciate that the health insurance questions on the parent questionnaire have updated help text to acknowledge the new health exchanges, etc., but these are not the best questions to be using.  In particular, we know that individuals have difficultly differentiating their coverage being public or private given the increase in private plans managing Medicaid plans. NCHS and Census have better, validated questions that we are asking agencies to use when they collect such data. We will send you the best set to use for a telephone survey tomorrow, and ECLS-K should use those instead.

(Additional information from OMB:)

We believe that the best health insurance questions to use given the mode is from the ACS (see Page 9, item 16), available at:
http://www.census.gov/acs/www/Downloads/questionnaires/2013/Quest13.pdf

The ACS is designed to get a Yes/No response for each type which is good for a telephone survey. The advantage is that the types of info ECLS-K is providing in its “help text” is directly incorporated into the question so that everyone hears the same thing.  One of the categories will need to be adapted somewhat for a study of children (it doesn’t have CHIP but it does have Medicare).  The same drop down menus for state programs names could be used.  And of course “this person” would be changed to {child} to match the ED format.

NCES: Thank you for your suggested items. We agree that they would be clear for parents. However, while they would work fairly well for a telephone interview, there are revisions that would be needed to make them applicable in the context of the ECLS-K:2011 (as the passback notes), and asking about the different types of health insurance separately would add time to the survey. We have made a concerted effort to reduce the interview time for parents and do not feel that the items on health insurance are important enough to collect at this time to lengthen the interview.

The issue of asking parents about health insurance was raised in early November during the study’s Technical Review Panel (TRP) meeting. TRP members suggested that these items may be of interest in studying how many children acquire health care because of the Affordable Care Act. Changing the question wording would preclude using the ECLS-K:2011 for such an analysis, and we believe that there are other sources of data that are better suited for that analysis anyway. The TRP members did not express interest in measuring health insurance receipt in-and-of itself. Also, we have collected information on health insurance in three prior rounds of the ECLS-K:2011, so we already have information on this topic, even if we eliminate questions about health insurance from the third-grade parent interview.

For these reasons, rather than change the questions we ask at this time, we prefer to drop the items on health insurance for the current data collection (CHQ 40 a, b, and c) and give the use of the ACS questions and how to make them appropriate for the context of the ECLS-K:2011 more consideration for the next round of data collection. We will be pilot testing our fourth-grade parent interview using cognitive interviews (planned for early 2014; OMB generic clearance package forthcoming) and can include the ACS items, revised to be applicable to the ECLS-K:2011, during those interviews.

Page 3 of 3