Cognitive Testing of ATUS Sleep Measures and Sleep Estimate Questions

Appendix A – Cognitive Interview Protocol

Part 1 - Introductory Material

• Hi! Thank you for coming in today.

• Have you participated in any of our studies before? (if yes, Which ones?)

• Study Introduction:

  • One of the surveys we conduct at BLS is the American Time Use Survey. In this survey, we call people and ask them about all the things they did the previous day.

  • Today we’ll be discussing different types of questions about how people spend their day?

  • What we are going to do today is go through a few questions from the survey and get your reactions to them. The purpose of this session is to help us find out more about how people respond to these questions. Basically, we’re trying to find out what you think a question is asking and how hard it is to answer. We are not here to evaluate you, we are looking to improve the questions, so there are no wrong answers. All the information you give us will be kept completely confidential, and will be used to improve the survey questions.

• Consent Form/Permission to audiotape

• Any questions before we begin?

Part 2 – ATUS Interview and Stylized Questions

The ATUS questions and stylized question order would be counterbalanced.

ATUS Style Interview

The ATUS interview will start at 4 a.m. and run through 10 a.m. or whenever the participant woke up. It will then skip to 7 p.m. and continue through the participant’s bedtime. Participants will not be asked where they were or who they were with.

Now I'd like to find out how you spent your time yesterday, <DAY & DATE>. If an activity is too personal, there's no need to mention it.

• Let’s begin with yesterday, <DAY>, at 4 a.m. What were you doing?

  • (if sleeping) What time did you wake up yesterday?

  • (if not sleeping) What did you do next?

• What did you do next?

  • How long did you spend <ACTIVITY>?

• What did you do next?

• …

• OK, now I would like to skip forward to last night at 7 p.m. What were you doing at 7 p.m.?

  • How long did you spend <ACTIVITY>?

• What did you do next?

  • How long did you spend <ACTIVITY>?

• *… (Continue until the participant mentions sleep)

• What time did you wake up?

• We skipped from about 10 a.m. to 7 p.m., during that time yesterday did you take any naps?

  • (if yes) What time did you fall asleep for your nap? What time did you wake up?

*Going/Went to bed (FROM ATUS TRAINING MANUAL)

• Ask the probe "What time did you fall asleep?"

Stylized Questions

The following sets of questions could be asked together (in their current order), only one set could be asked if we prefer one over the other, or half of the participants could receive one set, and another half could receive the other.

• How long do you sleep at night on an average weekday?

• Do you typically take naps?

  • (if yes) How long do your naps usually last?

• Thinking about the past week (between <DATE> and <DATE>), on average, how long have you slept each night?

• Did you take any naps during the last week?

  • (if yes) How long was your typical nap this week?

• Was the amount you slept during the last week typical of how you usually sleep?

  • (if no) In what way was your sleep this week unusual?

Part 3 – Debriefing

The debriefing will always be asked in this order, regardless of the order of the ATUS interview and stylized questions.

Thanks for going through those questions with me. We are going to switch gears now and talk a bit about what it was like for you to answer those questions. I have some follow-up questions that will help us better understand how people are responding to this survey. I want to emphasize that there are no ‘right’ or ‘wrong’ answers to these questions – we are just interested in your interpretations and reactions.

1. What were your reactions to these questions?

2. I asked you a number of questions about sleep. What does the word “sleep” mean to you?

3. What activities do you include as part of sleep?

   1. (if any non-sleep activities reported) Did you include any of these activities in your estimate of how long you slept?

Debriefing – ATUS

1. When we walked through your day yesterday, you said that you woke up at <WAKE TIME>. How did you arrive at your answer to this question?

   1. (if estimate) If I pressed you a bit, do you think you could remember the exact time you woke up yesterday?

   2. (if estimate) How precise do you think I wanted to you to be? Within a few minutes, within 5 minutes, within 10 minute, within 15 minutes, within a half hour, within an hour?

   3. (if exact) Can you describe how you know that is the exact time you woke up?

2. What time do you usually wake up?

   1. (if different than time reported for yesterday) Why was yesterday unusual?

3. You said that you went to sleep at <SLEEP TIME>. How did you arrive at your answer to this question?

   1. (if estimate) Can you remember the exact time you went to sleep yesterday?

   2. (if exact) How do you know that is the exact time you went to sleep?

   3. About how long did it take you to fall asleep last night?

4. Is <SLEEP TIME> when you usually go to sleep or was yesterday unusual?

   1. (if unusual) Why was yesterday unusual?

   2. (if unusual) Do you have a usual time that you go to sleep? When is that?

5. (if nap) I asked about naps and you said you slept for <NAP DURATION> yesterday. Is that time exact or an estimate?

   1. (if estimate) Do you remember the exact amount of time you napped?

   2. (if estimate) How precise do you think I wanted to you to be about naps?

6. Given the time you went to sleep last night and the time you woke up this morning, I calculate that you slept <SLEEP DURATION> last night. Does that sound correct?

7. Did you wake up during the night?

   1. (if yes) How many times?

   2. (if yes) Were you awake for more than 15 minutes?

8. Is it easy or difficult to remember what time you woke up yesterday? Why is that?

   1. response options provided on showcard: Very Easy, Somewhat Easy, Neither Easy Nor Difficult, Somewhat Difficult, Very Difficult

9. Is it easy or difficult to remember what time you fell asleep yesterday? Why is that?

   1. Response options provided: Very Easy, Somewhat Easy, Neither Easy Nor Difficult, Somewhat Difficult, Very Difficult

Debriefing – Stylized Questions

1. I asked you about how long you slept on the average night last week. How did you go about answering that question?

2. I also asked if you usually take naps and you said <ANSWER>. What does the word “usually” mean to you?

3. When was the last time you took a nap? How long did that nap last?

4. Is it easy or difficult to determine how long you sleep on an average weekday? Why is that?

   1. Very Easy, Somewhat Easy, Neither Easy Nor Difficult, Somewhat Difficult, Very Difficult

5. How certain are you that you slept <SLEEP DURATION> on average last week?

   1. Not at all certain, A little certain, Somewhat certain, Very certain, Extremely certain

Debriefing - Social Desirability

1. Is talking about how much you sleep a sensitive topic for you?

   1. (if yes) Why is that?

2. Were there any questions that I asked today that you thought were too personal?

3. Why do you think the government would ask questions about how much sleep people get?

4. Do you think there is an appropriate number of hours a person should sleep in one night?

   1. What is the minimum number of hours a person should sleep in one night?

   2. What is the maximum number of hours a person should sleep in one night?

5. Do you think it would be more embarrassing for someone to admit they have slept too much or slept too little? Why is that?

   1. If I told you that I am conducting a survey about employment and jobs, do you think it would be more embarrassing to admit having slept too much or too little?

   2. If I told you that I am conducting a survey about health, do you think it would be more embarrassing to admit having slept too much or too little?

6. Is there a time in the morning that you think everyone should be awake by?

7. Is there a time that you would be embarrassed to tell someone you slept until?

8. Is there a time at night you consider too early to be in bed? That is, is there a time that you would be embarrassed to tell someone you fell asleep for the night?

9. What is an appropriate length for a nap?

10. What would you consider too long of a time for a nap?

11. Would you be embarrassed to discuss naps you take during the day?

12. Do you discuss how much you sleep with friends and family?

    1. (if yes) Is the amount you sleep greater than, less than, or about the same as the people you talk to?

13. Do you think you sleep more than, less than, or about the same as the average American?