Note to Reviewer

May 9, 2016

NOTE TO THE REVIEWER OF:	OMB CLEARANCE 1220-0141 “Cognitive and Psychological Research”
FROM:	Brandon Kopp Research Psychologist Office of Survey Methods Research
SUBJECT:	Submission of Materials for the American Time Use Survey Sleep Measure Validation Study

Please accept the enclosed materials for approval under the OMB clearance package 1220-0141 “Cognitive and Psychological Research.” In accordance with our agreement with OMB, we are submitting a brief description of the study.

The total estimated respondent burden hours for this study are 77 hours.

If there are any questions regarding this project, please contact Brandon Kopp at

202-691-7514 or Robin Kaplan 202-691-7383.

1. Introduction and Purpose

The American Time Use Survey (ATUS) uses a time diary format to ask respondents about the activities they performed on the prior day. In 2014, the ATUS reported that members of the civilian, non-institutionalized population in the United States spend an average of 8.8 hours per day sleeping, and that number has remained similar over time. ATUS estimates of sleep are consistently higher than surveys that ask respondents directly about the amount of sleep they get (referred to in this protocol as stylized questions). The National Sleep Foundation (NSF), for example, asks respondents to estimate the amount of time they spend sleeping on an average weekday and weekend day. These numbers are then combined into an overall estimate. The NSF reports that American adults get an average of 6.9 hours of sleep per day. Other national surveys that use stylized questions about sleep, such as The National Health Interview Survey (NHIS), the Behavioral Risk Factor Surveillance System (BRFSS), and the National Health and Nutrition Examination Survey (NHANES) report sleep duration estimates ranging from 6.9 to 7.1 hours.

Diary Measures

While the estimates these two types of measures produce differ by quite a bit, it is likely that both have their sources of measurement error and neither is exactly correct. Since diary measures are not collected contemporaneously in the ATUS, memory may be a factor that pushes the estimate upward. Specifically, respondents may forget about times they woke up during the night or may misremember wake and sleep times. Respondents in time diary studies may also include more behaviors as part of sleeping than do respondents to stylized questions. That is, in a time diary, a respondent may report the time they “went to bed” or “got out of bed” which will differ from the time they actually fell asleep or woke up. For example, if a respondent reports “going/went to bed,” then the ATUS interviewer must ask him or her what time he or she actually fell asleep. If it took 30 minutes or less for the respondent to fall asleep, then the interviewer categorizes the entire duration of time as sleeping. If it took more than 30 minutes for the respondent to fall asleep, then the interviewer must probe in order to determine if the respondent was doing anything else. If the respondent were to say yes and report an activity, then the activity would be recorded. But, if the respondent were to say no, then the time would be logged in the time diary as time spent sleeping. If a respondent reports sleeping for 10 hours or more at one time, then the interviewers ask, “I have calculated that you slept [duration]. Is that correct?” in order to verify that the respondent meant to report that duration. This broad definition of sleep, coupled with the 30 minute rule of coding sleep, may lead ATUS sleep estimates to be overestimated in some instances.

In addition, the ATUS has broad categories for sleep. For instance, under the main category of personal care, includes three sub-categories: sleeping, sleeplessness, and sleeping not elsewhere classified. Sleeping includes activities such as sleeping, falling asleep, dozing off, napping, getting up, waking up, dreaming, cat napping, getting some shut-eye, and dozing; sleeplessness includes insomnia, tossing and turning, lying awake, and counting sheep. This broad definition of sleep may lead to longer average sleep durations in the ATUS.

Stylized Questions

Stylized sleep questions have not been validated in a general population (Lauderdale, 2015), and as mentioned, tend to be significantly lower than Diary measures. Indeed, studies comparing Diary and Stylized estimates across a wide variety of constructs (e.g., religious attendance, hours of paid vs. unpaid work, and performing household chores) have consistently shown that Stylized questions lead to overreporting of socially desirable behavior and underreporting of socially undesirable behavior (e.g., Brenner, 2011; Bonke, 2005; Kan & Pudney, 2007). Stylized questions make the construct of interest highly salient to participants, evoking self-presentation and self-identity concerns. This may lead people to overreport behaviors that portray them in a positive light. To the extent that a person’s self-worth is tied to staying busy and productive even at the expense of sleep, Stylized questions may lead to underreporting of sleep. Diary estimates do not make any particular construct more salient than others, and thus do not face this same systematic bias. Consequently, Diary estimates may result in more accurate and truthful measures of socially desirable behavior.

Stylized questions could also suffer from measurement error that drives estimates derived from these questions in a downward direction. When formulating an ‘average’ or ‘usual’ level of sleep, the respondent may recall salient episodes in which they got little sleep. To the extent these types of situations are more memorable than situations that fall within a person’s normal routine or situations where they slept longer than usual, these memories will figure more heavily into their estimate. Respondents to stylized questions may engage in a biased memory search or revise their answer in order to appear more desirable to whomever is asking them the question.

Objective Sleep Measures

In addition to diary and stylized measures, some researchers have employed objective measures of sleep, such as wrist actigraphy. This is a device people can wear while asleep that records sleep duration, sleep latency, and sleep efficiency. The results from these studies are mixed in terms of their relation to Diary versus Stylized estimates. Some studies have found objective sleep duration estimates that are lower than both Diary and Stylized estimates, around 6 hours (e.g., Lauderdale et al., 2006). Others have found that subjective sleep duration (typically measured with stylized, self-report estimates) are overestimated compared to objective sleep duration (e.g., Lauderdale et al., 2008; Tremaine et al., 2010). Conversely, others have found the opposite pattern – that subjective sleep duration is underestimated compared to objective measures duration (e.g., Silva et al., 2007). Reasons for these discrepancies are unclear, but some hypotheses are that people under appreciate night-awakenings when self-reporting their amount of sleep, aren’t always sure how long it took to fall asleep, or sleep was disrupted due to the actigraphy device. Diary and stylized measures are also prone to memory and judgment biases as described earlier.

A recent study by Arora et al. (2013) compared participants’ self-reported sleep duration using a stylized question, a sleep diary, and objective sleep actigraphy data. They found that the diary and actigraphy data were more similar to one another than the Stylized estimate. The authors hypothesized that the Stylized estimate was less accurate because it required participants generalize across the specified time period to produce an estimate, which may be subject to multiple biases. A sleep diary, however, may provide a more accurate representation if it is completed daily and within short proximity to waking.

The Present Study

The Office of Survey Methods Research (OSMR) at BLS conducted a cognitive interview study to help understand the measurement properties of both the ATUS interview as it relates to sleep and of a sample of stylized questions about sleep. In that study, we found that diary and stylized measures differ from one another, consistent with prior research (e.g., Kan & Pudney, 2007). In particular, we found that the diary sleep estimates were greater than the stylized sleep estimates. In addition, we found a number of differences across participants in how they answered questions about sleep in general and also across the diary versus stylized questions at each stage of the response process (as defined by Tourangeau, 1984).

The main findings included differences in comprehension – participants with a broad definition of sleep also reported getting more sleep than those with a narrower conception of sleep. Participants relied more on general estimation cues than memory to answer the stylized questions about sleep versus the diary questions. We also found evidence that stylized questions may be more susceptible to social desirability concerns than the Diary measure, that is, social desirability concerns about getting too much or too little sleep seemed to affect how participants answered the stylized questions about sleep, but not the diary measure. Participants also indicated that if they thought the survey was about jobs and employment, it would be more embarrassing to report oversleeping. In contrast, if a survey was about health, it would be more embarrassing to report undersleeping. These findings indicate that the context of the survey may also impact estimates of sleep duration. Since surveys across various agencies are framed differently, this may affect the sleep duration estimates obtained.

In 2016, OSMR conducted an online study with a large convenience sample of nearly 1,300 participants. In this study, several of the issues uncovered by the cognitive testing were explored by modifying the procedures for the time use diary and stylized questions for different subsets of the sample. Half of the participants received time diary and stylized measures of sleep after having received a definition of what is considered sleep, while half received no definition (which matches the current procedures in the ATUS). One-third of participants were told that the survey they were taking was about jobs, one-third were told the survey was about health, and another third were told nothing. These two types of variation; providing definitions (or not) and the context of the survey were crossed so that one-sixth of the total sample fell into each of the six combinations.

While the cognitive interview and online studies provided information about the relative differences between time use diary and stylized question measures of sleep, they did not definitively show whether diary or stylized measures were more accurate. The goal of this study is to explore the accuracy of individual diary and stylized measures of sleep by comparing them to a standard that does not require self-reports from study participants; a wearable accelerometer.

Wearable devices like the Fitbit have sensors that measure the angle and movement of the device. The device then interprets these measurements as physical actions (e.g., walking or running). The devices also measure the absence of movement. That is, if the sensors do not detect movement (or detect only subtle movements), this is interpreted as sleep. Wearable devices are light and unobtrusive so can be worn at all times, including while sleeping. Researchers have recently suggested that the popularity of such devices may facilitate conducting larger-scale studies that include diary, stylized, and objective measures of sleep (e.g., Miller et al., 2015).

2. Research Design

Participants in this study will be asked to visit the OSMR lab twice. On the first visit, they will be given a Fitbit watch and be told how to care for it. They will also answer some general questions about their typical routine and daily activities. They will be asked to wear the watch continuously for one week. At the end of that week, they will return to the OSMR lab where they will be debriefed about their experience, asked diary and stylized questions to measure sleep duration, and then will be asked follow-up questions. Finally, the participant will be asked to compare and contrast their self-reported sleep duration with the times measured by the Fitbit. The interview protocol is explained in greater depth below (see Appendix A for a full description).

Visit 1. At the first visit, participants will be introduced to the Fitbit watch. The interviewer will explain the function of the watch as well as how to care for it during the week-long wearing period. The interviewer will also explain that the participant should wear the watch at all times, except in situations where the watch may be submerged in water or otherwise damaged. The interviewer will explicitly state that the participant should wear the device at night when sleeping. The pamphlet in Appendix B will also be given to participants to reiterate what was discussed in the initial meeting. The interviewer will ask the participant some general questions about his or her typical routine and daily questions.

Finally, the interviewer will answer any questions the participant has and, if a second visit has not been scheduled, one will be made at that time.

The interviewer will call the participant approximately 3 days after Visit 1 to see if the he or she has any questions about the study and confirm the follow-up appointment.

Visit 2. The participant will return to the lab between 6 and 10 days after Visit 1, bringing the watch with them. At the beginning of the second interview, a research assistant will retrieve the watch from the participant and begin downloading and organizing the data for the interviewer.

While that is going on, the interviewer will debrief the participant about the experience of wearing the watch. Specifically, the participant will be asked how often they wore it, under what circumstances they removed it, and whether they reviewed any of the data from it.

Following the debriefing about their experience, participants will be asked to complete an abbreviated ATUS daily recall interview and answer a set of stylized questions regarding how much they sleep on an average weekday (NOTE: participants will be brought into the lab on Tuesday – Friday so their ATUS recall of the prior day’s activities will always cover a weekday). The abbreviated ATUS interview will ask participants to report their daily activity for the prior day focusing on times in which they are likely to have woken up and went to sleep (i.e., 4 a.m. – 10 a.m. and 7 p.m. – 3:59 a.m.).

The order of the ATUS interview and the stylized questions will be counterbalanced such that half of the participants will be asked the ATUS interview questions followed by the stylized questions and the other half will answer the stylized questions followed by the ATUS interview questions. Participants will be asked to think aloud while coming up with their answers to the survey questions.

Following the administration of the survey questions, the participant will be asked a series of follow-up questions. The goal of these questions is to understand how participants recall and/or estimate answers to both the ATUS interview and the stylized questions. Participants will be asked a series of questions to understand whether social desirability concerns may have caused them to modify their answers.

Finally, the interviewer will ask the participant to compare and contrast the self-report answers given with the data downloaded from the Fitbit watch. The interviewer will ask targeted probe questions to understand the differences between what the device measured and what the participant reported.

3. Participants

We will recruit 45 participants using a targeted advertisement in a newspaper or online billboard (see Appendix C). During the recruiting process, we will screen potential participants to find participants who are employed, unemployed, and not in the labor force. We will also screen participants to determine whether there are children in their household (see Appendix D for recruitment script and screening questions). These characteristics have been shown to correlate with the amount of sleep a person gets. Quota sampling will also be used to get roughly equal numbers of participants in each employment and household composition (i.e., with and without children) category.

4. Burden Hours

The total burden hours for this study will be 76. Forty-five participants will be recruited to participate in this study. We anticipate a moderate acceptance rate for the initial recruitment contact with potential participants (approximately 80%), so estimate we will need to contact approximately 55 people in order to get 45 interviews. We expect these initial recruitment calls to last 5 minutes. This will add 5 burden hours (55 participants X .083 hours) to the study.

The first visit will be short; 20 minutes at most. This will add 15 burden hours (45 participants X 0.33 hours/interview) to the study. Additionally, we expect that participants will spend no more than 15 minutes on study-related matters during the week (e.g., midweek phone call, care for the watch) for a total of 11 burden hours (45 participants X .25 hours). Finally, we expect the second interview to take up to one hour to complete, adding an additional 45 burden hours (45 participants X 1 hour/interview).

	N	Burden Hours/ Participant	Total Burden Hours
Initial Recruitment	55	0.083	5
Visit 1	45	0.33	15
At-Home Time	45	0.25	11
Visit 2	45	1	45
	TOTAL	1 hr. 40 min.	77 hours

5. Payment to Participants

Each participant will be compensated a total of $80 for their participation in this study. Because it is vital that the participant returns for the second interview, we will compensate the participant $30 for their first visit and $50 at the time of their second visit. We believe that offering a greater incentive for the second visit (while still maintaining an average of $40 per visit) will be helpful in limiting the number of breakoffs between the first and second study¹. If the participant does not come back for the second interview, we will not get the Fitbit watch back and we will not get their feedback; both key to the success of this study. A total of $3600 will be spent on payments to participants.

6. Data Confidentiality

Participants will be informed as to the voluntary nature of the study. Participants will also be informed that the study will be used for internal purposes to improve the design of the American Time Use Survey. Participants will be given a consent form to read and sign (See Appendix E). Information related to this study will not be released to the public in any way that would allow identification of individuals except as prescribed under the conditions of the Privacy Act Notice.

See the following attachments:

Appendix A – Interview Protocol

Appendix B – Watch Care Pamphlet

Appendix C – Recruiting Advertisement

Appendix D – Recruiting Script and Screening Questions

Appendix E – Consent Form

Appendix A – Interview Protocol

Visit 1

Introductory Material

• Hi! Thank you for coming in today.

• Have you participated in any of our studies before? (if yes, Which ones?)

• Study Introduction:

  • One of the surveys we conduct at BLS is the American Time Use Survey. In this survey, we ask people about all the things they did the previous day. And that’s what this study is about…how you spend your day. We are going to ask you to wear a Fitbit watch this whole week. We’ll then have you come back and we’ll ask you some questions about how you spent your time during the week. Your answers will help us improve the questions we ask in the American Time User Survey.

  • We are not evaluating you, we are evaluating the questions that we ask. All the information you give us and all of the information recorded by the Fitbit will be kept completely confidential, and will be used to improve the survey questions.

  • I’ll point out that you don’t get to keep the watch. We need it back so we can have more people go through this study. To compensate you for your time, we’ll pay you $30 this time and $50 when you come back next week. Also, if you would like, we will also give you a printout of what the Fitbit measured over the week.

• Consent Form/Permission to audiotape

• Any questions before we begin?

I would like to begin by asking you a few questions about you and your household.

• How many people are in your household, including yourself? ______

• How old are you?

• [if more than 2 in HH] OK, and who is the next person? I don’t need their name, just how they are related to you.

Name	Relationship	Age	Gender
	Participant		M F
			M F
			M F
			M F
			M F

• Are you currently employed? YES NO

  • (if NO) Are you currently looking for work?

  • (if YES) Do you work part-time (less than 35 hours per week) or full-time (35 hours or more)?

  • (IF YES) During what hours is your typical work shift?

  • (IF YES) Do you typically work on weekends? YES NO

  • (if YES) How long does it take you to commute to and from work?

  • (if YES) How do you typically commute to work? CAR PUBLIC TRANSPORTATION WALK BICYCLE OTHER

  • (if YES) Do you usually start and stop work around the same time each day? YES NO

• Do you own a car? YES NO

• Would you say that you have a typical daily routine? YES NO

• How many days per week do you usually exercise?

  • (if > 0) How long do you typically spend exercising when you do exercise?

• How many hours per day are you sitting down?

• How many hours per day do you spend interacting with computers, tablets, and smartphones?

• How many hours per day do you spend watching television?

Explanation of Fitbit

• This is the watch we will ask you to wear. Can you try it on to make sure it fits?

• When you’re putting it on, you have to make sure that both of the metal pegs are fully seated.

• It’s important that you wear the watch at all times, even when you’re exercising or sleeping. Of course, you can take it off to shower or bathe or any other time where the watch may get damaged, but please put it back on immediately after. It’s water resistant so you can wear it in the rain, while you’re working out, or while you’re washing your hands.

• When you do take the watch off, please do not reclasp it before putting it down. This will make the watch think that you’re still wearing it. Leave it open, like this [Demonstrate with watch].

• Feel free to use the watch. In addition to the time, it will show you the number of steps you’ve taken, the distance you’ve walked or ran, the number of floors you’ve climbed, and the number of calories you burned that day. To view these measurements you just gently tap on the front of the watch.

• We ask that you do not sync the watch with your phone or your computer. This may clear out the data and we need to be able to download it at the end of the week. We will show you your records at the end of the week.

• The watch is fully charged so you won’t need to charge it. It should last the whole week. If it doesn’t, that’s fine.

• We also have this pamphlet for you. It covers some of the basic operations of the watch and how to care for it. It also has the contact information for myself and my colleague should you have any questions during the week.

• Remember that we need the watch back at the end of the week, so be sure to bring it with you when you come back.

End of Visit 1

• That is all for this time.

• Have you scheduled your visit for next week?

  • [if No] OK, can we schedule you now?

• If you have any questions during the week, my phone number and email are written on the pamphlet. If I’m not there, please leave a message and I’ll get back to you as soon as possible.

• Is it alright if I call you on [THREE DAYS AFTER MEETING] to see how things are going?

  • [if Yes] Will the number that our recruiter has for you work, or is there another number I should call?

  • [if No] That’s OK. I’ll see you at our meeting next week and see how things went.

• Thank you for coming in.

Mid-Week Telephone Call

• Hi. This is ______ from BLS. I’m calling to see how things are going.

• Do you have any questions? YES NO

• Have you been wearing the watch all the time? YES NO

• Have you been wearing the watch while you exercise? YES NO

• Have you been wearing the watch while you sleep? YES NO

• Thank you for your participation so far.

• [If scheduled] I will see you on ______.

• [If not scheduled] We still need to find a time for you to come in. Can we do that now?

Visit 2

Fitbit Experience Questions

• How was your experience wearing the watch?

• Did you have any issues with the watch? YES NO

  • (if YES) Tell me more about that.

• About what percent of the time would you say you wore the watch this week?

• Did you take off the watch for any reason this week? YES NO

  • (if YES) For what reasons did you take it off?

• Would you say this week was typical of how you spend your time? YES NO

  • (if NO) How was it different?

• Did you work more or less than usual or about the same? MORE LESS ABOUT THE SAME

  • (if MORE or LESS) Why was this different?

• Did you exercise more or less than usual or about the same? MORE LESS ABOUT THE SAME

  • (if MORE or LESS) Why was this different?

• Did you sleep more or less than usual or about the same? MORE LESS ABOUT THE SAME

  • (if MORE or LESS) Why was this different?

• Do you think that wearing the watch affected what you did this week? YES NO

  • (if YES) Tell me more about that.

• Did you sync the watch with an app or computer program to view the data? YES NO

  • (if YES) Tell me more about that.

ATUS Interview and Stylized Questions

The ATUS questions and stylized question order would be counterbalanced.

ATUS Style Interview

The ATUS interview will start at 4 a.m. and run through 10 a.m. or whenever the participant woke up. It will then skip to 7 p.m. and continue through the participant’s bedtime. Participants will not be asked where they were or who they were with.

Now I'd like to find out how you spent your time yesterday, <DAY & DATE>. If an activity is too personal, there's no need to mention it.

• Let’s begin with yesterday, <DAY>, at 4 a.m. What were you doing?

  • (if sleeping) What time did you wake up yesterday?

  • (if not sleeping) What did you do next?

• What did you do next?

  • How long did you spend <ACTIVITY>?

• What did you do next?

• …

• OK, now I would like to skip forward to last night at 7 p.m. What were you doing at 7 p.m.?

  • How long did you spend <ACTIVITY>?

• What did you do next?

  • How long did you spend <ACTIVITY>?

• *… (Continue until the participant mentions sleep)

• What time did you wake up?

• We skipped from about 10 a.m. to 7 p.m., during that time yesterday did you take any naps?

  • (if yes) What time did you fall asleep for your nap? What time did you wake up?

*Going/Went to bed (FROM ATUS TRAINING MANUAL)

• Ask the probe "What time did you fall asleep?"

Stylized Questions

The following sets of questions could be asked together (in their current order), only one set could be asked if we prefer one over the other, or half of the participants could receive one set, and another half could receive the other.

• How long do you sleep at night on an average weekday?

• Do you typically take naps?

  • (if yes) How long do your naps usually last?

• Thinking about the past week (between <DATE> and <DATE>), on average, how long have you slept each night?

• Did you take any naps during the last week?

  • (if yes) How long was your typical nap this week?

• Was the amount you slept during the last week typical of how you usually sleep?

  • (if no) In what way was your sleep this week unusual?

• Thinking about the past week (between <DATE> and <DATE>), on average, how long have you slept each night?

• On average, at what time did you fall asleep on Monday through Friday during the past week?

• On average, at what time did you wake on Monday through Friday during the past week?

• On average, at what time did you fall asleep on Saturday and Sunday during the past week?

• On average, at what time did you wake up Saturday and Sunday during the past week?

Debriefing

The debriefing will always be asked in this order, regardless of the order of the ATUS interview and stylized questions.

Thanks for going through those questions with me. We are going to switch gears now and talk a bit about what it was like for you to answer those questions. I have some follow-up questions that will help us better understand how people are responding to this survey. I want to emphasize that there are no ‘right’ or ‘wrong’ answers to these questions – we are just interested in your interpretations and reactions.

1. What were your reactions to these questions?

2. I asked you a number of questions about sleep. What does the word “sleep” mean to you?

3. What activities do you include as part of sleep?

   1. (if any non-sleep activities reported) Did you include any of these activities in your estimate of how long you slept?

Debriefing – ATUS

1. When we walked through your day yesterday, you said that you woke up at <WAKE TIME>. How did you arrive at your answer to this question?

   1. (if exact) Can you describe how you know that is the exact time you woke up?

2. When someone asks, “What time did you get up?” what does this mean to you? Is this the same or different from what time you woke up?

3. What time do you usually wake up?

   1. (if different than time reported for yesterday) Why was yesterday different?

4. You said that you went to sleep at <SLEEP TIME>. How did you arrive at your answer to this question?

   1. (if estimate) Can you remember the exact time you went to sleep yesterday?

   2. (if exact) How do you know that is the exact time you went to sleep?

   3. About how long did it take you to fall asleep last night?

5. When someone asks, “What time did you go to bed?” what does this mean to you? Is this the same or different from the time you actually fell asleep?

6. Is <SLEEP TIME> when you usually go to sleep or was yesterday different?

   1. (if unusual) Why was yesterday different?

   2. (if unusual) Do you have a usual time that you go to sleep? When is that?

7. (if nap) I asked about naps and you said you slept for <NAP DURATION> yesterday. Is that time exact or an estimate?

   1. (if estimate) Do you remember the exact amount of time you napped?

   2. (if estimate) How precise do you think I wanted to you to be about naps?

8. Given the time you went to sleep last night and the time you woke up this morning, I calculate that you slept <SLEEP DURATION> last night. Does that sound correct?

9. Did you wake up during the night?

   1. (if yes) How many times?

   2. (if yes) Were you awake for more than 15 minutes?

10. Is it easy or difficult to remember what time you woke up yesterday? Why is that?

    1. response options provided on showcard: Very Easy, Somewhat Easy, Neither Easy Nor Difficult, Somewhat Difficult, Very Difficult

11. Is it easy or difficult to remember what time you fell asleep yesterday? Why is that?

    1. Response options provided: Very Easy, Somewhat Easy, Neither Easy Nor Difficult, Somewhat Difficult, Very Difficult

Debriefing – Stylized Questions

1. I asked you about how long you slept on the average night last week. How did you go about answering that question?

2. I also asked about the average time you fell asleep during the past week. How did you go about answering that question?

3. I asked you about the average time you woke up during the past week. How did you go about answering that question?

4. Is it easy or difficult to determine how long you sleep on an average weekday? Why is that?

   1. Very Easy, Somewhat Easy, Neither Easy Nor Difficult, Somewhat Difficult, Very Difficult

5. How certain are you that you slept <SLEEP DURATION> on average last week?

   1. Not at all certain, A little certain, Somewhat certain, Very certain, Extremely certain

Debriefing - Social Desirability

1. Why do you think the government would ask questions about how much sleep people get?

2. Do you think there is an appropriate number of hours a person should sleep in one night?

   1. What is the minimum number of hours a person should sleep in one night?

   2. What is the maximum number of hours a person should sleep in one night?

3. Do you think it would be more embarrassing for someone to admit they have slept too much or slept too little? Why is that?

   1. If I told you that I am conducting a survey about employment and jobs, do you think it would be more embarrassing to admit having slept too much or too little?

   2. If I told you that I am conducting a survey about health, do you think it would be more embarrassing to admit having slept too much or too little?

4. Is there a time in the morning that you think everyone should be awake by?

5. Is there a time that you would be embarrassed to tell someone you slept until?

6. Is there a time at night you consider too early to be in bed? That is, is there a time that you would be embarrassed to tell someone you fell asleep for the night?

7. Would you be embarrassed to discuss naps you take during the day?

8. Do you discuss how much you sleep with friends and family?

   1. (if yes) Is the amount you sleep greater than, less than, or about the same as the people you talk to?

9. Do you think you sleep more than, less than, or about the same as the average American?

Fitbit Comparison Questions

These questions will be somewhat unstructured. While we will attempt to follow the protocol below as much as possible, there will likely be a number of unique situations that we did not account for. The interviewer will ask unscripted questions to seek more information about these situations.

Now I’d like talk about what the Fitbit measured. I would like to compare it to what you reported and get your feedback about any differences. Remember, this is not an evaluation of you or your answers. There are a number of reasons why there might be differences, we are looking to find out what those reasons might be.

1. The Fitbit recorded that you slept ____ hours last night.

   1. [if equal to or close to ATUS report] That is close to what you reported. Does that number seem right to you?

   2. [if greater than or less than] Does that number seem right to you?

   3. [if greater than or less than] That is (greater than/less than) what you reported when you we walked through your day yesterday. Can you tell me more about that?

   4. [if greater than or less than] The Fitbit recorded _____ as the time you fell to sleep (last night/this morning). Does that sound right?

   5. [if greater than or less than] The Fitbit recorded _____ as the time you woke up (this morning/last night). Does that sound right?

   6. [if sleep data indicate time awake during night] Do you recall waking up during the night?

2. [if went to bed time is different than went to sleep time] The Fitbit recorded that you laid down (but hadn’t fallen asleep yet) at _____. Does that sound right?

3. [if got out of bed time is different than woke up time] The Fitbit recorded that you got out of bed at _____ (after having woken up at _____). Does that sound right?

4. The Fitbit recorded that you slept, on average, ____ hours per night during the weeknights and ____ per night on the weekends.

   1. [if equal to or close to stylized report] That is close to what you reported. Does that number seem right to you?

   2. [if greater than or less than] Does that number seem right to you?

   3. [if greater than or less than] That is (greater than/less than) what you reported when you we talked about the amount of sleep you get during an average week. Can you tell me more about that?

   4. [if sleep data indicate an irregular sleep pattern or deviations on an individual night] Were there any days this week, that your sleep was unusual or out of the ordinary?

      1. [if yes] Can you tell me more about that?

   5. [if sleep data indicate time awake during night] Do you recall waking up during the night?

5. [if sleep data indicate any sleep duration < 3 hours between 5 a.m. and 10 p.m.] Do you recall taking any naps this week?

   1. [if yes] On which days?

6. Do you have any additional comments about the Fitbit, about how you spent your time this week, or anything else?

Appendix B – Watch Care Pamphlet

See Attachment

Appendix C – Recruiting Advertisement

Research Participants Needed

The U.S. Bureau of Labor Statistics is looking for:

Individuals, ages 18 and up, to participate in a study about how people spend their time.

to help improve questions that are used in a national survey.

$80 for two sessions about 1 week apart

Call (202) 691-7524

For more information

Sessions will be conducted at the Bureau of Labor Statistics office near Union Station (2 Massachusetts Ave., Washington, DC 20212).

Appendix D – Recruiting Script and Screening Questions

Participants in this study will be recruited from a database maintained by OSMR and from individuals who call in response to an advertisement (see Appendix C). The database contains demographic information about the potential participants, so participants of varying income, education, occupation, and household size groups will be drawn and asked to participate via a phone call. The phone call will be roughly scripted, as shown below, but is impossible to anticipate all questions that potential participants may ask or what explanation may be required.

OMB Number 1220-0141

Expires: April 30, 2018

Recruitment script:

Hello, this is _____ from the Bureau of Labor Statistics. I am calling to see if you are interested in participating in an upcoming study. In this study, we will ask you to come to our offices at BLS two times…about a week apart. In between those meetings, we want you to wear a Fitbit watch. The watch will measure your activity level. There is no real time commitment, you just have to have wear the watch at all times during that week, including while exercising, working, and sleeping.

At the first meeting, we will give you some information about the watch and at the second meeting we’ll ask you some questions about how you spent your time during that week. The first meeting will be about 20 minutes long, the second meeting will be about an hour, and we estimate that you will spend about 15 minutes on the study during the week for a total of about 1 hour and 35 minutes of time commitment. To compensate you for your time, we are offering $80 for this study. Thirty dollars will be paid at the first meeting and $50 will be paid at the second meeting. You do not get to keep the watch.

Are you interested in participating?

[if Yes] OK, I have a few questions that will help us see if you are eligible to participate in this study. OK?.

1. Are you currently employed? YES NO

   1. (if yes) What is the length of your usual work day? ______

   2. (if yes) How many hours do you work in a typical week? _____

   3. (if no) Are you currently looking for work? YES NO

      1. (if no) Are you retired or a student? RETIRED STUDENT

2. How many people live in your household, including yourself? _____

   1. (if > 1) How many of those people are under the age of 18? _____

   2. (if > 1) How many of those people are under the age of 5? _____

3. Are you going on vacation or doing anything else that would disrupt your usual routine over the next couple weeks? YES NO [Make sure that these times don’t overlap with their time in the study]

[if No] OK, thank you for your time. Have a good day.

[if Qualified] You are eligible to participate in this study. Let’s get you scheduled. If you know your availability, I would like to schedule both meetings today. If you don’t, I’ll get you scheduled for the first meeting and you can work out a time for the second one with your interviewer when you come in.

[if Not Qualified] I’m sorry but you are not eligible to participate in this study. If you would like, I can get your contact information and we will keep you in mind for future studies.

[If the person requests to know why they were not eligible. The recruiter will explain the eligibility requirements.]

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless such collection displays a valid OMB control number. Public reporting burden for this collection of information is estimated to average 5 minutes per response, including time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. The obligation to respond to this collection is voluntary. Send comments regarding the burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to the U.S. Department of Labor, OASAM/OCIO 200 C Street NW Washington DC 20210 and reference the OMB Control Number 1220-0141.

Appendix E – Consent Form

CONSENT FORM

The Bureau of Labor Statistics (BLS) is conducting research to increase the quality of BLS surveys. This study is intended to suggest ways to improve the procedures the BLS uses to collect survey data.

The BLS, its employees, agents, and partner statistical agencies, will use the information you provide for statistical purposes only and will hold the information in confidence to the full extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act of 2002 (Title 5 of Public Law 107-347) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent. The Privacy Act notice on the back of this form describes the conditions under which information related to this study will be used by BLS employees and agents.

During this research you may be audio and/or videotaped, or you may be observed. If you do not wish to be taped, you still may participate in this research.

(Visit 1) We estimate it will take an average of 20 minutes to participate in this research today and another 15 minutes worth of your time at home over the next week.

(Visit 2) We estimate it will take an average of 60 minutes to participate in this research today.

Your participation in this research project is voluntary, and you have the right to stop at any time. If you agree to participate, please sign below.

Persons are not required to respond to the collection of information unless it displays a currently valid OMB control number. The OMB control number is 1220-0141 and expires on April 30, 2018.

------------------------------------------------------------------------------------------------------------

I have read and understand the statements above. I consent to participate in this study.

• (Visit 1) I understand that I must come back in one week and return the Fitbit watch to BLS

___________________________________ ___________________________

Participant's signature Date

___________________________________

Participant's printed name

___________________________________

Researcher's signature

PRIVACY ACT STATEMENT

In accordance with the Privacy Act of 1974, as amended (5 U.S.C. 552a), you are hereby notified that this study is sponsored by the U.S. Department of Labor, Bureau of Labor Statistics (BLS), under authority of 29 U.S.C. 2. Your voluntary participation is important to the success of this study and will enable the BLS to better understand the behavioral and psychological processes of individuals, as they reflect on the accuracy of BLS information collections. The BLS, its employees, agents, and partner statistical agencies, will use the information you provide for statistical purposes only and will hold the information in confidence to the full extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act of 2002 (Title 5 of Public Law 107-347) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent.

References

Arora, T., Broglia, E., Pushpakumar, D., Lodhi, T., & Taheri, S. (2013). An investigation into the

strength of the association and agreement levels between subjective and objective sleep duration in adolescents. PloS one, 8(8), e72406.

Bonke, J. (2005). Paid work and unpaid work: Diary information versus questionnaire

information. Social Indicators Research, 70(3), 349-368.

Brenner, P. S. (2011). Identity importance and the overreporting of religious service attendance:

Multiple imputation of religious attendance using the American Time Use Study and the

General Social Survey. Journal for the Scientific Study of Religion, 50(1), 103-115.

Kan & Pudney (2007). Measurement Error in Stylised and Diary Data on Time Use.

Sociological Methodology 38, 101-132.

Lauderdale, D. S. (2015). Commentary on “Agreement between simple questions about sleep

duration and sleep diaries in a large online survey”.Sleep Health: Journal of the National Sleep Foundation, 1(2), 138-139.

Lauderdale, D. S., Knutson, K. L., Yan, L. L., Rathouz, P. J., Hulley, S. B., Sidney, S., & Liu, K.

(2006). Objectively Measured Sleep Characteristics Among Early-Middle-Aged Adults. American Journal of Epidemiology, 164, 5-16.

Lauderdale, D. S., Knutson, K. L., Yan, L. L., Liu, K., & Rathouz, P. J. (2008). Self-Reported

and Measured Sleep Duration: How Similar Are They? Epidemiology, 19, 838-845.

Miller, C. B., Gordon, C. J., Toubia, L., Bartlett, D. J., Grunstein, R. R., D'Rozario, A. L., &

Marshall, N. S. (2015). Agreement between simple questions about sleep duration and sleep diaries in a large online survey. Sleep Health, 1(2), 133-137.

Silva, G. E., Goodwin, J. L., Sherrill, D. L., Arnold, J. L., Bootzin, R. R., Smith, T., Walsleben,

J. A., Baldwin, C. M., & Quan, S. F. (2007). Relationship Between Reported and Measured Sleep Times: The Sleep Heart Health Study (SHHS). Journal of Clinical Sleep Medicine, 3, 622- 630.

Tourangeau, R., Rasinski, K. A., & Bradburn, N. (1991). Measuring happiness in surveys: A test

of the subtraction hypothesis. Public Opinion Quarterly, 55(2), 255-266.

Tremaine, R. B., Dorrian, J., & Blunden, S. (2010). Subjective and objective sleep in children

and adolescents: measurement, age, and gender differences. Sleep and Biological Rhythms, 8(4), 229-238.

1 BLS will provide OMB with a report of this study’s breakoffs within one month of the conclusion of data collection.

24