July 22, 2013
NOTE TO THE REVIEWER OF: |
OMB CLEARANCE 1220-0141 “Cognitive and Psychological Research”
|
FROM: |
Brandon Kopp Research Psychologist Office of Survey Methods Research
|
SUBJECT: |
Submission of Materials for the Cognitive Testing of American Time Use Survey Sleep Measures and Sleep Estimate Questions |
Please accept the enclosed materials for approval under the OMB clearance package 1220-0141 “Cognitive and Psychological Research.” In accordance with our agreement with OMB, we are submitting a brief description of the study.
The total estimated respondent burden hours for this study are 32.5.
If there are any questions regarding this project, please contact Brandon Kopp at
202-691-7514 or Polly Phipps 202-691-7513.
Introduction and Purpose
The American Time Use Survey (ATUS) uses a time diary format to ask respondents about the activities they performed on the prior day. In 2012, the ATUS reported that members of the civilian, non-institutionalized population in the United States spend an average of 8.7 hours per day sleeping. ATUS estimates of sleep are consistently higher than surveys that ask respondents directly about the amount of sleep they get (referred to in this protocol as stylized questions). The National Sleep Foundation (NSF), for example, asks respondents to estimate the amount of time they spend sleeping on an average weekday and weekend day. These numbers are then combined into an overall estimate. The NSF reports that American adults get an average of 6.9 hours of sleep per day.
The ATUS differs from surveys that use stylized questions in methodology, sleep definition, and test population. It is unclear how these differences influence the estimates. The goal of the current study is to understand the measurement properties of both the ATUS interview as it relates to sleep and of a sample of stylized questions. How do respondents interpret the questions being asked? Are there activities that are more likely to be included or excluded from sleep reports given the different question styles? What memory and estimation processes are used to answer these different types of questions? Do social desirability concerns about getting too much or too little sleep affect how they answer the questions? Are respondents’ reports affected by their employment status or the presence of children in the household? To answer these questions, participants in this study will be asked both ATUS time diary questions and stylized sleep questions and then asked to report about their experience answering the questions using cognitive interview techniques.
2. Research Design
Cognitive interviews provide an in-depth understanding of a respondent’s thought processes and reactions to a question. The cognitive interviews for this study will be done one-on-one, in the Office of Survey Methods Research (OSMR) laboratory, and will be audiotaped. Interviews will be conducted by staff from the OSMR who are experienced in conducting these types of interviews. The testing protocol is included in Appendix A.
Participants will be asked to complete an abbreviated ATUS daily recall interview and answer a set of 5-8 stylized questions regarding how much they sleep on an average weekday (NOTE: participants will be brought into the lab on Tuesday – Friday so their ATUS recall of the prior day’s activities will always cover a weekday). The abbreviated ATUS interview will ask participants to report their daily activity for the prior day focusing on times in which they are likely to have woken up and went to sleep (i.e., 4 a.m. – 10 a.m. and 7 p.m. – 3:59 a.m.). Questions about where activities took place and who else was present (typical ATUS recall questions) will not be asked.
The order of the ATUS interview and the stylized questions will be counterbalanced such that half of the participants will be asked the ATUS interview questions followed by the stylized questions and the other half will answer the stylized questions followed by the ATUS interview questions. Participants will be asked to think aloud while coming up with their answers to the survey questions.
Following the administration of the survey questions, participants will be asked a series of cognitive interview questions. The goal of these questions is to understand how participants recall and/or estimate answers to both the ATUS interview and the stylized questions. Probes will also seek to verify the accuracy of participants’ responses. Finally, participants will be asked a series of questions to understand whether social desirability concerns may have caused them to modify their answers.
3. Participants
We will recruit 30 participants from the OSMR participant database. During the recruiting process, we will screen potential participants to find participants who are employed, unemployed, and not in the labor force. We will also screen participants to determine whether there are children in their household (see Appendix B for screening questions). These characteristics have been shown to correlate with the amount of sleep a person gets. Quota sampling will also be used to get roughly equal numbers of participants in each employment and household composition (i.e., with and without children) category.
4. Burden Hours
Our goal is to obtain feedback from 30 participants in the OSMR lab. We anticipate that each session will last a maximum of one hour, for a total of 30 burden hours. Participants will be compensated $40 for their time. In addition to time spent in the laboratory, participants will be screened to determine which of the relevant demographic categories they qualify for. Answering these questions and scheduling an appointment are expected to take no longer than 5 minutes and participants will not be compensated for time spent on screening and scheduling. This will add an additional 2.5 burden hours (30 participants X .08 hours) to the study, for a total of 32.5 burden hours.
5. Data Confidentiality
Participants will be informed as to the voluntary nature of the study. Participants will also be informed that the study will be used for internal purposes to improve the design of the American Time Use Survey. Participants will be given a consent form to read and sign (See Appendix C). Information related to this study will not be released to the public in any way that would allow identification of individuals except as prescribed under the conditions of the Privacy Act Notice.
Appendix A – Cognitive Interview Protocol
Part 1 - Introductory Material
Hi! Thank you for coming in today.
Have you participated in any of our studies before? (if yes, Which ones?)
Study Introduction:
One of the surveys we conduct at BLS is the American Time Use Survey. In this survey, we call people and ask them about all the things they did the previous day.
Today we’ll be discussing different types of questions about how people spend their day?
What we are going to do today is go through a few questions from the survey and get your reactions to them. The purpose of this session is to help us find out more about how people respond to these questions. Basically, we’re trying to find out what you think a question is asking and how hard it is to answer. We are not here to evaluate you, we are looking to improve the questions, so there are no wrong answers. All the information you give us will be kept completely confidential, and will be used to improve the survey questions.
Consent Form/Permission to audiotape
Any questions before we begin?
Part 2 – ATUS Interview and Stylized Questions
The ATUS questions and stylized question order would be counterbalanced.
ATUS Style Interview
The ATUS interview will start at 4 a.m. and run through 10 a.m. or whenever the participant woke up. It will then skip to 7 p.m. and continue through the participant’s bedtime. Participants will not be asked where they were or who they were with.
Now I'd like to find out how you spent your time yesterday, <DAY & DATE>. If an activity is too personal, there's no need to mention it.
Let’s begin with yesterday, <DAY>, at 4 a.m. What were you doing?
(if sleeping) What time did you wake up yesterday?
(if not sleeping) What did you do next?
What did you do next?
How long did you spend <ACTIVITY>?
What did you do next?
…
OK, now I would like to skip forward to last night at 7 p.m. What were you doing at 7 p.m.?
How long did you spend <ACTIVITY>?
What did you do next?
How long did you spend <ACTIVITY>?
*… (Continue until the participant mentions sleep)
What time did you wake up?
We skipped from about 10 a.m. to 7 p.m., during that time yesterday did you take any naps?
(if yes) What time did you fall asleep for your nap? What time did you wake up?
*Going/Went to bed (FROM ATUS TRAINING MANUAL)
|
Stylized Questions
The following sets of questions could be asked together (in their current order), only one set could be asked if we prefer one over the other, or half of the participants could receive one set, and another half could receive the other.
How long do you sleep at night on an average weekday?
Do you typically take naps?
(if yes) How long do your naps usually last?
Thinking about the past week (between <DATE> and <DATE>), on average, how long have you slept each night?
Did you take any naps during the last week?
(if yes) How long was your typical nap this week?
Was the amount you slept during the last week typical of how you usually sleep?
(if no) In what way was your sleep this week unusual?
Part 3 – Debriefing
The debriefing will always be asked in this order, regardless of the order of the ATUS interview and stylized questions.
Thanks for going through those questions with me. We are going to switch gears now and talk a bit about what it was like for you to answer those questions. I have some follow-up questions that will help us better understand how people are responding to this survey. I want to emphasize that there are no ‘right’ or ‘wrong’ answers to these questions – we are just interested in your interpretations and reactions.
What were your reactions to these questions?
I asked you a number of questions about sleep. What does the word “sleep” mean to you?
What activities do you include as part of sleep?
(if any non-sleep activities reported) Did you include any of these activities in your estimate of how long you slept?
Debriefing – ATUS
When we walked through your day yesterday, you said that you woke up at <WAKE TIME>. How did you arrive at your answer to this question?
(if estimate) If I pressed you a bit, do you think you could remember the exact time you woke up yesterday?
(if estimate) How precise do you think I wanted to you to be? Within a few minutes, within 5 minutes, within 10 minute, within 15 minutes, within a half hour, within an hour?
(if exact) Can you describe how you know that is the exact time you woke up?
What time do you usually wake up?
(if different than time reported for yesterday) Why was yesterday unusual?
You said that you went to sleep at <SLEEP TIME>. How did you arrive at your answer to this question?
(if estimate) Can you remember the exact time you went to sleep yesterday?
(if exact) How do you know that is the exact time you went to sleep?
About how long did it take you to fall asleep last night?
Is <SLEEP TIME> when you usually go to sleep or was yesterday unusual?
(if unusual) Why was yesterday unusual?
(if unusual) Do you have a usual time that you go to sleep? When is that?
(if nap) I asked about naps and you said you slept for <NAP DURATION> yesterday. Is that time exact or an estimate?
(if estimate) Do you remember the exact amount of time you napped?
(if estimate) How precise do you think I wanted to you to be about naps?
Given the time you went to sleep last night and the time you woke up this morning, I calculate that you slept <SLEEP DURATION> last night. Does that sound correct?
Did you wake up during the night?
(if yes) How many times?
(if yes) Were you awake for more than 15 minutes?
Is it easy or difficult to remember what time you woke up yesterday? Why is that?
response options provided on showcard: Very Easy, Somewhat Easy, Neither Easy Nor Difficult, Somewhat Difficult, Very Difficult
Is it easy or difficult to remember what time you fell asleep yesterday? Why is that?
Response options provided: Very Easy, Somewhat Easy, Neither Easy Nor Difficult, Somewhat Difficult, Very Difficult
Debriefing – Stylized Questions
I asked you about how long you slept on the average night last week. How did you go about answering that question?
I also asked if you usually take naps and you said <ANSWER>. What does the word “usually” mean to you?
When was the last time you took a nap? How long did that nap last?
Is it easy or difficult to determine how long you sleep on an average weekday? Why is that?
Very Easy, Somewhat Easy, Neither Easy Nor Difficult, Somewhat Difficult, Very Difficult
How certain are you that you slept <SLEEP DURATION> on average last week?
Not at all certain, A little certain, Somewhat certain, Very certain, Extremely certain
Debriefing - Social Desirability
Is talking about how much you sleep a sensitive topic for you?
(if yes) Why is that?
Were there any questions that I asked today that you thought were too personal?
Why do you think the government would ask questions about how much sleep people get?
Do you think there is an appropriate number of hours a person should sleep in one night?
What is the minimum number of hours a person should sleep in one night?
What is the maximum number of hours a person should sleep in one night?
Do you think it would be more embarrassing for someone to admit they have slept too much or slept too little? Why is that?
If I told you that I am conducting a survey about employment and jobs, do you think it would be more embarrassing to admit having slept too much or too little?
If I told you that I am conducting a survey about health, do you think it would be more embarrassing to admit having slept too much or too little?
Is there a time in the morning that you think everyone should be awake by?
Is there a time that you would be embarrassed to tell someone you slept until?
Is there a time at night you consider too early to be in bed? That is, is there a time that you would be embarrassed to tell someone you fell asleep for the night?
What is an appropriate length for a nap?
What would you consider too long of a time for a nap?
Would you be embarrassed to discuss naps you take during the day?
Do you discuss how much you sleep with friends and family?
(if yes) Is the amount you sleep greater than, less than, or about the same as the people you talk to?
Do you think you sleep more than, less than, or about the same as the average American?
Appendix B - Screening Questions
Are you currently employed?
(if yes) What is the length of your usual work day?
(if no) Are you currently looking for work?
How many people live in your household, including yourself?
(if > 1) How many of those people are under the age of 18?
(if > 1) How many of those people are under the age of 5?
Appendix C – Consent Form
The Bureau of Labor Statistics (BLS) is conducting research to increase the quality of BLS surveys. This study is intended to suggest ways to improve the procedures the BLS uses to collect survey data.
The BLS, its employees, agents, and partner statistical agencies, will use the information you provide for statistical purposes only and will hold the information in confidence to the full extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act of 2002 (Title 5 of Public Law 107-347) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent. The Privacy Act notice on the back of this form describes the conditions under which information related to this study will be used by BLS employees and agents.
During this research you may be audio and/or videotaped, or you may be observed. If you do not wish to be taped, you still may participate in this research.
We estimate it will take you an average of 45 minutes to participate in this research (ranging from 30 minutes to 60 minutes).
Your participation in this research project is voluntary, and you have the right to stop at any time. If you agree to participate, please sign below.
Persons are not required to respond to the collection of information unless it displays a currently valid OMB control number. OMB control number is 1220-0141, and expires 2-28-2015.
------------------------------------------------------------------------------------------------------------
I have read and understand the statements above. I consent to participate in this study.
___________________________________ ___________________________
Participant's signature Date
___________________________________
Participant's printed name
___________________________________
Researcher's signature
OMB Control Number: 1220-0141
Expiration Date: [02-28-2015]
In accordance with the Privacy Act of 1974, as amended (5 U.S.C. 552a), you are hereby notified that this study is sponsored by the U.S. Department of Labor, Bureau of Labor Statistics (BLS), under authority of 29 U.S.C. 2. Your voluntary participation is important to the success of this study and will enable the BLS to better understand the behavioral and psychological processes of individuals, as they reflect on the accuracy of BLS information collections. The BLS, its employees, agents, and partner statistical agencies, will use the information you provide for statistical purposes only and will hold the information in confidence to the full extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act of 2002 (Title 5 of Public Law 107-347) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent.
Page
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | kopp_b |
| File Modified | 0000-00-00 |
| File Created | 2021-01-31 |