Attachment I - Cognitive Testing Results for the 2011 Leave Module

Findings from the Cognitive Testing of the Leave Module Questions

American Time Use Survey

May 2010

Stephanie Denton and Rose Woods

1. Purpose

The American Time Use Survey (ATUS) is a nationally-representative monthly sample survey of households that measures how Americans spend their time. The Women’s Bureau at the Department of Labor is considering sponsoring an ATUS Leave module that would capture information about workers' access to paid and unpaid leave, the reasons for which they can take leave, the types of leave available to them, their use of leave, and their ability to vary their work schedules instead of taking leave. The proposed Leave module also would allow researchers to examine the relationship between time use and leave. This module would supplement the information available in the ATUS to measure the ability of working people to balance work with other demands or life circumstances.

This report summarizes the results of a cognitive test of the leave module. The primary objective of this study was to evaluate how well the leave questions capture the information sought, to explore participants’ understanding of the questions, and to identify problems of question wording or particular concepts. The questionnaire was tested for clarity, comprehension, length, potential sensitivity, and the flow through the instrument.

2. Methodology

Participants

Four members of the ATUS and OSMR staff conducted 24 cognitive interviews in two rounds (the first phase consisted of 19 interviews; the second phase consisted of 5 interviews). Wage and salary workers were recruited from an ad on Craig’s List, a general-population database maintained by the Office of Survey Methods Research (OSMR)¹, and people known to have various job characteristics (e.g., teachers, firefighters, persons on leave in the last week). Altogether, 18 of the participants had both access to paid and unpaid leave, and 6 had only unpaid leave available to them. Nine had taken leave within the past seven days.

Procedures

The test sessions were conducted in either the OSMR laboratory or via phone from May 13 – May 28, 2010. At the start of each session, the researcher explained the study’s purpose and procedures, and obtained informed consent from the participant. The ATUS leave module questions were administered over the phone; all responses were recorded on paper by the researcher. Following the ATUS interview, the researcher asked participants general debriefing questions about the survey and retrospective probes on the new questions. Test sessions lasted approximately 45 minutes on average, and participants who came into the OSMR laboratory were paid $40 upon completion of the study. (The test sessions done over the phone were not eligible for compensation.)

3. Phase 1 Results

The key findings from the first phase of testing are summarized below. These results are based on the mock ATUS interviews and debriefing sessions. With experimental, lab-based research, it is sometimes difficult to pick up the full range of target behaviors, and to get enough sample to answer the target questions about themselves. This is the main reason we conducted targeted recruiting from known persons with varying occupations – to ensure that we could get sufficient coverage of the concepts. We were successful at bringing in a relatively diverse set of wage and salary workers (see Table 1, Appendix A).

Leave Access

Q1

(IF MULTIPLE JOB HOLDER: Thinking about the job where you worked the most hours last week) Do you receive paid leave on your job?

Fourteen of 19 people reported receiving paid leave on their job. Participants seemed to understand “paid leave” as intended and the testing revealed no problems with this question.

Q2

I’m going to read you a list of types of paid leave that employers might offer. Please tell me which ones [your employer/the employer where you worked the most hours last week] offers:

1. Sick leave (for yourself or to care for other family members)

2. Vacation or annual leave

3. General or personal leave

4. Paid holidays

5. Family leave, including leave for birth or adoption of a child

This question evinced several substantive problems. Several participants reported having access to types of leave during the interview and later revealed in the debriefing questions that they did not have this type of leave after all. In at least one case, vacation and general or personal leave was mistakenly reported (when vacation was really the leave type available). Similarly, in some cases respondents reported having paid family leave, when in fact they would have to use sick leave to cover leave for birth of a child. Participants seemed to interpret this question as a reason for taking leave rather than the type of leave available.

Q2a-Q2b

Did I miss any other types of paid leave that are available to you in your (main) job? What other types of paid leave are available to you in your (main) job?

These questions were only asked in the cognitive interview to identify whether we were missing any other common leave types. Five participants reported having a paid time off (PTO) plan to this question. All 5 of the participants with PTO reported different answers to the types of leave available to them in Q2. One person reported having all but family leave available, and then volunteered that she had a PTO plan and could use leave for any reason. Another participant reported in Q2 that she had vacation leave, but not sick leave. It was later revealed that she had a PTO plan. Two participants with PTO reported that they had all types of paid leave.

When asked what other types of paid leave are available to them, other answers included professional leave, jury duty, and disability leave.

Q3

[In addition to your paid leave/In your main job,] Are you allowed to take time off from work without pay?

No substantive problems were found with this question. Eighteen respondents reported being able to take time off from work without pay and one respondent reported “don’t know.”

Q3a

I’m going to read a list of reasons why you might have to miss work. Assuming that you have received your employer’s approval, for each reason, please tell me whether or not you are able to take time off from work without pay.

[Read all, except items in parentheses]

1. Own illness or medical care

2. Illness or medical care of another family member

3. Childcare (other than for illness)

4. Eldercare (other than for illness)

5. Vacation or personal reasons

6. Birth or adoption of a child

Respondents generally understood this question as intended, however a few problems were found. Two participants did not seem to hear the term “without pay” and thought the question was asking about reasons for taking time off with pay. One respondent was not sure if the question was about what her employer offers her, or what the employer offers the rest of her company since she has different benefits from her coworkers.

Another problem uncovered during the interview itself and during the debriefing was that several respondents seemed uncertain about whether their employer offered unpaid leave for the listed reasons yet still reported “yes.” At least four participants reported that they knew of coworkers who had taken unpaid leave for various reasons, and so they assumed that this unpaid leave was available to them as well.

Several participants also did not understand the terms “childcare” and “eldercare” as intended. During the debriefing respondents were asked what types of examples would fall under each of the listed categories. At least nine participants reported caring for a sick child under “childcare” or caring for a sick parent under “eldercare.”

Leave Activity – Past Week

Q4_Intro –Q4

Now I want to ask you about your work time in the past week.

Did you take any time off from your job over the past seven days, that is from last [Fill DAY] through yesterday?

Eight participants reported taking time off in the last week. Most participants did not have a problem with this question, and during debriefing said that it was clear and easy to answer. However,

• Two participants did not understand that this question was referring to times when they had taken actual leave. These two participants reported “yes” to having taken leave, but later in the interview reported that their time off last week was actually a variation in their schedule, not actual leave taken. They worked the same total hours.

• This question set was not well-suited for people on long-term leave. The core ATUS already includes questions that identify respondents who did not work in the previous 7 days. It becomes repetitive to ask this group of workers the module questions about work activity in the past 7 days since they already have said they did not work at all.

• Two participants wondered why the reference period was only seven days and thought it should have been longer.

Q4a

Did you take any time off from your job yesterday?

One person thought it was redundant to ask this question since we had just asked about time off from work in the last seven days. No other problems were found with this question. Three respondents reported taking time off yesterday.

Q4b

[If Q4a = 1:] Did you use paid leave for any of the time that you took off from work yesterday?

[ Else:] Thinking about the most recent leave that you took from work during the past seven days…..did you use paid leave for that time off?

1. Yes, paid for all

2. Yes, paid for some

3. No, not paid

4. DK

5. REF

[Interviewer will probe to determine which “yes” – 1 or 2, above – is applicable.]

[GOTO Q4c]

This question seemed to work as intended. Four of the respondents who took leave in the past week reported taking paid leave, and two reported taking unpaid leave. One of the participants that mistakenly answered yes to Q4 reported that she took unpaid leave. Participants that did not have access to paid leave were not asked this question.

Q4c

What type(s) of leave did you use? (open-ended question, allow check all)

1. Sick leave

2. Vacation or annual leave

3. General or personal leave

4. Paid holidays

5. Family leave, including leave for birth or adoption of a child

6. Unpaid leave

7. Other

[GOTO Q4d]

This question was generally easy for respondents to answer. One respondent reported taking unpaid family leave (FMLA), two reported sick leave, one reported personal leave and another reported vacation leave.

Q4d

[IF Q4a = 1, then read:] What was the primary reason you had to take off from work yesterday?

[Else:] Again, thinking about the most recent leave you took in the last seven days, what was the primary reason that you had to take off from work?

[Read as necessary]

1. Own illness or medical care

2. Illness or medical care of another family member

3. Childcare (other than for illness)

4. Eldercare (other than for illness)

5. Vacation

6. Errands or other personal business

7. Birth or adoption of a child

8. Other

[If Q4d = 2, then goto Q4e; else goto Q4f]

The response options seem to cover the types of reasons that one needs to take leave from work. Two respondents reported birth of a child, three reported doctor’s appointments or medical care, one reported vacation and one person reported “Other – Didn’t feel like working.” For two of the respondents that reported “doctor’s appointment,” the interviewer had to follow up with the respondents to find out if the medical care was for them or for another family member.

The read as necessary response instructions were somewhat awkward for the interviewer and respondent. There was a pause from several respondents after asking this question. One respondent said it was a little sensitive to have open-ended responses, as we could get very detailed and unnecessary answers.

Q4e

Which member(s) of your family were ill?

[Select all that apply]

1. Spouse or partner

2. Child

3. Parent

4. Other relative

5. Other

[GOTO Q4f]

No respondents reported “illness or medical care of another family member” in Q4d and thus this question was not tested.

Q4f

[Altogether,] How long were you on leave [yesterday/for this reason]?

________ hours/days

[Convert reported minutes into fractions of hour, rounding to closest quarter hour]

[GOTO Q4g]

Respondents thought this question was easy to answer. Respondents reported various answers but almost all reported their answers in hours (e.g., 2, 1, 6.5, 8 hours). One respondent reported 3 months. Another respondent based it on the amount of time she left early (2 hours), and two respondents based it on the amount of time their shifts usually run (5 hours).

Q4g

Was any of the time you took off [yesterday/last week/for that most recent leave] covered by the Family Medical Leave Act or FMLA?

1. Yes

2. No

3. DK

4. REF

[GOTO Q5]

This question was problematic. Two people reported that they had taken FMLA leave. One respondent had, in fact, used FMLA, but the other incorrectly thought FMLA covered her doctor’s appointment. Many of the participants had heard of FMLA, but were not clear whether it was paid or unpaid leave, and were not sure of all the reasons one could use FMLA leave.

Schedule Adjustments

Q5

[If Q1 = 2 and Q3 = 2, then ask:] You said that you don’t have leave available (in your main job), but can you vary your work schedule if you needed to take time off work?

[Else if Q1 = 2 and Q3 = 1, then ask:] Can you vary your work schedule instead of taking unpaid leave?

[Else if Q1 = 1 and Q3 = 2, then ask:] Can you vary your work schedule instead of taking paid leave?

[Else if Q1 = 1 and Q3 = 1, then ask:] Can you vary your work schedule instead of taking leave?

1. Yes [goto Q5a]

2. No

3. DK [goto Q6_Intro]

4. REF

Thirteen respondents reported the ability to vary their work schedule instead of taking leave. When asked what “vary your work schedule” meant, respondents reported examples such as “working less one day and working more the next,” the “ability to adjust shifts to fit around appointments.” Respondents thought the phrase “instead of taking leave” meant things such as “reducing work hours,” or “time you took that won’t be used against you or won’t affect your leave.”

Q5a

[If Q1 = 2 and Q3 = 2, then ask:] In the last seven days – that is from [DAY] through yesterday – did you change your work schedule because you needed to take time off work?

[Else if Q4 = 1 AND (Q1 = 2 and Q3 = 1) OR (Q1 = 1 and Q3 = 2) OR (Q1 = 1 and Q3 = 1), then ask:] In the last seven days – that is from [DAY] through yesterday –did you change your work schedule rather than use additional leave?

[Else if Q4 = 2 AND (Q1 = 2 and Q3 = 1) OR (Q1 = 1 and Q3 = 2) OR (Q1 = 1 and Q3 = 1), then ask:] In the last seven days – that is from [DAY] through yesterday – did you change your work schedule rather than using leave?

1. Yes [If Q1 = 2 and Q3 = 2, then goto Q5c; else goto Q5b]

   

2. No

3. DK goto Q6_Intro

4. REF

Five respondents answered that they varied their work schedule in the past week. Three of the respondents who answered “yes” to Q4 (taken leave in the last seven days) said they had also varied their work schedule rather than using leave. One of these respondents thought this question was redundant. The debriefing revealed that two of the respondents were reporting the same time “taken off” for both Q4 and Q5a and that they had not used leave, but instead varied their work schedule (worked the same total hours over the week).

Q5b

[If Q4 = 1 AND (Q1 = 2 and Q3 = 1) OR (Q1 = 1 and Q3 = 2) OR (Q1 = 1 and Q3 = 1), then ask:] Why did you vary your schedule instead of taking additional leave?

[If Q4 = 2 AND (Q1 = 2 and Q3 = 1) OR (Q1 = 1 and Q3 = 2) OR (Q1 = 1 and Q3 = 1), then ask:] Why did you vary your schedule instead of taking leave?
[Read as necessary/Select all that apply]

1. Wanted to conserve my paid leave

2. Preferred to work rather than to take paid leave

3. Paid leave balance was too low

4. Employer did not approve use of available paid leave

5. Had too much work to be feel comfortable taking paid leave

6. Only unpaid leave was available

7. Other

[Goto Q5c]

This question generally worked as intended. However, the “read as necessary” options were awkward for both the respondent and interviewer as respondents were not sure how to answer. One respondent thought response options 1 and 2 were the same and didn’t see a difference.

Q5c

In what way did you change your schedule in the last seven days?
(Read aloud/Select all that apply)

1. Worked different hours than usual (but the same total)

2. Worked fewer hours than usual

3. Worked more hours than usual

4. Worked different days than usual (but same total)

5. Worked fewer days than usual

6. Worked more days than usual

7. Other

[GOTO Q5d]

Respondents seemed to understand this question. Because they knew exactly how they changed their schedules, respondents could easily identify which of the options applied to them.

Q5d

What was the main reason you changed your work schedule instead of taking leave?

[Read as necessary]

1. Own illness or medical care YES/NO/DK/REF

2. Illness or medical care of another family member YES/NO/DK/REF

3. Childcare (other than for illness) YES/NO/DK/REF

4. Eldercare (other than for illness) YES/NO/DK/REF

5. Vacation or personal reasons YES/NO/DK/REF

6. Birth or adoption of a child YES/NO/DK/REF

[GOTO Q6_Intro]

Three of the five participants that (correctly) reported varying their work schedule in the past seven days reported a doctor’s appointment (own illness or medical care) as the reason for varying their work schedule. Again, the response option instructions (read as necessary) were somewhat awkward for the interviewer. The interviewer had to follow up with two respondents to find out if the medical care was for them or for another family member.

Non-use of leave

Q6_Intro-Q6

The next few questions are about times when you may have needed to take off from work but could not.

During the past seven days, that is from last [FILL DAY] through yesterday, were there situations in which you needed to take off from work but did not?

1. Yes (goto Q6a)

2. No (END)

This question generally worked as intended. Two participants (both teachers) reported a time during the last seven days when they needed to take off from work but could not. Two other respondents laughed and thought of times when they wanted to take off work but didn’t; however, they ultimately answered “no” to this question.

Q6a

Why did you need to take off work? (Select all that apply)

1. Own illness or medical care

2. Illness or medical care of another family member

3. Childcare (other than for illness)

4. Eldercare (other than for illness)

5. Vacation

6. Errands or other personal business

7. Birth or adoption of a child

8. Other

[If Q1 = 2 and Q3 = 2, then END; else goto Q6b]

This question was only tested on two respondents but revealed no problems. One respondent reported a doctor’s appointment for herself. Another respondent reported needing to take time off to attend to errands and household management.

Q6b

Why did you decide not to take leave?

[Read as necessary/Select all that apply]

1. Too much work

2. Wanted to save leave

3. Leave was denied

4. No leave available

5. Fear of job loss or other negative employment-related consequence

6. Could not afford the loss in income

7. Other

The read as necessary response options were somewhat awkward for this question. One respondent at first reported that she had too many responsibilities at work and was encouraged not to take off. Because this response did not fit neatly into the response options, the interviewer read aloud the response options and the respondent reported “fear of job loss or negative employment-related consequence” since she said she didn’t want to deal with her boss who would reprimand her if she had taken off that time. The other respondent initially reported that he did not take leave because he had responsibilities at work. Later it was revealed that he did not take the time off because he had already used his allotted personal days.

4. Phase 1 Summary

In summary, the results of this study showed that the majority of the ATUS leave module questions worked well in our cognitive test session. Participants generally understood the concepts of paid and unpaid leave. However, respondents seemed to confuse the types of paid leave with the reasons for taking paid leave. The module failed to cleanly capture paid time off (PTO) leave plans, a type of leave plan that 5 of the 19 participants in this test session were offered.

Additionally, the module was able to capture leave activity over the past week but the questions need some modifications to work better. First, the leave activity questions were not well-suited for people on long-term leave. This was especially true for the questions about schedule adjustment and the need to take time off over the past 7 days. Since respondents are already asked about their work schedule in the last 7 days during the basic ATUS interview and their reason for absence, it becomes repetitive to ask those on long-term leave if they worked or altered their schedule during the module. Second, “taking time off” was unclear to some and in two instances was mistakenly confused with times when the respondents varied their schedule but worked the same total hours.

5. Recommendations

Paid leave questions

1. Add a question to collect the reasons for taking time off (since we collect reasons for unpaid) before asking about the types of paid leave. This may help respondents separate the reasons for taking time off from the types of leave available to them.

2. Add a question to collect PTO leave. Five of the participants had PTO plans, suggesting that this is a common type of plan that the current question does not collect.

Unpaid leave questions

3. Adding reasons for taking paid leave earlier in the survey will help make a distinction that Q3a is asking for reasons for taking unpaid leave. Interviewers should also be instructed to emphasize “without pay” and that the question is about what employers offer to the respondent, not the company as a whole, if asked.

4. Add a question about whether the respondent has ever taken unpaid leave for any of the reasons they identified in Q2. This will act as a flag to determine the accuracy of responses to Q2.

5. The question about FMLA should be dropped. First, the question assumes that the respondent knows what FMLA leave is. Many respondents did not know what FMLA was, although some thought they did, when in fact they did not. Second, the question cannot be used to measure awareness or compliance and has little analytical value for this survey.

Schedule Adjustments

6. We recommend adding a question about whether a respondent can vary their location in addition to their work schedule since this was brought up by a respondent as a way that the can make work adjustments. Further, we recommend adding a question to find out how one can vary their schedule (hours, days, and/or location).

Response options

7. Instead of reading "childcare" and "eldercare" as possible reasons for taking leave, the interviewers should read "childcare, other than illness" and "eldercare, other than illness" so that these reasons are more clearly defined. This occurs in questions Q2a, Q3a, Q3c, Q4c, Q4b, Q5e, and Q6a (see Appendix B).

8. Read aloud answer options to make the questions less sensitive and awkward to read; Q4c, Q4h, Q5c, Q5e, Q6a, and Q6b would be read aloud (see Appendix B).

9. Response options to similar questions should be consistent across questions. The questions about reasons for taking leave (Q2a, Q3a, Q3c, Q4c, Q4b, Q5e, and Q6a) and decisions not to take leave (Q5c and Q6b) would have consistent response options.

Skip Patterns

10. Check and modify skip patterns to

    1. Ensure that people who report being absent from work during the Labor Force questions in the basic ATUS do not receive unnecessary questions about leave activity in the past seven days.

    2. Limit conflicting data (types of leave used should match the types of leave available to the respondent)

6. Phase II Results

The key findings from the second phase of testing are summarized below. The recommendations from the first phase of testing were implemented for more clarity and to make the survey flow better (minimize awkwardness) for both respondents and interviewers. Five interviews were conducted during the second phase. The participants of the second phase included a retail store manager, a security officer, one lawyer, one IT consultant, and one part-time accountant. The results of phase II testing of the changes, while limited because of the small sample size, are summarized below.

Types of Leave

Q2a

I’m going to read you a list of reasons why you might have to miss work. For each reason, please tell me if you are able to take paid leave in your [current/main] job.

1. Own illness or medical care YES/NO/DK/REF

2. Illness or medical care of another family member YES/NO/DK/REF

3. Childcare, other than for illness YES/NO/DK/REF

4. Eldercare, other than for illness YES/NO/DK/REF

5. Vacation YES/NO/DK/REF

6. Errands or personal reasons YES/NO/DK/REF

7. Birth or adoption of a child YES/NO/DK/REF

This question was added to collect information about the reasons for which workers can take paid leave and to help distinguish between reasons and types of leave. It is asked before the question that collects information about which types of leave are available to the respondent. Four of the five additional participants had access to paid leave and thought the question was clear.

Q2b-Q2c.

Q2b. Does your employer offer a single type of paid leave that can be used for multiple purposes, such as sick leave, vacation, or personal leave – these plans are sometimes called consolidated leave plans or PTOs?

1. Yes

2. No

3. Don’t Know

[if Q2b = 1, then goto Q2c; else goto Q2d]

Q2c. Does your employer also offer separate paid leave for… [Read all]

1. Holidays Yes/No/DK/REF

2. Maternity or paternity leave Yes/No/DK/REF

3. Additional sick leave, not already included in your PTO

This question worked as intended during the second round of testing. Of the four respondents who had access to paid leave, only one had a Paid Time Off (PTO) plan. The three remaining respondents who received this question answered correctly that they did not have a PTO plan.

Q2d

Employers offer different types of paid leave plans. Does [your employer/the employer where you worked the most hours last week] offer you separate paid….

[Read all]

1. Sick leave (for yourself or to care for other family members) Yes/No/DK/REF

2. (Paid) Vacation or annual leave Yes/No/DK/REF

3. (Paid) Personal leave Yes/No/DK/REF

4. Paid holidays Yes/No/DK/REF

5. (Paid) Maternity or paternity leave Yes/No/DK/REF

No problems were found with this question during phase II testing. Adding a list of reasons why people can take paid leave (Q2a) before asking about the types of paid leave available seemed to address the confusion about types of paid leave available versus reasons for taking paid leave that was identified during phase I.

Q3_Intro-Q3c

Now I’m going to ask you about unpaid leave.

Q3. [In addition to your paid leave/In your main job,] are you allowed to take time off from work without pay?

1. Yes (goto Q3a)

   

2. No

3. DK (if Q1=2 and Q3=2, goto Q5; else if LF1 = 1, goto Q4_Intro1;

else if LF4 = 1, goto Q4_Intro2)

4. REF

Q3a. I’m going to read a list of reasons why you might have to miss work. Assuming that you have received your employer’s approval, for each reason, please tell me whether or not you are able to take time off from work without pay.

[Read all]

1. Own illness or medical care YES/NO/DK/REF

2. Illness or medical care of another family member YES/NO/DK/REF

3. Childcare, other than for illness YES/NO/DK/REF

4. Eldercare, other than for illness YES/NO/DK/REF

5. Vacation YES/NO/DK/REF

6. Errands or personal reasons YES/NO/DK/REF

7. Birth or adoption of a child YES/NO/DK/REF

Q3b. In your [current/main] job, have you ever taken unpaid leave for [this reason/these reasons]?

1. Yes

2. No

3. Don’t Know

4. Refused

Q3c. Which ones? That is, for what reasons have you taken unpaid leave in your [current/main] job? [Highlight only those response options that were reported in Q3a; gray out remainder. Read highlighted options as necessary]

1. Own illness or medical care

2. Illness or medical care of another family member

3. Childcare, other than for illness

4. Eldercare, other than for illness

5. Vacation

6. Errands or personal reasons

7. Birth or adoption of a child

[If LF1 = 1, then goto Q4_Intro1; else if LF4 = 1, then goto Q4_Intro2]

Adding reasons for taking paid leave earlier in the survey helped to make a distinction that this question is asking about reasons for taking unpaid leave. Respondents were also able to answer if they had ever taken unpaid leave in their current job and the reason for doing so. One respondent did struggle with the concept of taking leave without pay, saying that he didn’t understand why someone would take leave if he or she wasn’t going to get paid for it. The testers do not think that this question warrants additional changes. However, it may be helpful to have an interviewer definition of what “time off from work without pay” means (e.g., ability to take time off from work and still have a job when you come back, usually occurs when no paid leave is available).

Leave Activity

Q4_Intro1-Q4

Now I want to ask you about your work time in the past week.

Q4. Did you take any paid or unpaid leave from your (main) job over the past seven days, that is from last [Fill DAY] through yesterday?

1. Yes (goto Q4a)

   

2. No

3. DK (goto Q5)

4. REF

Respondents seemed to understand that this question was referring to times when they took actual leave rather than times when they varied their schedule. Only one respondent reported taking leave in the past seven days, and he thought the follow-up questions regarding his leave activity (reasons and time taken) were clear.

Schedule Adjustments

Q5

Can you vary your work schedule or location instead of taking leave?

5. Yes [goto Q5a]

   

6. No

7. DK [goto Q6_Intro]

8. REF

The addition of the location option in the schedule adjustment questions worked well. Respondents seemed to understand “varying your work schedule or location” as intended.

One item pointed out during phase II testing was the awkward transition from the leave activity questions straight into the schedule adjustment questions. Adding an introductory sentence would create a smoother transition.

Q5a

How can you vary your work schedule or location? Can you change:

1. The days that you work? YES/NO/DK/REF

2. The hours that you work? YES/NO/DK/REF

3. The location where you work? YES/NO/DK/REF

[if LF4 = 1, END; else goto Q5b]

Respondents thought this question was clear. Three respondents were able to vary the days they work, all five respondents were able to vary the hours they work, and one respondent was able to vary the work location.

Read Aloud Response Options

During phase II, reading aloud response options to the reasons for paid leave or schedule adjustments was tested for length and repetitiveness. Reading aloud the response options seemed to help respondents answer the question with less awkwardness as they had a better idea of how to respond. Reading aloud the response options did not add a significant amount of time to conduct the survey and seemed to take about as long as not reading them aloud since respondents did not have to think about how to answer the question. One respondent felt the survey was a bit repetitive, but thought the difference between questions was clear.

7. Additional changes not tested

After additional expert review, inconsistencies in the reference period for leave activity were identified as something that would make it difficult to provide consistent estimates (some respondents were asked about the leave they took yesterday and others were asked about their most recent leave period in the past 7 days). The recommendation was made to ask more uniform questions of all respondents; the leave access questions were thus modified so that respondents are asked about their longest period of leave over the past 7 days. Further, respondents are then asked if they used additional leave for any other reason to find out if there were multiple periods of leave over the past 7 days. Two questions collect information about leave taken yesterday: one asks the respondent if she took leave from work yesterday and the second collects information about the reason for taking leave. The questions about yesterday will allow researchers to examine the relationship between time use and leave.

8. Conclusion

The results of this study showed the potential to capture information on variations in access to leave, use of leave, and schedule flexibility. While this study is limited by a small sample size and some populations could not be tested, in general, respondents understood the questions as intended. Further, the questions were clear and respondents’ interpretations of the phrases and concepts were consistent overall.

The cognitive tests also highlighted several areas to check before administering to the general public. Because respondents will receive different questions based on their labor force status and answers to various questions, care should be taken to thoroughly test the instrument to ensure all skip patterns are working correctly. This includes disabling some response options in later questions based on answers to earlier questions (for example, if a respondent receives PTO and no additional paid leave, response options other than PTO and unpaid leave should be disabled for questions about types of leave used in the past seven days).

Finally, the leave module took a reasonable amount of time to administer. On average, the questions took about 5 minutes to complete.

Appendix A. Table 1. List of Study Participants

	Demographics					Leave available/taken
Respondent #	Gender	Age	Occupation	Education	HH Size	Paid leave	Unpaid leave	Leave taken	Can vary sched
1	F	40	Pharmaceutical Sales Rep		5	Y	Y	N	Y
2	F	35	Director Non-Profit	Master's	3	Y	Y	Y	Y
3	F	32	Economist - Fed Employee	Master's	4	Y	Y	Y	Y
4	F	46	Economist & Researcher, Consulting	PhD	1	Y	Y	Y	Y
5	M	59	Ski School Director, seasonal full-time	Bachelor's	2	N	Y	N	N
6	F	54	Nurse	Bachelor's and RN	2	Y	Y	N	Y
7	F	33	Professional full-time	Master's	2	Y	Y	N	Y
8	M	53	Professional, works 30-32 hours/week at University library		1	Y	Y	Y	Y
9	M	56	Merchandiser for nursery	Some College	1	N	Y	N	Y
10	M	61	Administrative staff for church, part-time	Some College	2	Y	Y	N	N
11	F	37	Office Administration, part-time, student	Master's	2	N	Y	Y	Y
12	F	44	Office administration, full-time		3	Y	Y	Y	Y
13	M	44	Multiple job holder: federal employee and teacher	PhD	3	Y	Y	Y	Y
14	F	39	Food Service worker, multiple jobholder	Some College	3	N	Y	Y	N
15	F	34	IT Consultant, works 32 hours/week, 4 days/week, paid hourly	BS	4	Y	Y	N	Y
16	F	29	High School special education teacher	MS	2	Y	Y	N	N
17	M	36	Multiple job holder: public school science teacher; waiter	MS	2	Y	DK	N	N
18	M	24	Works in sales, usually 30-35 hours/week	Bachelor's	3	N	Y	N	Y
19	M	34	Fire fighter for county government; 56 hours/week; 24 hour shifts; 9-day 'weeks'	MS	4	Y	Y	N	N
20	M	47	IT Consultant, works full-time	MS	4	Y	Y	Y	Y
21	M	29	Store manager	Bachelor's	1	Y	Y	N	Y
22	M	52	Accountant, Part-time	BA	1	N	Y	N	Y
23	F	33	Security-officer, Full-time	AA	4	Y	Y	N	Y
24	M	28	Lawyer, full-time	JD	2	Y	Y	N	Y

Appendix B. Recommended Final Questionnaire

ATUS Leave Module – BLS Proposal

Note: The leave questions only will be asked of persons who are coded as employed wage and salary workers in the ATUS. Questions LF1-LF5 is a partial list of the labor force questions currently asked in the ATUS and will not be re-asked in the module.

Items appearing in bold will be read aloud.

LF1. In the LAST SEVEN DAYS, did you do ANY work for either pay or profit?”

• If “yes”, ask LF2: “In the LAST SEVEN DAYS, did you have more than one job, including part-time, evening, or weekend work?”

  • If “yes”, ask LF3a: “How many hours do you USUALLY work at your main job? By main job we mean the one at which you usually work the most hours.”

• If "no", ask LF3b: "How many hours do you USUALLY work at your job?"

• If LF1 ="no", ask LF4: "In the LAST SEVEN DAYS, did you have a job either full time or part time? Include any job from which you were temporarily absent."

  • If "yes", ask LF5: "What was the main reason you were absent from work during the LAST SEVEN DAYS?"

    • On layoff

    • Slack work/business conditions

    • Waiting for a new job to begin

    • Vacation/personal days

    • Own illness/injury/medical problems

    • Childcare problems

    • Other family/personal obligation

    • Maternity/paternity leave

    • Labor dispute

    • Weather affected job

    • School/training

    • Civic/military duty

    • Does not work in the business

    • Other

  • If “yes,” ask LF2: "In the LAST SEVEN DAYS, did you have more than one job, including part-time, evening, or weekend work?"

    • If "yes," ask LF3a: "How many hours per week do you USUALLY work at your main job? By main job we mean the job at which you usually work the most hours."

• If "no,” ask LF3b: "How many hours do you USUALLY work at your job?"

The next few questions are about paid and unpaid leave from a job. [Goto Q1.]

Leave access

Q1. (IF MULTIPLE JOB HOLDER: Thinking about the job where you worked the most hours last week) Do you receive paid leave on your job?

1. Yes Go to Q2a

   

2. No

3. DK Go to Q3

4. Ref

Q2a. I’m going to read you a list of reasons why you might have to miss work. For each reason, please tell me if you are able to take paid leave in your [current/main] job.

[Read all]

1. Own illness or medical care YES/NO/DK/REF

2. Illness or medical care of another family member YES/NO/DK/REF

3. Childcare, other than for illness YES/NO/DK/REF

4. Eldercare, other than for illness YES/NO/DK/REF

5. Vacation YES/NO/DK/REF

6. Errands or personal reasons YES/NO/DK/REF

7. Birth or adoption of a child YES/NO/DK/REF

Q2b. Does your employer offer a single type of paid leave that can be used for multiple purposes, such as sick leave, vacation, or personal leave – these plans are sometimes called consolidated leave plans or PTOs?

1. Yes

2. No

3. Don’t Know

[if Q2b = 1, then goto Q2c; else goto Q2d]

Q2c. Does your employer also offer separate paid leave for… [Read all]

1. Holidays Yes/No/DK/REF

2. Maternity or paternity leave Yes/No/DK/REF

3. Additional sick leave, not already included in your PTO

[goto Q3]

Q2d. Employers offer different types of paid leave plans. Does [your employer/the employer where you worked the most hours last week] offer you separate paid….

[Read all]

1. Sick leave (for yourself or to care for other family members) Yes/No/DK/REF

2. (Paid) Vacation or annual leave Yes/No/DK/REF

3. (Paid) Personal leave Yes/No/DK/REF

4. Paid holidays Yes/No/DK/REF

5. (Paid) Maternity or paternity leave Yes/No/DK/REF

[Goto Q3_Intro]

Q3_Intro. Now I’m going to ask you about unpaid leave. [Goto Q3]

Q3. [In addition to your paid leave/In your main job,] are you allowed to take time off from work without pay?

1. Yes (goto Q3a)

2. No

3. DK (if Q1=2 and Q3=2, goto Q5_intro; else if LF1 = 1, goto Q4_Intro1;

else if LF4 = 1, goto Q4_Intro2)

4. REF

Q3a. I’m going to read a list of reasons why you might have to miss work. Assuming that you have received your employer’s approval, for each reason, please tell me whether or not you are able to take time off from work without pay.

[Read all]

1. Own illness or medical care YES/NO/DK/REF

2. Illness or medical care of another family member YES/NO/DK/REF

3. Childcare, other than for illness YES/NO/DK/REF

4. Eldercare, other than for illness YES/NO/DK/REF

5. Vacation YES/NO/DK/REF

6. Errands or personal reasons YES/NO/DK/REF

7. Birth or adoption of a child YES/NO/DK/REF

[If there are any “Yes” responses to Q3a, goto Q3b; else if LF1 = 1, goto Q4_Intro1;

else if LF4 = 1, goto Q4_Intro2]

Q3b. In your [current/main] job, have you ever taken unpaid leave for [this reason/these reasons]?

1. Yes

2. No

3. Don’t Know

4. Refused

[if there are multiple “Yes” responses to Q3a AND Q3b = 1, goto Q3c; else if LF1 = 1, goto Q4_Intro1; else if LF4 = 1, goto Q4_Intro2]

Q3c. Which ones? That is, for what reasons have you taken unpaid leave in your [current/main] job? [Highlight only those response options that were reported in Q3a; gray out remainder. Read highlighted options/Select all that apply]

1. Own illness or medical care

2. Illness or medical care of another family member

3. Childcare, other than for illness

4. Eldercare, other than for illness

5. Vacation

6. Errands or personal reasons

7. Birth or adoption of a child

[If LF1 = 1, then goto Q4_Intro1; else if LF4 = 1, then goto Q4_Intro2]

Leave activity – Past Week

Q4_Intro1. Now I want to ask you about your work time in the past week. [goto Q4]

Q4. Did you take any paid or unpaid leave from your (main) job over the past seven days, that is from last [Fill DAY] through yesterday?

1. Yes (goto Q4a)

2. No

3. DK (goto Q5_intro)

4. REF

Q4a. In the past seven days, how many hours of leave did you take in total?

________ hours

Allow DK/REF

[[if Q1=1, then goto Q4b; else goto Q4c]

Q4_Intro2. Earlier in the interview, you said that you did not work in the last seven days. How long have you been absent from your [current/main] job?

________ days/weeks/months

Allow DK/REF

[if Q1=1, then goto Q4b; else goto Q4c]

Q4b. Did you use paid leave for any of the time that you took off from work in the past seven days?

1. Yes, paid for all

2. Yes, paid for some

3. No, not paid

4. DK

5. REF

[Interviewer will probe to determine which “yes” – 1 or 2, above – is applicable.]

[GOTO Q4c]

Q4c. Thinking about your longest period of leave in the last seven days, what was the main reason you had to take off from work? [Read all]

1. Own illness or medical care

2. Illness or medical care of another family member

3. Childcare, other than for illness

4. Eldercare, other than for illness

5. Vacation

6. Errands or personal reasons

7. Birth or adoption of a child

8. Other

[If Q4c = 2, then goto Q4d;

else if (LF4=1 and Q4b=3), goto Q5_intro;

else if (LF1=1 and Q4b=3), goto Q4f;

else goto Q4e]

Q4d. Which member(s) of your family were ill?

[Select all that apply]

1. Spouse or partner

2. Child

3. Parent

4. Other relative

5. Other

[If (LF4=1 and Q4b=3), goto Q5_intro;

else if (Q4=1 and Q4b=3), goto Q4f;

else if LF1=1 and [(Q1=2 and Q3=2) or (Q1=2 and Q3=1)], goto Q4f;

else goto Q4e]

Q4e. What type(s) of leave did you use for this reason? (open-ended question, select all that apply; allow interviewers to specify other leave types if reported)

1. Sick leave (for yourself or to care for other family members)

2. (Paid) Vacation or annual leave

3. (Paid) Personal leave

4. Paid holidays

5. (Paid) Maternity or paternity leave

6. PTO or consolidated leave

7. Unpaid leave

8. Other, specify ___________________________

[GOTO Q4f]

Q4f. You said you were on leave for (the) [fill reason from Q4c]. In the past seven days, did you take additional leave for any other reason?

1. Yes

2. No

3. DK

4. REF

[GOTO Q4g]

Q4g. Did you take any paid or unpaid leave from your job yesterday?

1. Yes

2. No

[if Q4f=1 and Q4g=1 then goto Q4h; else goto Q5_intro]

Q4h. What was the main reason you had to take off from work yesterday? [Read all]

1. Own illness or medical care

2. Illness or medical care of another family member

3. Childcare, other than for illness

4. Eldercare, other than for illness

5. Vacation

6. Errands or personal reasons

7. Birth or adoption of a child

8. Other

[GOTO Q5_intro]

Schedule Adjustments

Q5_Intro. Now I'm going to ask you about schedule adjustments. [goto Q5]

Q5. [If Q1 = 2 and Q3 = 2, then ask:] You said that you don’t have leave available (in your main job), but can you vary your work schedule or location if you needed to take time off work?

[Else if Q1 = 2 and Q3 = 1, then ask:] Can you vary your work schedule or location instead of taking unpaid leave?

[Else if Q1 = 1 and Q3 = 2, then ask:] Can you vary your work schedule or location instead of taking paid leave?

[Else if Q1 = 1 and Q3 = 1, then ask:] Can you vary your work schedule or location instead of taking leave?

1. Yes [goto Q5a]

2. No

3. DK [goto Q6_Intro]

4. REF

Q5a. How can you vary your work schedule or location? Can you change: [Read all]

1. The days that you work? YES/NO/DK/REF

2. The hours that you work? YES/NO/DK/REF

3. The location where you work? YES/NO/DK/REF

[if LF4 = 1, END; else goto Q5b]

Q5b. [If Q1 = 2 and Q3 = 2, then ask:] In the last seven days – that is from [DAY] through yesterday – did you change your work schedule or location because you needed to take time off work?

[Else if Q4 = 1 AND [(Q1 = 2 and Q3 = 1) OR (Q1 = 1 and Q3 = 2) OR (Q1 = 1 and Q3 = 1)], then ask:] In the last seven days – that is from [DAY] through yesterday –did you change your work schedule or location rather than use additional leave?

[Else if Q4 = 2 AND [(Q1 = 2 and Q3 = 1) OR (Q1 = 1 and Q3 = 2) OR (Q1 = 1 and Q3 = 1)], then ask:] In the last seven days – that is from [DAY] through yesterday – did you change your work schedule or location rather than using leave?

1. Yes [If Q1 = 2 and Q3 = 2, then goto Q5d; else goto Q5c]

2. No

3. DK goto Q6_Intro

4. REF

Q5c. [If Q4 = 1 AND [(Q1 = 2 and Q3 = 1) OR (Q1 = 1 and Q3 = 2) OR (Q1 = 1 and Q3 = 1)], then ask:] Why did you decide not to take additional leave?

[If Q4 = 2 AND [(Q1 = 2 and Q3 = 1) OR (Q1 = 1 and Q3 = 2) OR (Q1 = 1 and Q3 = 1)], then ask:] Why did you decide not to take leave instead?
[Read aloud/Select all that apply]

1. Too much work

2. Wanted to save leave

3. Leave was denied

4. Did not have enough leave

5. Fear of job loss or other negative employment-related consequence

6. Could not afford the loss in income

7. Other

[Goto Q5d]

Q5d. In what way did you change your schedule in the last seven days?
(Highlight response options based on responses to Q5a:

if Q5a-1 = Yes, highlight options 4,5,and 6;

if Q5a-2 = Yes, highlight options 1,2, and 3;

if Q5a-3 = Yes, highlight option 7; gray out other response options.

[Read highlighted options /Select all that apply]

1. Worked different hours than usual, but the same total

2. Worked fewer hours than usual

3. Worked more hours than usual

4. Worked different days than usual, but same total

5. Worked fewer days than usual

6. Worked more days than usual

7. Worked at a different location than usual

8. Other

[GOTO Q5e]

Q5e. Why did you need to change your work schedule or location [instead of taking leave]? [Bracketed phrase conditional on whether they have leave]

[Read aloud/Select all that apply]

1. Own illness or medical care

2. Illness or medical care of another family member

3. Childcare, other than for illness

4. Eldercare, other than for illness

5. Vacation

6. Errands or personal reasons

7. Birth or adoption of a child

[GOTO Q6_Intro]

Non-use of leave

Q6_Intro. The next few questions are about times when you may have needed to take off from work but could not. [goto Q6]

Q6. During the past seven days, that is from last [FILL DAY] through yesterday, were there situations in which you needed to take off from work but did not?

1. Yes (goto Q6a)

2. No (END)

Q6a. Why did you need to take off work? [Read aloud/Select all that apply]

1. Own illness or medical care

2. Illness or medical care of another family member

3. Childcare, other than for illness

4. Eldercare, other than for illness

5. Vacation

6. Errands or personal reasons

7. Birth or adoption of a child

8. Other

[If Q1 = 2 and Q3 = 2, then END; else goto Q6b]

Q6b. Why did you decide not to take leave?

[Read aloud/Select all that apply]

1. Too much work

2. Wanted to save leave

3. Leave was denied

4. Did not have enough leave

5. Fear of job loss or other negative employment-related consequence

6. Could not afford the loss in income

7. Other

[END]

1 The authors thank Stella Godbolt (OSMR) for her assistance with project recruitment.