Attachment F Brandon
Kopp July 8,
2011
cps
disability Supplement questions
summary of findings from
cognitive interviews
cps disability Supplement questions
summary of findings from cognitive interviews
Minor wording changes are proposed for Questions 1, 3, 4, 7, 8, 9, 10, 11, 16, and 19.
Major wording or question structure changes are proposed for Questions 4, 6, 11, and 20.
Elimination of a question is proposed for Questions 4, 15, and 20.
The addition of conditional fills within a Blaise instrument is proposed for Questions 3, 5, and 19.
Background
The Current Population Survey (CPS) is a nationally-representative monthly sample survey of households that provides information on labor force activity in the United States. The CPS is the source of the monthly national unemployment rate, as well as other rich demographic details, including age, sex, race, Hispanic origin, educational attainment, marital status and family attributes, foreign-born status, veteran status, and other demographic characteristics.
Pursuant to Executive Order 13078 titled “Increasing the Employment of Adults with Disabilities,” the Bureau of Labor Statistics (BLS) began measuring disability status as part of the CPS in June 2008. Currently, the BLS uses a set of six questions, also used in the American Community Survey (ACS), to determine disability status. Data from the CPS have shown that people with disabilities have markedly different levels of labor market participation than those without disabilities (21.8% and 70.1%, respectively1). While these data show that there are differences in the employment situations of individuals with disabilities, they do not offer any information on what labor market challenges may be leading to these differences.
With this in mind, the Office of Disability Employment Policy (ODEP) and the BLS developed a set of twenty questions (See Appendix B) that will be included as a supplement to the CPS survey in May 2012. These questions, referred to collectively as the CPS Disability Supplement, are meant to address the following objectives:
Learn more about the low labor force participation rates for people with disabilities,
Better understand the use of and satisfaction with current programs designed to prepare people with disabilities for employment,
Learn more about the work history of people with disabilities,
Identify the different types of barriers to employment people with disabilities experience,
Determine the types of workplace accommodations that assist people with disabilities,
Measure the use of financial assistance programs among those with disabilities and whether these programs affect the likelihood of working.
study PURPOSE
The purpose of this study was to determine whether (1) the questions measure what they were intended to measure, (2) respondents can quickly and accurately answer these questions in the context of a CPS interview, and (3) respondents can accurately answer these question on behalf of other members of their household (these types of responses are referred to as proxy responses and the people answering on behalf of other household members are referred to as proxy respondents or proxies).
PARTICIPANTS
Whether a question from the CPS Disability Supplement was asked was contingent upon the labor force and disability status of the participant and the members of their household. Thus, participants in this study were selectively recruited to vary on these dimensions. This led to six groups: two (Disability Status: Disabled, Non-Disabled) X three (Labor Force Status: Employed, Unemployed, Not in the Labor Force). Participants were asked about themselves and up to two other members of their household (answers for other household members are referred to as proxy through the remainder of the document).
The participants for the Disabled Group were recruited by distributing flyers to support groups for people with disabilities. The Non-Disabled Group participants were recruited through the participant pool maintained by the Office of Survey Methods Research (OSMR) at the BLS. Participants were paid $40 for the interview.
Twenty-three participants were interviewed for this study. Their average age was 50 years old. Sixteen of these participants had at least one disability as determined by the CPS disability status questions and seven did not report having a disability2. Eight participants reported being employed during the week prior to their interview, three reported being unemployed, and twelve reported that they were not in the labor force. Participant characteristics and their distribution across groups are shown in Table 1.
Table 1: Number of participants by disability and labor force status and relationship to participant.
|
|
Self |
Proxy |
Total |
Disabled |
Employed |
6 |
1 |
7 |
Unemployed |
1 |
1 |
2 |
|
Not in the Labor Force |
9 |
3 |
12 |
|
Non-Disabled |
Employed |
2 |
4 |
6 |
Unemployed |
2 |
1 |
3 |
|
Not in the Labor Force |
3 |
1 |
4 |
|
Total |
23 |
11 |
|
|
Table 2 shows that participants reported a range of disabilities (for themselves and other household members) that included vision impairments (either full or partial blindness), physical disabilities that limited their ability to walk or climb stairs, and mental or emotional disabilities.
Table 2: Number of participants by disability.
NOTE: This table includes self and proxy responses. Also, some participants reported more than one disability.
Disability |
Frequency |
Hearing |
0 |
Seeing |
10 |
Remembering, Concentrating |
9 |
Walking or Climbing Stairs |
12 |
Dressing or Bathing |
0 |
Doing Errands |
7 |
PROCEDURE
I, Brandon Kopp, conducted individual interviews. Each session lasted approximately one hour. When participants arrived, they were escorted to the lab, where they signed the informed consent form. Participants were, then, administered an abbreviated Computer Assisted Personal Interview (CAPI) version of the CPS including the proposed Disability Supplement questions (See Appendices A & B). They were instructed to answer each question as if they were being interviewed for the survey. Following the administration of the survey, participants were debriefed and asked retrospective probes about question meaning and some of the major concepts studied in the survey. The debriefing only covered questions from the CPS Disability Supplement.
As
stated earlier, the set of Disability Supplement questions
participants received, for themselves and other household members,
was contingent upon their answers to the labor force and disability
status questions. Table 3 shows which questions participants received
based on labor force and disability group. Black shaded boxes with
underlined numbers (e.g., 1)
denote questions that were asked of everyone in that category. Grey
shaded boxes with numbers that have a strikethrough (e.g., 7)
denote questions that are contingent upon answers to previous
supplement questions. For example, participants that were disabled
and employed could receive between 9 and 17 questions while
participants that were not disabled and were unemployed may receive
only 1 or 2 questions. Participants were only debriefed about
questions they received during the survey portion of the testing.
The Disability Supplement questions were administered in a serial fashion. That is, I read through the series of questions for the participant (based on his or her group), then read through the questions again for the next household member (based on his or her group), and so on.
Table 3: Disability Supplement questions asked by disability and labor force status.
Disabled |
Employed |
1 |
2 |
3 |
4 |
5 |
6 |
|
8 |
|
|
11 |
12 |
|
|
|
|
17 |
18 |
19 |
|
Unemployed |
1 |
2 |
|
4 |
|
6 |
|
8 |
9 |
10 |
11 |
12 |
13 |
14 |
15 |
16 |
17 |
18 |
19 |
|
|
Not In The Labor Force |
1 |
2 |
|
4 |
|
6 |
|
8 |
9 |
10 |
11 |
12 |
13 |
14 |
15 |
16 |
17 |
18 |
19 |
|
|
Non-Disabled |
Employed |
1 |
2 |
3 |
4 |
5 |
6 |
7 |
8 |
|
|
11 |
12 |
|
|
|
|
17 |
18 |
19 |
|
Unemployed |
1 |
2 |
3 |
4 |
5 |
6 |
7 |
8 |
9 |
10 |
11 |
12 |
13 |
14 |
15 |
16 |
17 |
18 |
19 |
|
|
Not In The Labor Force |
1 |
2 |
3 |
4 |
5 |
6 |
7 |
8 |
9 |
10 |
11 |
12 |
13 |
14 |
15 |
16 |
17 |
18 |
19 |
|
STATEMENT OF LIMITATIONS
The cognitive interview approach seeks to develop insight and direction rather than quantitatively precise or absolute measures. Because of the limited number of participants and the restricted recruiting, this research must be considered as qualitative in nature.
The recommendations suggested below are based on interviewer observations and participants’ answers to questions during the debriefing. Recommendations should be accepted with the understanding that, without additional testing, there is no way to verify the suggested change will fix the problem or whether the change will cause new problems. If time allows, another round of testing should be done to verify the efficacy of the recommended changes.
The context of a cognitive interview is different from that of a regular CAPI or CATI interview. First, participants for this study were purposefully selected based on their labor force and disability status and were asked to “opt-in.” This is opposed to the way CPS is normally conducted where a household is selected regardless of demographic characteristics and people must “opt-out” if they do not wish to participate. Also, participants in this study were compensated for their time. These differences can make interpreting how questions will perform in production difficult. For example, all participants in this study said they did not find the questions overly personal or sensitive and, though they were told they could decline to answer a question if they found it objectionable, no participant did. In production, it is likely there will be some item non-response for these questions.
In this section, I will go through question-by-question and report information obtained from the cognitive interviews, my observations as the interviewer, and recommendations for changes, if any.
Question Number |
Question |
Response Options |
Frequency |
1 |
Previously, you mentioned that (you/Name) had difficulty _________. How has this affected (your/his/her) ability to complete current work duties? Would you say this has caused no difficulty, a little difficulty, moderate difficulty, or severe difficulty? |
1. No difficulty |
-- |
4 |
|||
3 |
|||
-- |
|||
-- |
|||
-- |
|||
|
|
Total |
7 |
Debriefing Responses: Six participants answered for themselves on this question and one participant answered for another household member. The six participants who answered for themselves were consistent in interpreting this question as being about them and their abilities and not about something external to them (e.g., particularly challenging work duties, lack of assistive technology or help on the job, etc.). The proxy respondent was unsure how to answer this question.
Participants were also consistent in interpreting the scale as a relative measure but the standards they used for comparison were different and could have affected their answers. Several participants compared their ability to complete current work duties to an imaginary self without assistive technology or some other form of help, saying things like “I chose ‘A little difficulty’ because if I have a problem I can always ask for help.” One compared himself to other people with disabilities saying, “Other people may have an even more difficult time.” Finally, one participant, when asked why she chose “Moderate difficulty” over “Severe difficulty,” said “There’s no use feeling sorry for myself, I have to learn to deal with [my disability].” I have no recommendation for how to fix this but these varying strategies for answering this question should be kept in mind when interpreting the data.
Other Recommendations for Change: The response options in the white paper read “Would you say this has caused no difficulty, a little difficulty, a moderate amount of difficulty, or severe difficulty?” This was changed to “Would you say this has caused no difficulty, a little difficulty, moderate difficulty, or severe difficulty?” for testing. I recommend using the changed wording in the production survey so that response options are consistent. |
|
2 |
[(Have you)/(Has Name)] EVER worked for pay at a job or business? |
1. Yes 2. No 3. Don't Know 4. Refused |
11 |
||||
|---|---|---|---|---|---|---|---|---|
|
3 |
|||||||
|
-- |
|||||||
|
-- |
|||||||
|
|
|
Total |
14 |
||||
Debriefing Responses: There were a range of circumstances across those who received this question. Some answered for themselves, some for other household members. Some had held their last job years ago, while some had held a job recently. All participants found this question straightforward and were able to answer without difficulty.
3 |
Earlier it was reported that (you/Name) had difficulty ____. Did (you/he/she) ever leave a job because of reasons related to (this difficulty/these difficulties)? |
1. Yes 2. No 3. Don't Know 4. Refused |
10 |
|||
8 |
||||||
-- |
||||||
-- |
||||||
|
|
Total |
18 |
|||
Debriefing Responses: Participants did not express any difficulty answering this question. However, there was a range of interpretations of what the question was asking. One participant, offering a proxy response, answered this question with “Yes” because he thought the question included instances where a person had to take time off of work to receive physical therapy. The other 15 participants who answered this question for themselves (two also answered for other household members) interpreted the question correctly as the ending of a period of employment, though some thought the question referred exclusively to times when the employee chose to quit their job for reasons related to their disability, while others thought it referred to any situation in which the job ended, whether they chose to quit or they were fired or laid off for reasons related to their disability. The participants who responded “Yes” to this question offered a range of personal experiences that matched their interpretations.
Recommendation for Change: The impact of these varying interpretations could have a significant impact on the data. If the intent of this question is to measure only situations in which someone left a job because of their own choice the question is fine as is. If, however, the intent of the question is to also measure situations in which someone was fired or laid off I recommend the following wording: “Did you ever leave a job, or were you ever asked to leave a job, because of reasons related to this difficulty?” |
Interviewer Observations: For participants who are employed and disabled, Questions 1 & 3 come back-to-back. Both questions have introductions that read "Previously you mentioned…" This can be tedious for the interviewer to read and difficult for participants to listen to.
Recommendation for Change: I recommend having a conditional fill for Question 3 that leaves out the introductory sentence (i.e., “Earlier it was reported…”) for those that received Question 1. |
4 |
The purpose of this next question is to identify barriers to employment faced by persons with difficulties. What would you say the main barriers to employment are for (you/Name)? |
1. Education or training 2. Job counseling 3. Transportation 4. Loss of government assistance 5. Need special features at the job 6. Discrimination 7. Other 8. Not interested in working 9. None 10. Don't Know 11. Refused 12. Own Disability |
4 |
-- |
|||
-- |
|||
-- |
|||
-- |
|||
3 |
|||
-- |
|||
-- |
|||
-- |
|||
-- |
|||
-- |
|||
7 |
|||
|
|
Total |
14 |
Debriefing Responses: This question did not perform as intended. As you can see from the distribution of responses, most participants referred to their disability (Option 12) when asked about barriers to employment. When prompted for things outside of themselves or aspects of a job that might be barriers to employment, the participants who answers were categorized as “Own Disability” could not think of anything else. During the debriefing, participants were shown the list of response options and all could pick out one or two that they thought were applicable, but they said they had not thought of those as possible answers to the question.
Recommendation for Change: Depending on the needs of the stakeholders I recommend either (a) eliminating the question, (b) leaving the question as is and understanding that a large proportion of respondents will indicate their disability as a barrier to employment, or (c) changing the response format to be similar to Questions 6 & 19: |
|
Question |
Response Options |
The purpose of this next question is to identify barriers to employment faced by persons with difficulties. Do you consider any of the following a barrier to employment for (you/Name)?
1. Education or training 2. Job counseling 3. Transportation 4. Loss of government assistance 5. Need special features at the job 6. Employer or coworker attitudes 7. Own Disability |
1. Yes 2. No 3. Don't Know 4. Refused |
Interviewer Observations: Participants whose answers were coded as “Discrimination” said things like “there is a lack of understanding,” “hiring us is a risk for employers,” and “you can tell by employers’ reactions...” When asked during the debriefing if they consider discrimination a barrier to employment, only one of these 3 participants said they consider their statements to be indicative of discrimination. The other two said that they would not use that word because they couldn’t prove it or because they thought the actions were unintentional.
Recommendation for Change: Though respondents won’t be able to see the response options, I recommend changing “Discrimination” to the more general and less offensive “Employer and coworker attitudes.” |
5 |
If these barriers could be removed, would (you/Name) be able to work? |
1. Yes 2. No 3. Don't Know 4. Refused |
6 |
1 |
|||
-- |
|||
-- |
|||
|
|
Total |
7 |
Debriefing Responses: For participants that made it through Question 4, this was a straightforward question. This question was only asked of those whose answer fit into one of the predefined categories. The one “No” response to this question was from a woman giving a proxy response for her sister. She said a doctor had told her that her sister would probably never work and she used this as the basis for her answer to this question.
Interviewer Observations: Reading this question, as written, can be awkward if a person only indicated one barrier.
Recommendation for Change: I recommend using a conditional fill to alter the wording to “If this barrier could be removed…” if a person only indicates one barrier on Question 4. |
6 |
The purpose of this next question is to find out if (you have/Name has) taken advantage of any of the following sources that help people prepare for work or advance on the job. [(Have you)/(Has Name)] received assistance from: |
1. Vocational Rehabilitation Programs 2. One Stop Career Centers 3. Ticket to Work Program 4. Assistive Technology Act Program 5. Center for Independent Living for Individuals with Disabilities 6. Client Assistance Program 7. Other |
123 |
3 |
|||
2 |
|||
2 |
|||
5 |
|||
5 |
|||
6 |
|||
|
|
Total |
35 |
Debriefing Responses: Participants had considerable trouble with this question for two main reasons. First, there were a number of programs participants had never heard of. None of the participants had heard of an “Assistive Technology Act Program” before. The two participants who indicated they had received assistance from an Assistive Technology Act Program said that they inferred that it was a program that provided some type of assistive technology (in the case of both of these participants it was computer software to help the blind). Ticket to Work programs and Client Assistance Programs were also difficult for participants.
Another problem participants had was overlap between some of these programs. Some programs taught life skills (e.g., navigation for the blind) that are also helpful for work. This caused a number of people to become confused about whether to say “Yes” to a Vocational Rehabilitation Program or to a Center for Independent Living for Individuals with Disabilities. The same confusion occurred for organizations that purchase assistive technologies for people at work and provide training. Client Assistance Programs, broadly defined, can overlap with any of the other ones. This may cause problems if respondents are unsure where to count it, so count it as both.
Recommendation for Change: I recommend that the stakeholders reevaluate the listed programs and then either (1) reword the program names to make them more easily understood and less overlapping for respondents (e.g., saying “a program that provides job skills training” instead of “Vocational Rehabilitation Program” or “a program that provides life skills training” instead of “Center for Independent Living…”), (2) determine whether specific program names can be substituted for more general categories such as “Client Assistance Program,” or (3) provide adequate definitions within the instrument so interviewers can pass this information along to confused respondents. |
It is also worth noting that participants, correctly, answered this question without respect to a given reference period. Some participants said they had last participated in these programs years, even decades, ago. One participant responded “Yes” to having participated in a “Vocational Rehabilitation Program” though the last time he had been there was 22 years ago, shortly after developing mobility issues as the result of an accident on the job (so he was very sure of the time).
Recommendation for Change: I recommend defining a reference period that will make this question more useful (e.g., does knowing someone found a Vocational Rehabilitation Program they were in 20 years ago “Not at all useful” help direct current policy?) but not be so narrow that you miss useful information. I recommend 5 years. The question would then read like this: “…In the past 5 years, [(have you)/(has Name)] received assistance from:” |
7 |
How helpful was (this source)? Would you say it was not at all helpful, a little helpful, somewhat helpful, or very helpful? |
1. Not at all helpful |
14 |
3 |
|||
14 |
|||
16 |
|||
1 |
|||
-- |
|||
|
|
Total |
35 |
Debriefing Responses: Participants did not express any difficulty in answering this question. All answered this question quickly and with certainty, even the participant who had last participated in a program more than 20 years ago.
Other Recommendation for Change: The response options in the white paper read “Would you say it was not at all helpful, a little help, some help, or very helpful?” This was changed to “Would you say it was not at all helpful, a little helpful, somewhat helpful, or very helpful?” for testing. I recommend using the changed wording in the production survey so that response options are consistent (i.e., helpful). |
8 |
Have (you/Name) ever requested any change in your current workplace, for example, in equipment or work processes, to help you do your job better? |
1. Yes 2. No 3. Don't Know 4. Refused |
6 |
7 |
|||
-- |
|||
-- |
|||
|
|
Total |
13 |
Debriefing Responses: Participants did not express any difficulty with this question or Question 9, however you can see from the distribution of responses in Question 9 that participants were primarily focused on new or modified equipment (i.e., a trackball, a standing workstation, a larger stove, a new screenreading program, text enlargement software, an office phone, and a van). When asked, during the debriefing, what types of changes other people might request, participants still remained focused on new or modified equipment. This may be due to the question specifically mentioning equipment as an example.
Recommendation for Change: I recommend removing the phrase “for example, in equipment or work processes,” from the question. This will likely keep respondents from focusing solely on equipment changes. |
9 |
What change did (you/Name) request? |
1. New or modified equipment 2. Physical changes to the workplace 3. Policy changes to the workplace 4. Changes in work tasks, job structure or schedule 5. Changes in communication or information sharing 6. Changes to comply with religious beliefs 7. Accommodations for family or personal obligations 8. Other changes 9. Don't Know 10. Refused |
7 |
-- |
|||
-- |
|||
-- |
|||
-- |
|||
-- |
|||
-- |
|||
1 |
|||
-- |
|||
-- |
|||
|
|
Total |
8 |
Debriefing Responses: See Question 8.
Interviewer Observations: There is some theoretical overlap between “New or Modified Equipment” and “Physical Changes to the Workplace” that made categorizing a “standing desk” difficult. This should be addressed with interviewer training.
Other Recommendations for Change: I recommend allowing more than one response for this question. Only two participants in the cognitive testing offered more than one change but, unless there is some limitation in the programming, there is no reason not to collect more information if someone wants to offer it. |
One of the participants who said they requested multiple changes said she had requested a new computer assistive technology program and training for it. I didn’t see training as an option and listed it as an “Other change.” If you consider training a valuable potential change, I recommend adding it to the list. |
10 |
Was the change granted? |
1. Yes 2. Partially 3. No 4. Don't Know 5. Refused |
6 |
-- |
|||
2 |
|||
-- |
|||
-- |
|||
|
|
Total |
8 |
Debriefing Responses: For participants that made it through questions 8 and 9, this was a straightforward question. Participants who answered “Yes” to this question said that the change they requested was granted fully and in the way they had originally requested. The participant who answered “No” to this question said neither of the changes he requested were granted in any way.
Interviewer Observations: For the one participant that said the changes they requested were not granted, I mis-categorized their response as “Partially” because I had gotten used to “2” meaning “No” for these Yes/No questions.
Recommendation for Change: I recommend moving the “Partially” option to follow “No.” |
11 |
How [(do you)/(does Name)] typically commute to work? |
1. Bus 2. Specialized bus or van service 3. Train/subway 4. Taxi 5. Own vehicle 6. Passenger in a family member's car 7. Passenger in friend's car 8. Passenger in carpool 9. Driver in carpool 10. Motorcycle 11. Bicycle 12. Walk 13. Other 14. Work from home 15. Don't Know 16. Refused |
4 |
3 |
|||
5 |
|||
-- |
|||
4 |
|||
-- |
|||
1 |
|||
-- |
|||
-- |
|||
-- |
|||
-- |
|||
-- |
|||
-- |
|||
-- |
|||
-- |
|||
-- |
|||
|
|
Total |
17 |
Debriefing Responses: This question performed well in testing. Participants were able to answer quickly and decisively. When asked during the debriefing what typically means in this question, participants gave answers ranging from what they do most often to what they do more than half of the time to what they do all of the time. I think these interpretations will have a negligible effect on the data and do not warrant any change in wording.
Interviewer Observations: As an interviewer I found answers to this question difficult to categorize. For participants who do not use public transportation, the modal answer is "(I/he/she) drive(s)," to which I had to probe further about whether that is in a carpool. A similar problem will likely arise for respondents who say "(I/he/she) gets a ride," an interviewer would have to probe to see whether that was with a friend or in a carpool.
Recommendation for Change: I recommend considering the way the data from this question will be used, determining what level of categorization is necessary, and consolidating categories as warranted. For example, will different decisions be made if more respondents are the passenger in a carpool vs. whether they are the driver, or will it simply be enough to know that they are in a carpool? Consolidating the categories will mean less probing for the interviewers and less categorization error. |
Two participants simply said they used “Metroaccess.” Similar, regional, name-brand transportation alternatives will require probing on the part of interviewers. This can be addressed through training.
Four participants spontaneously offered more than one answer. Two of them used one mode of transportation to get to another (e.g., one participant took a bus to get to the subway then boarded another bus). Two participants said they used different means depending on various constraints such as scheduling (e.g., one participant got a ride from a friend but occasionally had to use a van service).
Recommendation for Change: I recommend allowing more than one answer to this question. |
12 |
(Do you/Does Name) do any work at home for (your/his/her) job or business? |
1. Yes 2. No 3. Don't Know 4. Refused |
5 |
9 |
|||
-- |
|||
-- |
|||
|
|
Total |
14 |
Debriefing Responses: Participants had no apparent difficulty answering this question. Several of the participants had jobs for which this question was not directly applicable. For example, two participants and one participant’s wife were cooks. This job requires them to be on the job. These people acknowledged that there is some work you could do at home for these jobs, such as scheduling but none of them did this type of task.
13 |
[When (you/he/she) (work/works) at home, how/How] many hours per week (do/does) (you/he/she) usually work at home as part of this job? |
1. Free Response ___ 2. Hours vary 3. Don’t Know 4. Refused |
8,5,3,40 |
1 |
|||
-- |
|||
-- |
|||
|
|
Total |
5 |
Debriefing Responses: Only one participant who works at home sporadically (i.e., once every couple months) had difficulty coming up with an estimate of the number of hours for this question. The other four participants, three of whom gave proxy responses, were confident in their answers. The three proxy respondents were unemployed or not in the labor force and said that they are at home when their employed household member is working from home and that is how they know how to answer this question.
14 |
Are those hours worked at home usually considered paid work hours? |
1. Yes 2. No 3. Don't Know 4. Refused |
3 |
2 |
|||
-- |
|||
-- |
|||
|
|
Total |
5 |
Debriefing Responses: Participants had less trouble with this question than with Question 15. One participant noted that his girlfriend occasionally answers e-mails at home and is not paid for that time but usually when she does work at home she gets paid for it. Another participant said that she works at home sporadically and not for a regular number of hours but, when she does work at home, the time is paid. One participant, who said he does not get paid for work done at home has two jobs, one of which is a personally owned business. During the debriefing he said he responded “No” to this question because he might work a lot of hours but only get paid if the business is profitable so he does not consider those hours paid. I don’t think any of these observations warrants altering the wording of this question.
15 |
(Do/Does) (you/he/she) have a formal arrangement with (your/his/her) employer to be paid for the work that (you/he/she) (do/does) at home, or (were/was) (you/he/ she) just taking work home from the job? |
1. Paid 2. Taking work home |
|
-- |
|||
5 |
|||
|
|||
|
|
Total |
5 |
Debriefing Responses: The difference in answers between Question 14 and 15 are due primarily to the fact that the participants don’t have a formal arrangement with their employer. One of the three people that receive pay for work done at home and answered “No” to this question did so because she is paid salary and did not think of work done at home as something separate from work done in an office. The other two participants were responding for another household member and said that their boss knew they were working at home but they didn’t have a formal arrangement.
Recommendation for Change: I am not sure what the purpose of this question is, outside of what’s captured in Question 14, but I would recommend discussing amongst the stakeholders whether this question is necessary and if it is not then eliminating it. |
16 |
What is the main reason why (you work/Name works) at home? |
1. Less commuting 2. Reduce expenses for transportation, food, clothing, etc. 3. Coordinate work schedule with work and family needs 4. More control over own life 5. Illness, disability, health reasons 6. Mandated by employer to reduce employer costs 7. Mandated by employer to meet local transportation management and pollution abatement requirement 8. More productive 9. Other 10. Don't Know 11. Refused |
-- |
-- |
|||
-- |
|||
-- |
|||
1 |
|||
-- |
|||
-- |
|||
1 |
|||
3 |
|||
-- |
|||
-- |
|||
|
|
Total |
5 |
Debriefing Responses: Not many people received this question so it is difficult to draw conclusions. Of those that did receive this question, three provided proxy responses for another household member. One of these participants said his mother works from home because her boss says she can. When probed during the interview for other reasons, the participant could not provide any. During the debriefing I showed him a list with response options 1-8 on it and he said they all seemed to apply. Another participant, who offered proxy responses on behalf of his wife, said that it was because she was more productive at home and that they talk about it quite often so he was sure that that was the reason. The third participant said his wife works from home only when she can’t get all of her work done during the day because her office is understaffed. I coded this response as “Other.” Of the two participants who answered for themselves, one said he worked at home due to his disability and the other said he did because he has a home business and does not have other options.
Recommendation for Change: If, using data collected elsewhere in the CPS survey, it is possible to skip this question for people who have a business at home that would be ideal. Otherwise, I recommend adding a category for “Self-employed/Business at home.” |
17 |
Do (you/Name) have flexible work hours that allow (you/him/her) to vary or make changes in the time [(you begin and end)/(he begins and ends)/(she begins and ends)] work? |
1. Yes 2. No 3. Don't Know 4. Refused |
5 |
6 |
|||
1 |
|||
-- |
|||
|
|
Total |
12 |
Debriefing Responses: Only one participant had noticeable difficulty answering this question and he responded “Don’t know” to this question. That participant had an atypical work situation. He had two jobs, one of which was a personally owned business. He said he worked an average of 40 hours a week at home but occasionally had to report to an office. He said when he worked at home he could vary his beginning and end times but had specific times he had to report to the office, when required. I don’t believe this situation requires making any changes to the question. I am sure CPS interviewers are trained about how to deal with complex work situations. This training will be helpful for participants such as the one mentioned here.
18 |
Some people are in temporary jobs that last only for a limited time or until the completion of a project. Is your job temporary? |
1. Yes 2. No 3. Don't Know 4. Refused |
3 |
9 |
|||
-- |
|||
-- |
|||
|
|
Total |
12 |
Debriefing Responses: Participants found this question straightforward and were able to answer without any apparent difficulty. During the debriefing all participants gave a satisfactory answer to what the question is asking.
19 |
A variety of programs exist to help people in different situations. In the past year did (you/Name) receive assistance from any of the following programs? |
1. Workers Compensation 2. Social Security Disability Income 3. Supplemental Security Income 4. Veterans Disability compensation 5. Disability Insurance Payments 6. Other disability payments 7. Medicaid 8. Medicare 9. Other |
15 |
14 |
|||
7 |
|||
-- |
|||
2 |
|||
2 |
|||
17 |
|||
9 |
|||
4 |
|||
|
|
Total |
56 |
Debriefing Responses: This question performed much better than Question 6. Participants were more likely to be familiar with these sources than the types of organizations mentioned in Question 6. I believe it helped that specific names were mentioned and not categories. There was some confusion about what counted as “Other disability payments.” Answers given were “Interim Disability Assistance or IDA” and “early retirement” which one participant receives due to loss of eyesight. The inclusion of the “Other” category for testing did not yield answers that fit with the intent of this question and should not be added to the production survey. Answers for “Other” included food stamps and other nutritional programs, state unemployment, and Temporary Assistance for Needy Families (TANF).
Participants paid attention to the reference period. When asked, during the debriefing, about the last time they received assistance, most participants said their involvement with these programs is ongoing (i.e., they’ve received assistance within the last month) or, if it wasn’t ongoing they provided a specific month within the last year that they had last received assistance.
Interviewer Observations: Participants who are (unemployed or not in the labor force) and disabled go directly from Questions 6 & 7 to Questions 19 & 20. This transition can be awkward. Both questions are about programs and the questions themselves sound vaguely similar, with the exception of the reference period.
Recommendations for Change: Add a conditional fill for respondents that are (unemployed or not in the labor force) and disabled that reads "Another set of programs exist to help people in different situations." Also, emphasize the reference period by capitalizing "PAST YEAR" in the question. |
For participants who are (unemployed or not in the labor force) and disabled, the slightly different wording between Questions 6 & 19 is not a problem because they follow the format of Question 6 (i.e., they say yes/no after each program). For those who are (unemployed or not in the labor force) and not disabled, going straight to Q19 causes some confusion. The way the question reads people wait for a list to be read to them before saying yes/no.
|
Recommendation for Change: Revise the wording of Question 19 to be parallel to Question 6. That is, "In the past year, did you receive assistance from:" |
|
||
|---|---|---|---|---|
20 |
Did (source) affect whether or not [(you worked)/(Name worked)]? |
1. Yes 2. No 3. Don't Know 4. Refused |
106 |
|
46 |
||||
-- |
||||
-- |
||||
|
|
Total |
56 |
|
Debriefing Responses: Of the 23 participants in this study, 21 received assistance from one or more of the organizations mentioned in Question 19, or had a household member who did. When asked Question 20, for themselves or for another family member, there was often a delay in answering. Participants’ answers during the debriefing showed that most understood the intent of the question. When asked to paraphrase the question in their own words, answers ranged from “you want to know whether a person would prefer to sit home and collect a check or go to work” to “if I worked would (source) still provide me benefits?” Several people said some variation on “People become dependent on receiving the benefit and are fearful that if they get a job they will be cut off…and the job might not be permanent or pay as much as their benefit.”
When asked whether the assistance they receive is dependent upon whether they work or the number of hours they worked, most participants said that assistance from these organizations is based on income, many knew exact dollar figures that set the upper limit on what they could earn. Nine out of the 21 participants who were asked Question 20 said that they do consider the work/income limitations imposed by these programs when seeking employment or more hours at their current job. More pertinent to this question, however, is that only 2 out of these 9 people answered “Yes” to Question 20. This offers some proof that the question may not be working as intended. While people are able to paraphrase this question when given a sufficient amount of time during the debriefing, I think its meaning is hard to decipher in the context of the interview. There are also other factors that affect people’s answers. Several participants said that their disability was a larger factor in their deciding not to work than the loss of benefits and for that reason they answered “No” to this question.
Recommendations for Change: I recommend either (1) using the more pointed language used in the debriefing for this question: “Do you choose (not to work/to limit the amount of work you do, or the amount of money you earn) in order to continue participating in this program?” or (2) eliminating/replacing this question. During the interviews several participants who had also received Questions 6 & 7 spontaneously offered opinions about how helpful these programs were. Replacing this question with one similar to Question 7 may provide similar information. Both of these recommendations present some risk however. The language used in Option 1 may offend some people and result in breakoffs or noncontact on subsequent interview waves. Without additional testing, it is difficult to say whether Option 2 would provide useful information. |
Question Number |
Questions |
Response Options |
Who Receives Question |
Demographics |
|||
Intro |
I will formally begin the interview by asking you for the names of all the people living in your household. Are you ready to begin? |
|
All |
1 |
Household Roster |
|
All |
Labor Force Participation Questions |
|||
Intro |
Next, I am going to ask a few questions about work related activities LAST WEEK. By last week I mean the week beginning on Sunday, ____ and ending on Saturday, ___ |
|
All |
1 |
Does anyone in this household have a business or farm? |
1. Yes |
All |
2 |
LAST WEEK, did you do ANY work for (either) pay (or profit)? |
1. Yes |
All |
3 |
LAST WEEK, (in addition to the business,) did you have a job either full or part time? Include any job from which you were temporarily absent. |
1. Yes |
1. Q1=1 |
4 |
What was the main reason you were absent from work LAST WEEK? |
1. On Layoff |
Q3=1 |
5 |
LAST WEEK, were you on layoff from a job? |
1. Yes |
Q3=2 |
6 |
Has your employer given you a date to return to work? |
1. Yes |
Q4=1 OR 2 |
7 |
Have you been given any indication that you will be recalled to work within the next 6 months? |
1. Yes |
Q6=2 |
8 |
Could you have returned to work LAST WEEK if you had been recalled? |
1. Yes |
Q7=1 |
9 |
Why is that? |
1. Own Temporary Illness |
Q8=2 |
10 |
Have you been doing anything to find work during the last 4 weeks? |
1. Yes |
Q2=2 OR Q5=2 |
11 |
What are all of the things you have done to find work during the last 4 weeks? |
Active: 9. Passive 10. Don’t Know 11. Refused |
Q10=1 |
12 |
LAST WEEK, could you have started a job if one had been offered? |
1. Yes |
Q11 = 1 - 8 |
13 |
Why is that? |
1. Waiting for a new job to begin |
Q12=2 |
Disability Questions |
|||
Intro |
This month we want to learn about people who have physical, mental, or emotional conditions that cause serious difficulty with their daily activities. Please answer for household members who are 15 years old or over. |
|
All |
1 |
Is anyone deaf or does anyone have serious difficulty hearing? |
1. Yes |
All |
1a |
Who is it? |
1. Family Member Name |
Q1=1 |
1b |
Anyone else? |
1. Yes |
Q1=1 |
2 |
Is anyone blind or does anyone have serious difficulty seeing even when wearing glasses? |
1. Yes |
All |
2a |
Who is it? |
1. Family Member Name |
Q2=1 |
2b |
Anyone else? |
1. Yes |
Q2=1 |
3 |
Because of a physical, mental, or emotional condition, does anyone have serious difficulty concentrating, remembering, or making decisions? |
1. Yes |
All |
3a |
Who is it? |
1. Family Member Name |
Q3=1 |
3b |
Anyone else? |
1. Yes |
Q3=1 |
4 |
Does anyone have serious difficulty walking or climbing stairs? |
1. Yes |
All |
4a |
Who is it? |
1. Family Member Name |
Q4=1 |
4b |
Anyone else? |
1. Yes |
Q4=1 |
5 |
Does anyone have difficulty dressing or bathing? |
1. Yes |
All |
5a |
Who is it? |
1. Family Member Name |
Q5=1 |
5b |
Anyone else? |
1. Yes |
Q5=1 |
6 |
Because of a physical, mental, or emotional condition, does anyone have difficulty doing errands alone such as visiting a doctor’s office or shopping? |
1. Yes |
All |
6a |
Who is it? |
1. Family Member Name |
Q6=1 |
6b |
Anyone else? |
1. Yes |
Q6=1 |
Question Number |
Questions |
Response Options |
Who Receives Question |
CPS Disability Supplement Questions |
|||
Intro |
This month we would like to learn more about how people in different circumstances deal with labor market challenges. |
|
All |
1 |
Previously, you mentioned that (you/Name) had difficulty _________. How has this affected (your/his/her) ability to complete current work duties? Would you say this has caused no difficulty, a little difficulty, moderate difficulty, or severe difficulty? |
1. No difficulty |
Disability and Employed |
2 |
[(Have you)/(Has Name)] EVER worked for pay at a job or business? |
1. Yes |
Disability and Not in the Labor Force (& Unemployed for testing) |
3 |
Earlier it was reported that (you/Name) had difficulty ____. Did (you/he/she) ever leave a job because of reasons related to (this difficulty/these difficulties)? |
1. Yes |
1. Q2 = 1 OR Disability and Employed |
4 |
The purpose of this next question is to identify barriers to employment faced by persons with difficulties. What would you say the main barriers to employment are for (you/Name)? |
1. Education or training 6.
Discrimination |
Disability and Not in Labor Force (& Unemployed for testing) |
5 |
If these barriers could be removed, would (you/Name) be able to work? |
1. Yes |
Q4=1-7 |
6 |
The purpose of this next question is to find out if (you
have/Name has) taken advantage of any of the following sources
that help people prepare for work or advance on the job. [(Have
you)/(Has Name)] received assistance from: |
1. Yes |
Disability |
7 |
How helpful was (this source)? Would you say it was not at all helpful, a little helpful, somewhat helpful, or very helpful? |
1. Not at all helpful |
Q6=1 for each option |
8 |
Have (you/NAME) ever requested any change in your current workplace, for example, in equipment or work processes, to help you do your job better? |
1. Yes |
Employed |
9 |
What change did (you/Name) request? |
1. New or modified equipment |
Q8=1 |
10 |
Was the change granted? |
1. Yes |
Q8=1 |
11 |
How [(do you)/(does Name)] typically commute to work? |
1. Bus |
Employed |
12 |
(Do you/Does Name) do any work at home for (your/his/her) job or business? |
1. Yes |
Employed |
13 |
[When (you/he/she) (work/works) at home, how/How] many hours per week (do/does) (you/he/she) usually work at home as part of this job? |
1. Free Response ___ 3. Don’t Know 4. Refused |
Q12=1 |
14 |
Are those hours worked at home usually considered paid work hours? |
1. Yes |
Q12=1 |
15 |
(Do/Does) (you/he/she) have a formal arrangement with (your/his/her) employer to be paid for the work that (you/he/she) (do/does) at home, or (were/was) (you/he/ she) just taking work home from the job? |
1. Paid 3. Don’t Know 4. Refused |
Q12=1 |
16 |
What is the main reason why (you work/Name works) at home? |
1. Less commuting |
Q12=1 |
17 |
(Do/Does) (you/Name) have flexible work hours that allow (you/him/her) to vary or make changes in the time [(you begin and end)/(he begins and ends)/(she begins and ends)] work? |
1. Yes |
Employed |
18 |
Some people are in temporary jobs that last only for a limited time or until the completion of a project. Is your job temporary? |
1. Yes |
Employed |
19 |
A variety of programs exist to help people in different
situations. In the past year did (you/Name) receive assistance
from any of the following programs? |
1. Yes |
All |
20 |
Did (source) affect whether or not [(you worked)/(Name worked)]? |
1. Yes |
Q19=1 for each response |
1 These data based on CPS Disability Statistics annual release, dated June 24, 2011.
2 Three respondents reported having "a disability" and receiving assistance for that disability though they did not respond affirmatively to any of the disability status questions. This result is not unexpected as there are many definitions of disability and assistance program criteria. This test was not designed to evaluate the six CPS disability questions. For the purpose of these interviews these respondents were asked the questions for those without a disability.
3 This column indicates the number of people who said “Yes” they had received assistance from this type of organization.
4 This column shows the number of people who used the provided terms to describe any of the programs.
5 This column indicates the number of people who said “Yes” they had received assistance from this type of organization.
6 This column shows the number of people who used the provided terms to describe any of the programs.
| File Type | application/msword |
| File Title | Report |
| Author | rho_c |
| Last Modified By | mcmenamin_t |
| File Modified | 2011-09-19 |
| File Created | 2011-09-19 |