CPS Cognitive Study

October 1, 2009

NOTE TO THE REVIEWER OF:	OMB No. 1220-0141 Cognitive and Psychological Research
FROM:	Jennifer Edgar, Scott Fricker, and Kathy Downey Research Psychologists Office of Survey Methods Research (OSMR)
SUBJECT:	Submission of Materials for a CPS Cognitive Study

Please accept the enclosed materials for approval under the OMB clearance package “Cognitive and Psychological Research.” In accordance with our agreement with OMB, we are submitting a brief description of the research, and the materials to be used in the research.

Participants will be recruited from the OSMR participant database (i.e., general public). The estimated number of burden hours is estimated to be 30.

If there are any questions regarding this project, please contact Jennifer Edgar at (202) 691-7528.

1. Introduction and Purpose

Since the introduction of revised CPS labor force question wording in 1994, more respondents change their labor force status (i.e., move from employed to unemployed) between waves then they did with the original wording. It is currently unknown if the increased variability is a result of more accurate data collection, or if the revised question wording introduced confusion or problems for respondents.

The objective of this testing is to explore the cognitive processes participants undergo while answering the CPS labor force questions to gain insight into their understanding of the questions and identify potential problems with the question wording.

2. Procedures

The purpose of this study is to explore comprehension issues surrounding the CPS labor force questions. To accomplish this goal, the current CPS question wordings will be tested using cognitive interviewing. Participants will be asked to answer the CPS questions using their own information, then will be given 5 vignettes (Attachment I), which will be assigned using a counter balanced design where they will be asked to answer select CPS labor force questions (Attachment II), using the information provided in the scenario. After answering all the CPS questions, participants will be debriefed using a mixture of pre-determined questions and spontaneous probes based on their answers to the questions (also Attachment II, the Cognitive Interviewer Packet).

Census interviewers in the Hagerstown and Jeffersonville call centers will conduct the CPS labor force interview, using the same procedures they use in production CPS surveys. The Census interviewers will call and interview participants in the Office of Survey Methods Research (OSMR) laboratory. Each interview will be audio taped. Following the CPS interview, staff from the BLS Office of Survey Methods will conduct the vignette phase of the research. Following both interviews, OSMR staff will conduct a debriefing to explore the participant’s understanding and interpretation of key CPS labor force terms.

3. Participants

Thirty participants will be recruited from the OSMR participant database. Participants will be compensated $40. Efforts will be made to select participants with varying levels of education, age and employment status, based on self-reported information provided during the initial recruitment process.

4. Burden Hours

We anticipate that each session will last no longer than one hour, for a total of 30 burden hours.

5. Data Confidentiality

Participants will be informed as to the voluntary nature of the study. Participants will also be informed that the study will be used for internal purposes to improve the design of a national labor force survey. Participants will be given a consent form to read and sign (Attachment III). Information related to this study will not be released to the public in any way that would allow identification of individuals except as prescribed under the conditions of the Privacy Act Notice.

6. Attachments

Attachment I:	Vignettes
Attachment II:	Cognitive Interviewer Packet
Attachment III:	Consent agreement form and Privacy act statement

Attachment I: Vignettes

Vig #	CPS Concept Tested	Complicated/Correct Answer	Straightforward/Correct Answer
1Y	"JOB" and "TEMPORARILY ABSENT"	Wendy is a 10th grader who babysits most Tuesdays and Thursdays for two neighborhood families. She usually babysits for a total of about 8 hours a month, but her schedule can vary - some weeks she doesn't do any babysitting. Last week, Wendy had to study for several exams so she did not do any babysitting./YES	Wendy graduated last year from college with a degree in early childhood education. Since graduating, she has babysat her neighbor’s daughter for four hours each weekday afternoon, for a total of 20 hours each week. Last week, Wendy was on a trip with her family, but she will resume her babysitting duties next week when she returns from vacation./YES
1N	"JOB" and "TEMPORARILY ABSENT"	Wendy is a 10th grader who picks up babysitting jobs whenever she can to earn a little extra money. Last week, Wendy didn't get any calls for babysitting so she had extra time to get caught up on her schoolwork./NO	Wendy recently graduated from college with a degree in early childhood education. Since graduating, she has been searching for a full-time teaching job. Last week, Wendy worked on her resume and went on several job interviews, but did no work for pay./NO
2Y	"WORK" and "PAY OR PROFIT"	Cynthia is a stay-at-home mom for her three children, but occasionally she helps with her husband's catering business when things get very busy. Last week, Cynthia spent about 20 hours preparing menus and coordinating one of her husband's largest catering jobs of the season. She received no pay for her work./YES	Cynthia stays at home with her three young children and also works about five hours each week doing the accounting for her husband's catering business. Last week, Cynthia did 6 hours of bookkeeping for the business and received a paycheck for $180./YES
2N	"WORK" and "PAY OR PROFIT"	Cynthia is a stay-at-home mother, but three days a week she goes to her sister's house to help prepare food for her sister's catering business. She does not get paid for her time, but likes the work and is learning a lot about the business. Cynthia typically spends about 8 hours a week working with her sister, but last week she spent 16 hours helping to prepare food for a big catering job./NO	Cynthia does not have a regular job because she stays at home with her three young children, but she frequently volunteers in the community. Last week, Cynthia volunteered at the local hospital for 15 hours./NO
3Y	"JOB" and "TEMPORARILY ABSENT"	Elena has worked as a social studies teacher for 15 years at the local high school and plans to return to her position in the fall when school starts back up. From June through August, however, she does not teach or receive a pay check from the school. Last week, Elena was still on her summer break, visiting with friends and family and tending to her garden./YES	Elena was a social studies teacher last year at the local high school. To make ends meet during the summer months, Elena teaches summer school classes in history and English. This summer she has worked about 25 hours per week teaching and grading papers. Last week, Elena had to go out of town to care for her elderly mother, so a substitute teacher took her class for the week. Elena will return to her class next week./YES
3Y	"JOB" and "TEMPORARILY ABSENT"	Elena works as a carpenter for a residential housing contractor. She gets paid for the hours that she actually works, and usually works 40 hours a week. Last week, severe storms made it too wet to do any work, however, so Elena was told not to report to the job site and did not receive pay for those days./YES	Elena works as a carpenter for a residential housing contractor. She gets paid for the hours that she actually works, and usually works 40 hours a week. Last week, Elena worked 35 hours and received a paycheck for $550./YES
3N	"JOB" and "TEMPORARILY ABSENT"	Elena works as a nurse for a geriatric care manager service. The agency calls her to work whenever one of their clients needs hands-on nursing care. She gets paid for the time she actually works with clients, and her hours vary from week to week. Last week, Elena did not receive a call from her agency, and she used the extra time to finish several projects around her house./NO	Elena recently finished nursing school, but has been unable to find work because the local hospital is not hiring. Last week, Elena went on several job interviews at neighboring hospitals, but so far she hasn't received any job offers./NO
4Y	"WORK" and "PAY OR PROFIT"	Cathy recently quit her job as a graphic designer at the local newspaper, and plans to start her own graphic design company out of her home. Last week, Cathy cleaned and painted the back room of her house and met with several potential clients in preparation for her new business./YES	Cathy works as a graphic designer in her own home-based business. Last week, she finished large projects for two clients and worked 50 hours./YES
4N	"WORK" and "PAY OR PROFIT"	Cathy lost her job at the assembly plant last spring. For the past two months, she has been taking vocational training courses that she hopes will help her start her own small business. While she is attending training, Cathy receives a public assistance check from the state to help with housing and food expenses. Last week, Cathy received an assistance check for $550 and participated in 20 hours of classroom training./NO	Cathy lost her job at the assembly plant last spring and had to move back in with her parents. While she has been looking for work, she has also been taking computer skills classes at the local community college. Last week, Cathy went on two job interviews and attended 5 hours of classes./NO
5Y	"LAYOFF"	Pat lives in Vail, CO, where she worked as a manager at a resort hotel. At the end of July, the hotel was closed for major renovations. The hotel owners told Pat that her job would be waiting for her when the hotel reopens in the fall. Pat has been out of work since July, but last week went on several job interviews./YES	Pat lives in Vail, CO, where she worked as a manager at a resort hotel. At the end of July, the hotel was closed for major renovations. The hotel owners told Pat that her job would be waiting for her when the hotel reopens in October. Pat has been out of work since July, but has enough savings to make ends meet until October./YES
5N	"LAYOFF"	Pat lives in Vail, CO, where she worked as a manager at a resort hotel. At the end of January, the hotel had to shut down temporarily because the owners didn't make their rent payments on time. While the owners try to get a new business loan for the hotel, Pat has been out of work. The owners have assured Pat that her job will be waiting for her if the hotel reopens, but have not given her a specific date to return. Last week, Pat called the owners to check on the status of the hotel opening./NO	Pat lives in Vail, CO, where she worked as a manager at a resort hotel. At the end of July, the hotel went out of business, and Pat has had to look for new work. Last week, Pat went on several job interviews./NO

Attachment II:

Cognitive Interviewer Packet

Part 1: Introduce the project

Read by the cognitive interviewer:

Hello. My name is __________, and I work at the Bureau of Labor Statistics.

(Name of Recruiter) may have told you that we will be testing one of our forms that is used to collect demographic information for the Current Population Survey. This survey is with people all across the country – approximately 60,000 households. The survey is the source of the monthly national unemployment rate.

We want to make improvements to the survey and we are asking for your help to see how well the current set of questions works. There are two parts to the survey. The first part of the survey asks about general demographic and employment information. Please answer the questions as you would if you were being interviewed for this survey. That is, simply answer these questions for yourself and your household. This first part of the survey will be conducted by an actual Census interviewer over the telephone. We will be connecting you with him/her shortly. When this part is done, we’ll introduce the rest of the interview.

Part 2: Vignette Questions

Read by the cognitive interviewer:

This is the second portion of the survey where you will be using the fictional scenarios.

• You are not answering questions about yourself or your household

• You are role playing and answering questions about the different persons we’ve created.

• There are 5 different persons with different work situations – we will be referring to them as “vignettes.”

• Sometimes these vignettes contain more detail than is needed to answer the questions about someone’s work situation.

• Please take out your vignette package and turn to Vignette 1.

• Take a moment to read the first vignette and let me know when you’re ready to begin.

Interviewer: Always say parenthetical listed in question text.

Vignette 1: 20B-A

Q20. LAST WEEK, did Wendy do ANY work for (either) pay (or profit)?

• Yes  Go to next vignette

• No

• Retired

• Disabled

• Unable to work

• Don’t know

• Refused

Q20B-A. LAST WEEK, did Wendy have a job either full or part time? Include any job from which she was temporarily absent.

• Yes  Go to next vignette

• No

• Retired

• Disabled

• Unable to work

• Don’t know

• Refused

Q20B-b. LAST WEEK, was Wendy on layoff from a job?

• Yes

• No

• Retired

• Disabled

• Unable

• Don’t know

• Refused

Q22. Has Wendy been doing anything to find work during the last 4 weeks?

• Yes

• No

• Retired

• Disabled

• Unable

• Don’t know

• Refused

Vignette 2: 20, 20-1, 20-2

Moving on to the second vignette, please answer these questions based on the information provided in vignette 2.

Q20. LAST WEEK, did Cynthia do ANY work for (either) pay (or profit)?

• Yes  go to Q20I-1

• No

• Retired

• Disabled

• Unable to work

• Don’t know  go to next vignette

• Refused  go to next vignette

Q20-1. LAST WEEK, did Cynthia do any unpaid work in the family business or farm?

• Yes

• No  go to next vignette

• Don’t know  go to next vignette

• Refused  go to next vignette

Q20-2. Did Cynthia receive any payments or profits from the business?

• Yes

• No  go to next vignette

• Don’t know  go to next vignette

• Refused  go to next vignette

Q20I-1. LAST WEEK, how many hours did Cynthia ACTUALLY work at her (main) job?

• Number of hours  _____________

• Don’t know

• Refused

Vignette 3: 20B-A (same as 1)

We are now on Vignette 3. Please turn to that page and use the information there to answer the following question.

Q20. LAST WEEK, did Elena do ANY work for (either) pay (or profit)?

• Yes  Go to next vignette

• No

• Retired

• Disabled

• Unable to work

• Don’t know

• Refused

Q20B-A. LAST WEEK, did Elena have a job either full or part time? Include any job from which she was temporarily absent.

• Yes  Go to next vignette

• No

• Retired

• Disabled

• Unable to work

• Don’t know

• Refused

Q20B-b. LAST WEEK, was Elena on layoff from a job?

• Yes

• No

• Retired

• Disabled

• Unable

• Don’t know

• Refused

Q22. Has Elena been doing anything to find work during the last 4 weeks?

• Yes

• No

• Retired

• Disabled

• Unable

• Don’t know

• Refused

Vignette 4: 20, 20-1, 20-2 (same as 2)

Looking at the next page, Vignette 4, please read the information and use it to answer the following questions.

Q20. LAST WEEK, did Cathy do ANY work for (either) pay (or profit)?

• Yes  go to Q20I-1

• No

• Retired

• Disabled

• Unable to work

• Don’t know  go to next vignette

• Refused  go to next vignette

Q20-1. LAST WEEK, did Cathy do any unpaid work in the family business or farm?

• Yes

• No  go to next vignette

• Don’t know  go to next vignette

• Refused  go to next vignette

Q20-2. Did Cathy receive any payments or profits from the business?

• Yes

• No  go to next vignette

• Don’t know  go to next vignette

• Refused  go to next vignette

Q20I-1. LAST WEEK, how many hours did Cathy ACTUALLY work at her (main) job?

• Number of hours  _____________

• Don’t know

• Refused

Vignette 5: 20B-A, 20B-B, 21, 21A

Finally, please turn to the last page and use the information in Vignette 5 to answer these questions.

20B-A. LAST WEEK, (in addition to the business,) did Pat have a job either full or part time? Include any job from which Pat was temporarily absent.

• Yes

• No

• Retired

• Disabled

• Unable

• Don’t know

• Refused

Q20B-b. LAST WEEK, was Pat on layoff from a job?

• Yes

• No  END

• Retired  END

• Disabled  END

• Unable  END

• Don’t know  END

• Refused  END

Q21. Has Pat’s employer given her a date to return to work?

• Yes  go to Q21A-1

• No

• Don’t know

• Refused

Q21A. Has Pat been given any indication that she will be recalled to work within the next 6 months?

• Yes

• No  END

• Don’t know  END

• Refused  END

Q21A-1. Could Pat have returned to work LAST WEEK if she had been recalled?

• Yes

• No

• Don’t know

• Refused

Part 3: Debriefing

I’d like to go back through some of the questions you just answered and talk about them for a minute.

Let’s talk about answering for yourself or your household first. Then, we can discuss the answers to the vignettes.

I. Work for pay or profit (Q20) - “work” and “pay or profit”

– asked in the first four vignettes

A. Answering the CPS

One of the questions asked was “Last week, did you do any work for either pay or profit?”

You answered: [look at sheet for their answers]

• Could you describe your employment situation to me, and explain why you said yes/no to this question?

• When you were answering the questions about yourself, was this question confusing or difficult?

• How about when you were answering for other family members?

B. Answering the Vignettes

Now, let’s talk about the vignettes. We asked “Last week, did you do any work for either pay or profit?” about Wendy, Cynthia, Elena, Cathy, and Pat.

• Take a look at the vignettes again, were there any scenarios that deciding if the person was working for ‘pay or profit’ was more difficult than others? Which? How did you make your final decision?

• If participant gave an incorrect answer to this question for any of the vignettes, talk about those specifically – or for the complicated vignette

C. Defining the term

• In the context of “Last week, did you do any work for either pay or profit?” what does the phrase “work for either pay or profit” mean to you?

• What is included under pay or profit that people might get from working, for yourself or for others?

II. Unpaid work in family business (Q20-1) - “unpaid work”

– asked in vignettes 2 and 4

Another question asked was “Last week did you do any unpaid work in the family business or farm?”

You answered: [look at sheet for their answers]

A. Answering the CPS – IF ASKED DURING INTERVIEW

• When you were answering the questions about yourself, was this question confusing or difficult?

• How about when you were answering for other family members?

B. Answering the Vignettes

Now, let’s talk about the vignettes. [IF ASKED UNPAID WORK] We asked “Last week did you do any unpaid work in the family business or farm?” in the vignettes with [FILL Cynthia and/or Cathy].

• Were there any scenarios where deciding whether an activity was unpaid work was more difficult than others? Which? How did you make your final decision?

• If participant gave an incorrect answer to this question for any of the vignettes, talk about those specifically – or for the complicated vignette

C. Defining the term

• How would you define “unpaid work?”

• Can you give me some examples of unpaid work that someone might do in a family business?

  • How about these examples? Would you say yes or no to unpaid work?

    • Work a few hours every week in a family business but don’t earn a salary

    • Doing the accounting for the family store but get paid directly

    • Picking peaches at the family farm and getting to take home as many peaches as you want, but don’t get paid cash

III. Temporarily Absent (Q20B-a.)

– asked in vignettes 1, 3, and 5

Another question asked “Last week, did you have a job either full or part time? Include any job from which you were temporarily absent.”

You answered: [look at sheet for their answers]

A. Answering the CPS – IF ASKED DURING INTERVIEW

• When you were answering the questions about yourself, was this question confusing or difficult?

• How about when you were answering for other family members?

B. Answering the Vignettes

Now, let’s talk about the vignettes. [IF ASKED JOB] We also asked “Last week, did you have a job either full or part time? Include any job from which you were temporarily absent” in the vignettes with [FILL Wendy, Elena, and/or Pat].

• Were there any scenarios that deciding whether the person was temporarily absent was more difficult than others? Which? How did you make your final decision?

• If participant gave an incorrect answer to this question for any of the vignettes, talk about those specifically – or for the complicated vignette

C. Defining the term

• In the context of the question “Last week, did you have a job either full or part time? Include any job from which you were temporarily absent,” how would you define a job?

• The question asks about jobs from which you may have been “temporarily absent.” What does that term “temporarily absent” mean to you?

• What types of situations would be classified as temporarily absent?

  • How about these examples? Would you say yes or no to being temporarily absent?

    • Maternity leave

    • 1 week paid vacation

    • 1 week unpaid vacation

    • Teacher on summer vacation

    • Landscaper off during the winter

IV. Layoff (Q20B-b)

– asked in vignettes 1, 3, and 5

Another question asked “Last week, were you on layoff from a job?”

You answered: [look at sheet for their answers]

A. Answering the CPS – IF ASKED DURING INTERVIEW

• When you were answering the questions about yourself, was this question confusing or difficult?

• How about when you were answering for other family members?

• If yes, indicated self or other HH member was on layoff:

  • "What did your employer say that made you think you were being laid off versus some other way to describe it?"

  • Probe to get the ‘right’ answers for Q21, Q21A and Q21A-1

• If no, indicated was not on layoff:

  • What about your situation made you decide that you were no on layoff?

  • How would you classify your situation

B. Answering the Vignettes

Now, let’s talk about the vignettes. [IF ASKED LAYOFF] We asked “Last week, were you on layoff from a job?” in the vignettes with [FILL Wendy, Elena, and/or Pat].

• Were there any scenarios that deciding whether the person was on layoff was more difficult than others? Which? How did you make your final decision?

• If participant gave an incorrect answer to this question for any of the vignettes, talk about those specifically – or for the complicated vignette

C. Defining the term

• Is this a term you’re familiar with?

  • If yes Could you tell me what the term layoff means to you?

  • If no what do you think it might mean

• I’d like to talk about definitions for several related terms, so we can understand how people talk about these things:

  • Is being on layoff different from being fired? What’s the difference?

  • Is being on layoff different from being temporarily absent? What’s the difference?

  • What about if you lost your job because your company downsized, would that be considered layoff? Why? Why not?

V. Other items on Layoff

For the vignette about Pat, we asked if the person had been given a date to return to work

1. What did you base your answer to that question?

2. Given the information you had available, was that question easy or difficult to answer? How so?

3. If participant gave an incorrect answer to this question for any of the vignettes, talk about those specifically

Finally we asked if Pat had been given any indication that they would be recalled to work in the next 6 months

1. What did you base that answer on?

2. Given Pat’s situation, were you clear about what the question meant?

3. What does being “recalled to work” mean to you?

4. Using the information you had available, was that question easy or difficult to answer? How so?

5. If participant gave an incorrect answer to this question for any of the vignettes, talk about those specifically

V. Ending Question

Is there anything else you’d like to tell me about this interview? Other feedback you think we should know about these questions?

Attachment III:

Consent agreement form and Privacy act statement

Consent Form

The Bureau of Labor Statistics (BLS) is conducting research to increase the quality of BLS surveys. This study is intended to suggest ways to improve the procedures the BLS uses to collect survey data.

The BLS, its employees, agents, and partner statistical agencies, will use the information you provide for statistical purposes only and will hold the information in confidence to the full extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act of 2002 (Title 5 of Public Law 107-347) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent. The Privacy Act notice on the back of this form describes the conditions under which information related to this study will be used by BLS employees and agents.

During this research you may be audio and/or videotaped, or you may be observed. If you do not wish to be taped, you still may participate in this research.

We estimate it will take you an average of 60 minutes to participate in this research.

Your participation in this research project is voluntary, and you have the right to stop at any time. If you agree to participate, please sign below.

Persons are not required to respond to the collection of information unless it displays a currently valid OMB control number. OMB control number is 1220-0141 and expires February 29^th, 2012.

------------------------------------------------------------------------------------------------------------

I have read and understand the statements above. I consent to participate in this study.

___________________________________ ___________________________

Participant's signature Date

___________________________________

Participant's printed name

___________________________________

Researcher's signature

OMB Control Number: 1220-0141

Expiration Date: [02/29/12]

PRIVACY ACT STATEMENT

In accordance with the Privacy Act of 1974, as amended (5 U.S.C. 552a), you are hereby notified that this study is sponsored by the U.S. Department of Labor, Bureau of Labor Statistics (BLS), under authority of 29 U.S.C. 2. Your voluntary participation is important to the success of this study and will enable the BLS to better understand the behavioral and psychological processes of individuals, as they reflect on the accuracy of BLS information collections. The BLS, its employees, agents, and partner statistical agencies, will use the information you provide for statistical purposes only and will hold the information in confidence to the full extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act of 2002 (Title 5 of Public Law 107-347) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent.

19