Form Appendix A-C Appendix A-C Interview Protocols

Appendix A: Individual Cognitive Interviewing Protocol

Cognitive Testing of the 2016 CPS

SOGI questions

Round 1

Cognitive Interview Protocol Guide

DRAFT 8.12.16

PARTICIPANT ID #: _________________________ DATE: ____ / ____ / 2016

COGNITIVE INTERVIEWER NAME:

START TIME: ______: ______ AM / PM END TIME: ______: ______ AM / PM

SECTION I. INTERVIEW CONSENT
	(Cognitive Interviewer: Read) PLACE THE CONSENT FORM IN FRONT OF PARTICIPANT

A. Hello, I’m [NAME OF COGNITIVE INTERVIEWER]. I work for the

(Census Bureau/Bureau of Labor Statistics). Thanks for agreeing to help me today.

{(If applicable) Observers:

[Before we get started let me mention that there are some observers watching the interview in another room/ This is XXXX]. [They are/He is/She is] only watching to get an idea of how the questions in the survey we are going to talk about are working.}

Before we start, I would like you to read over the document in front of you. This document explains a little bit about this interview and provides information about your rights as a participant. It also asks for your permission to have this session audio recorded. The main reason we tape these interviews is so that we don't have to rely on notes or our memories later. This allows me to concentrate on what you're saying during the interview. What you say on the tape will be used only for research purposes and cannot be told to anybody else not working on this project as required by law.

PARTICIPANT READS AND SIGNS FORM

Please ask me any questions you have about this document. Once you have finished reading the document, please sign it.

B. IF PARTICIPANT PROVIDES CONSENT TO HAVE THE SESSION AUDIO-TAPED: I will now turn on the audio recorder.

TEST AUDIO RECORDER AND TURN IT ON AGAIN AFTER TEST

A. Introduction

(Cognitive Interviewer: Read/Paraphrase)

“Let me begin by telling you a little more about what we’ll be doing today. With your help, we will be testing some new questions that were developed for a national survey called the Current Population Survey, which is jointly conducted by the Census Bureau and the Bureau of Labor Statistics, and is the primary source of labor force statistics for the United States, like employment and unemployment. I will first ask you some survey questions and you’ll answer the questions just like you would if you were doing it with an interviewer in a regular survey.

Our goal here is to get a better idea how well the new questions work with people such as yourself before we actually use them in the survey. So the purpose of our session today is to collect your thoughts and opinions on these new questions and materials. I just want to remind you that I'm only asking for your thoughts and opinions on these materials. There is no right or wrong answer. We are simply trying to make sure the question wording is clear and easy for most people to understand. Your feedback will be very useful for helping make sure these questions will make sense to other people.

Your participation in this interview and the review of the materials is very important because it will help the Census Bureau and Bureau of Labor Statistics with improving these questions.

Do you have any questions before we begin?”

SECTION II. COGNITIVE INTERVIEW

As we go through, feel free to share any reactions or comments you have about the questions. Especially let me know if you find anything confusing, surprising, difficult or you have other reactions.

Q1-2. Ask for each member of the household as you build the roster.

Okay, let’s get started. I’ll ask the series of questions from the CPS, please answer for your household.

1. NAME Now I will ask you some questions about the people who live with you. What are the names of all persons living or staying with you? Start with the name of the person, or one of the persons, who owns or rents your home.

2. RES Is this [your/NAME’s] usual place of residence?

Q3-5. Ask once for household.

3. MCHILD

I have listed . . . READ NAMES

Have I missed any babies or small children?

1 Yes (who is that? Add to roster)

2 No

4. MAWAY

Have I missed anyone who usually lives with you but is away now -traveling, at school, or in a hospital?

1 Yes (who is that? Add to roster)

2 No

5. MBOARDER

Have I missed any lodgers, boarders, or persons you employ who live with you?

1 Yes (who is that? Add to roster)

2. No

For Q 6 -7 complete both questions before moving to the next household member. Complete all household members before moving to sex questions.

6. DOB What is [your/NAME’s] date of birth?

7. AGE That means [you/they] are [AGE calculated from DOB] is that correct? (If 15 or over go to SEXB if under 15 go to SEXA)

For household members under 15 complete question 8 for household members 15 and over complete questions 9 and 10. Complete sex questions for all household members before moving to relationship question. Complete relationship question for all household members before moving to next question

8. SEXA What is [your/NAME’s] sex? Skip to 11 REL

9. SEXB

[Was your/To the best of your knowledge, was [NAME]’s] sex recorded as male or female at birth?

1. Male

2. Female

3. DK

4. REF

10. Gender [Do you/To the best of your knowledge, does [NAME]] describe [yourself/themselves] as male, female, or transgender?

1. Male

2. Female

3. Transgender

4. DK

5. REF

11. REL IF NOT the person on line 1 of the roster (card A): How [are/is] [you/NAME] related to [fill name of person on line 1 of the Household Roster]?

1. Opposite sex spouse (husband/wife)

2. Opposite sex unmarried partner

3. Same sex spouse (husband/wife)

4.Same sex unmarried partner

5. Child

6. Grandchild

7. Parent

8. Brother/ sister

9. Other relative (aunt, uncle, cousin, in law)

10. Foster child

11. Housemate/roommate

12. Roomer/boarder

13. Other nonrelative

14. DK

15. REF

99. self

For Q12-16 complete questions 12-16 before moving to the next person. Complete the set of questions for everyone in the household before moving to education. Complete these questions only for respondents 15 and over.

12. MARRTL [Are/is] [You/NAME] Currently married, widowed, divorced, separated or never married?

1. Married Proceed to 18. WHOSPOUSE

2. Widowed

3. Divorced

4. Separated Skip to 19. PARTNER

5. Never married

6. DK

7. REF

13. WHOSPOUSE Who is [your/[NAME]’s] spouse? Skip to 21. IDENTITY

14. PARTNER [Do/Does] [You/NAME] have a boyfriend, girlfriend, or partner living in the household? Only for people with no spouse

1. Yes Proceed to 20. WHOPARTNER

2. No

3. DK Skip to 21. IDENTITY

4. REF

15. WHOPARTNER Who is [your/[NAME]’s] partner?

16. IDENTITY [Which/To the best of your knowledge, which] of the following best represents how [you think of yourself/[NAME] thinks of their self]?

1. Gay or Lesbian

2. Straight, that is not gay, lesbian, or bisexual

3. Bisexual

4. Something else

5. DK

6. REF

For Q17complete for everyone in the household before moving to military. Complete these questions only for respondents 15 and over.

17. EDUCATION What is the highest level of school [you/[NAME]] has completed or the highest degree [you/[NAME]] [have/has] received?

(CARD B)

1. 1^st, 2^nd , 3^rd, or 4^th grade

2. 5^th or 6^th grade

3. 7^th or 8^th grade

4. 9^th grade

5. 10^th grade

6. 11^th grade

7. 12^th grade, no diploma

8. High school graduate-high school diploma or equivalent

9. Some college – no diploma

10. Associates degree- occupational / vocational program

11. Associates degree- academic program

12. Bachelors degree (BA, BS, AB)

13. Master’s degree (MS,MA, MBA, MEd, MEng,MSW)

14. Professional degree (MD,DDS,DMV,LLB, JD)

15. Doctorate degree (PhD, EdD)

For Q18-19 complete questions 18-19 before moving to the next person. Complete the set of questions for everyone in the household before moving to HISPA. Complete these questions only for respondents 15 and over.

18. MILITARY Did [You/NAME] ever serve on active duty in the U.S. Armed forces?

1. Yes Proceed to 24. INMILITARY

2. No

3. DK Skip to 26.HISPA

4. REF

19. INMILITARY [Are/is] [You/NAME] still in the Armed forces?

1. Yes

2. No

3. DK

4. REF

For Q20-22 complete questions 20-22 before moving to the next person. Complete the set of questions for everyone in the household before moving to RACEA

20. Hisp A [Are/is] [You/NAME] Spanish, Hispanic , or Latino?

1. Yes

2. No

3. DK

4. REF

21. RACEA I am going to read you a list of five race categories. Please choose one or more races that (you/NAME) (consider yourself/ himself/considers herself) to be: White; Black or African American; American Indian or Alaska Native; Asian; OR Native Hawaiian or Other Pacific Islander. (CARD C)

1. White

2. Black or African American

3. American Indian or Alaska Native Skip to work intro.

4. Asian

5. Native Hawaiian or Other Pacific Islander

6. Other – DO NOT READ > Proceed to 28. RACEB

22. RACEB What is your race?

For Q23-44 complete questions 23-44 before moving to the next person. Complete the set of questions for everyone in the household before moving to disability intro Complete these questions only for respondents 15 and over.

Next, I am going to ask a few questions about work related activities LAST WEEK. By last week I mean the week beginning on Sunday, ____ and ending on Saturday, ___

23. BUSINESS Does anyone in this household have a business or farm?

1.Yes
2. No
3. Don't Know
4. Refused

24. WORK LAST WEEK, did [YOU/NAME] do ANY work for (either) pay (or profit)?

1.Yes skip to 32. LWWORK
2. No
3. Don't Know if 23. BUSINESS=1 proceed to 25.JOB
4. Refused if BUSINESS= 2 skip to 40. WFIND

25. JOB LAST WEEK, (in addition to the business,) did [you/name] have a job either full or part time? Include any job from which [you/name] were temporarily absent.

1.Yes skip to 27. ABSENT
2. No
3. Don't Know proceed to 26. LAYOFF
4. Refused

26. LAYOFF LAST WEEK, were you on layoff from a job?

1.Yes skip to 28. RETURN
2. No go to
3. Don't Know skip to 40. WFIND
4. Refused

27. ABSENT What was the main reason [you/name] were absent from work LAST WEEK?

1. On Layoff proceed to 28. RETURN
2. Slack Work/business conditions
3. Waiting for a new job to begin
4. Own illness/injury/medical problems
5. Does not work in the business skip to 32. LWWORK
6. Other
7. Don't Know
8. Refused

28. RETURN Has your employer given you a date to return to work?

1.Yes
2. No
3. Don't Know
4. Refused

29. R6MONTH Have you been given any indication that you will be recalled to work within the next 6 months?

1.Yes
2. No
3. Don't Know
4. Refused

30. RECALLED Could you have returned to work LAST WEEK if you had been recalled?

1.Yes
2. No
3. Don't Know
4. Refused

31. WHY1 Why is that?

1.Own Temporary Illness
2. Going to School
3. Other (Specify)
4. Don't Know
5. Refused

IF 25. JOB=1 proceed to32. LWWORK IF 25. JOB=2 skip to WFIND

32. LWWORK LAST WEEK, did [you/name] have more than one job, including part-time, evening or weekend work?

1. Yes

2. No

3. Don’t Know

4. Refused

32. HOURSM How many hours per week do [you/name] USUALLY work at [your/his/her] (main) job?
    
    By main job we mean the one at which you USUALLY work the most hours.

32. HOURSO How many hours per week do [you/name] USUALLY work at [your/his/her] other job(s)? If 32. LMWORK = 1

32. EMPTYPE Now I have a few questions about the (main) job -

(a) (at which you worked LAST WEEK. If 24. WORK=1)

(b) (from which you were absent LAST WEEK. IF 24. WORK=2)

Were [you/name] employed by government, by a private company, a non-profit organization, or [were you/was name] self-employed or working in a family business?

1. Government

2. Private company

3. Non-profit organization incl. tax exempt and charitable organizations

4. Self-employed

5. Working in family business

32. COMPANY What is the name of the (company)(non-profit organization) (business) for which [you/name] work?

32. INDUSTRY What kind of business or industry is this? READ IF NECESSARY What do they make or do where you work?

32. OCCUPATION What kind of work (do you/does NAME) do, that is, what is (your/NAME’s) occupation?

32. DUTIES What (are) your usual activities or duties at this job?

(For example: typing, keeping account books, filing, selling cars, operating printing press, laying brick)

Skip to to 44. DIS1

40. WFIND Have you been doing anything to find work during the last 4 weeks?

1.Yes proceed to 41. HOWFIND
2. No
3. Don't Know skip to STARTJOB
4. Refused

41. HOWFIND What are all of the things you have done to find work during the last 4 weeks?

Active: 1. Contacted Employer directly/interview 2. Contacted public Employment agency 3. Contated private employment agency 4. Contacted Friends or relatives 5. Contacted School/university employment center 6. Sent out resumes/filled out applications 7. Check union/professional registers 8. Place or answer ads 9. Other Active

Passive: 10. Looked at Ads 11. Attended job training programs/Courses 12. Other Passive 13. Nothing 14. Don’t Know 15. Refused

42. STARTJOB LAST WEEK, could you have started a job if one had been offered?

1.Yes skip to 44. DIS1
2. No proceed to 43. WHY2
3. Don't Know
4. Refused skip to DIS1

43. WHY2 Why is that?

1. Waiting for a new job to begin
2. Own Temporary Illness
3. Going to School
4. Other
5. Don't Know
6. Refused

For Q44-55 complete before moving to Nativity

We want to learn about people who have physical, mental, or emotional conditions that cause serious difficulty with their daily activities.

44. DIS1 [Are you/ is[NAME]/ is anyone] Deaf or [do you/does [NAME]/does anyone] have serious difficulty hearing?

1. Yes Proceed to 45.DIS1WHO

2. No

3. DK Skip to 46. DIS2

4. REF

45. DIS1WHO Who is that? Anyone else?

46. DIS2 [Are you/ is[NAME]/ is anyone] blind or [do you/does [NAME]/does anyone ]have serious difficulty seeing, even when wearing glasses?

1. Yes Proceed to 47.DIS2WHO

2. No

3. DK Skip to 48. DIS3

4. REF

47. DIS2WHO Who is that? Anyone else?

48. DIS3 Because of a physical, mental, or emotional condition, [do you/does [NAME]/ does anyone] have serious difficulty concentrating, remembering, or making decisions?

1. Yes Proceed to 49.DIS3WHO

2. No

3. DK Skip to 50. DIS4

4. REF

49. DIS3WHO Who is that? Anyone else?

50. DIS4 [Do you/does [NAME]/ does anyone] have serious difficulty walking or climbing stairs?

1. Yes Proceed to 51.DIS4WHO

2. No

3. DK Skip to 52. DIS5

4. REF

51. DIS4WHO Who is that? Anyone else?

52. DIS5 [Do you/does [NAME]/ does anyone] have difficulty dressing or bathing?

1. Yes Proceed to 53.DIS5WHO

2. No

3. DK Skip to 54. DIS6

4. REF

53. DIS5WHO Who is that? Anyone else?

54. DIS6 Because of a physical, mental, or emotional condition, [Do you/does [NAME]/ does anyone] have difficulty doing errands alone such as visiting a doctor's office or shopping?

1. Yes Proceed to 55.DIS6WHO

2. No

3. DK Skip to 56 NATIVITYA

4. REF

55. DIS6WHO Who is that? Anyone else?

For Q56-60 complete questions 56-60 before moving to the next person. Complete the set of questions for everyone in the household before moving to Income

56. NATIVITYA In what Country [were/was] [you/[NAME]] born?

57. NATIVITYB In what Country was [your/[NAME’s]] mother born?

58. NATIVITYC In what Country was [your/[NAME’s]] father born?

If person was born in the U.S., skip to 61; any other answer, proceed to 59.

59. CITIZEN [ARE/Is] [you/[NAME]] a citizen of the United States?

1.Yes
2. No
3. Don't Know
4. Refused

60. YEARUS In what year did [You/[NAME]] come to live in the United States?

61. Income Which category represents [your/name of reference person/the total combined income] [total combined income during the past 12 months/of all members of [your/this] FAMILY during the past 12 months]?

This includes money from jobs, net income from business, farm or rent, pensions, dividends, interest, social security payments and any other money income received (by members of this FAMILY who are 15 years of age or older. CARD D)

1. Less that $5,000

2. 5,000 to 7,499

3. 7,500 to 9,999

4. 10,000 to 12,499

5. 12,500 to 14,999

6. 15,000 to 19,999

7. 20,000 to 24,999

8. 25,000 to 29,999

9. 30,000 to 34,999

10. 35,000 to 39,999

11. 40,000 to 49,999

12. 50,000 to 59,999

13. 60,000 to 74,999

14. 75,000 to 99,999

15. 100,000 to 149,999

16. $150,000 or over

After the interview:

SECTION III. DEBRIEFING QUESTIONS

Now, I am going to ask you some questions about your answers and about the questions themselves. I am really interested in how these questions work for you, so there are no right or wrong answers.

• Overall, what did you think of the interview?

• Was the interview easy or difficult?

• What topics did we cover in this interview? What other topics were covered?

• Did any of the questions stand out for any reason? Which ones? Can you tell me why (question) stood out to you?

• Did any of the questions bother you? Which ones? Can you tell me why (question) bothered you?

In the survey we’re looking at, we ask one person to answer questions for everyone in the household.  Most of the questions are only for household members 15 and older, since it doesn’t make sense to ask about employment for younger children.  For the questions we’ll be focusing on today, we’re only asking about people in your house who are 15 or over.

I’d like to talk about a few particular items from the survey. The first one I’d like to talk about asked “Because of a physical, mental, or emotional condition, does anyone have serious difficulty concentrating, remembering, or making decisions?

• In your own words, what is this question asking?

• What was your overall reaction to this question?

• How did you go about answering this question for yourself?

  • What about for the other people in your household?

• Did you have any difficulty answering this question? Probe if necessary, for yourself or someone else in your household?

• How certain are you about your responses for other people in the household?

• This survey is often conducted in people’s homes. If other members of your household were in the room, would you be comfortable answering this question? For yourself? For others?

• If someone else in the household were answering this question about you how certain are you that they could answer the way you did?

  • Would they be able to accurately answer this type of question about you?

  • Would they be able to accurately answer this type of question about everyone in the house?

• Do you think that there are particular types of government surveys that should be asking this question? What types of surveys?

  • What about employment surveys, what would your reaction be if this question was asked on a government survey about employment?

Moving to another question. This one asked “Was your sex recorded as male or female at birth?”

• Do you remember that question?

• In your own words what is this question asking?

• What was your reaction to this question?

• How did you go about answering this question for yourself?

  • What about for the other people in your household?

• Did you have any difficulty answering this question for yourself or anyone else in your household?

• How certain are you about your responses for other people in the household?

• If other members of your household were in the room, would you be comfortable answering this question? For yourself? For others?

• If someone else in the household were answering this question about you, how certain are you that they would give the answer you did? [specify someone over 15 if needed]

  • Would they be able to accurately answer this type of question about you?

  • Would they be able to accurately answer this type of question about everyone in the house?

• Do you think that there are particular types of government surveys that should be asking this question? What types of surveys?

  • What about employment surveys, what would your reaction be if this question was asked on a government survey about employment?

Moving to another question I’d like to get your feedback on. It asked “Do you describe yourself as male, female or transgender?”

• In your own words, what is this question asking?

• What was your overall reaction to this question?

• How did you go about answering this question for yourself?

  • What about for the other people in your household?

• Did you have any difficulty answering this question? Probe if necessary, for yourself or someone else in your household?

• What does transgender mean to you?

• How certain are you about your responses for other people in the household?

• If other members of your household were in the room, would you be comfortable answering this question? For yourself? For others?

• If someone else in the household were answering this question about you, how certain are you that they could answer the way you did?

  • Would they be able to accurately answer this type of question about you?

  • Would they be able to accurately answer this type of question about everyone in the house?

• Do you think that there are particular types of government surveys that should be asking this question? What types of surveys?

  • What about employment surveys, what would your reaction be if this question was asked on a government survey about employment?

Moving to another question. This one asked Which category represents the total combined income of all members of your FAMILY during the past 12 months?

• In your own words, what is this question asking?

• What was your overall reaction to this question?

• How did you go about calculating the answer to this question?

• Did you have any difficulty answering this question?

• How certain are you about your inclusion of income information for other members of your family?

• If other members of your household were in the room, would you be comfortable answering this question?

• If someone else in the household were answering this question about you how certain are you that they could answer the way you did?

  • Would they be able to accurately answer this type of question about you?

  • Would they be able to accurately answer this type of question about everyone in the house?

• Do you think that there are particular types of government surveys that should be asking this question? What types of surveys?

  • What about employment surveys, what would your reaction be if this question was asked on a government survey about employment?

We have another question we’d like your feedback on. It asks “which of the following best represents how you think of yourself?”

Give them a copy of the identity question (card H)

• In your own words, what is this question asking?

• What was your overall reaction to this question?

• How did you go about answering this question for yourself?

  • What about for the other people in your household?

• Did you have any difficulty answering this question for yourself or anyone else in your household?

• How certain are you about your responses for other people in the household?

• Do the answer choices make sense to you?

• Are there any additional answer choices that should be included?

• If other members of your household were in the room, would you be comfortable answering this question? For yourself? For others?

• If someone else in the household were answering this question about you how certain are you that they could answer the way you did?

  • Would they be able to accurately answer this type of question about you?

  • Would they be able to accurately answer this type of question about everyone in the house?

• Do you think that there are particular types of government surveys that should be asking this question? What types of surveys?

  • What about employment surveys, what would your reaction be if this question was asked on a government survey about employment?

One last question I want to ask about, which asked LAST WEEK, did [you/name] have more than one job, including part-time, evening or weekend work?

• In your own words, what is this question asking?

• How did you go about answering this question for yourself?

  • What about for the other people in your household?

• Did you have any difficulty answering this question for yourself or anyone else in your household?

• How certain are you about your responses for other people in the household?

• If other members of your household were in the room, would you be comfortable answering this question? For yourself? For others?

• If someone else in the household were answering this question about you how certain are you that they could answer the way you did?

  • Would they be able to accurately answer this type of question about you?

  • Would they be able to accurately answer this type of question about everyone in the house?

• Do you think that there are particular types of government surveys that should be asking this question? What types of surveys? What about employment surveys?

Card Sort

Now we are going to switch to a different task. I am going to give you a stack of cards with survey questions on them. I would like you to sort them into two piles. Place the questions that you found difficult to answer about yourself in one pile and questions you did not find difficult to answer about yourself in another pile.

AFTER TASK IS COMPLETE LAY OUT CARDS SO THEY CAN BE SEEN – ONE LINE FOR “DIFFICULT” QUESTIONS AND ANOTHER FOR “NOT DIFFICULT”

AFTER TASK IS COMPLETE

• How did you decide what to put in the difficult pile?

• Please describe the questions in the ‘difficult’ pile

• What do they have in common?

• Other than difficult, what words would you use to describe this pile?

• Please describe the questions in the ‘not difficult’ pile

• Do you think other people in your household would sort them the same way? What would they do differently?

• Would you have sorted them differently if we had asked about how difficult’ they were to answer about someone else in the household instead of you? How would it have been different?

Take the cards with questions you found difficult and place them in order from top to bottom with the top card being the most difficult and the bottom card being the least difficult.

• So you think this is the most difficult question?

• What did you find most difficult about XXX [the top card]?

  • What about it makes it difficult?

• Is the question at the bottom of the pile more difficult than the questions in the not difficult pile?

Let’s look at your piles again.

• Do you think other people in your household would sort them the same way? What would they do differently?

• Thinking about answering these questions for the other people in your household instead of answering for yourself, do you want to move any of the cards between the piles? Feel free to do so

• What makes that question/those questions more/less difficult to answer about someone else instead of yourself?

TAKE BACK CARDS AND SHUFFLE THEM

I would now like you to sort the cards into two piles again. This time, place the questions that you found sensitive when answering about yourself in one pile and questions you did not find sensitive to answer in another pile.

AFTER TASK IS COMPLETE LAY OUT CARDS SO THEY CAN BE SEEN – ONE LINE FOR “SENSITIVE” QUESTIONS AND ANOTHER FOR “NOT SENSITIVE”

• How did you decide what to put in the sensitive pile?

• Please describe the questions in the ‘sensitive’

  • What do they have in common?

• Other than sensitive, what words would you use to describe this pile?

• Please describe the questions in the ‘not sensitive’ pile

• What do they have in common?

• Do you think other people in your household would sort them the same way? What would they do differently?

• Would you have sorted them differently if we had asked about how sensitive they were to answer about someone else in the household instead of you? How would it have been different?

Take the cards with questions you found difficult and place them in order from top to bottom with the top card being the most difficult and the bottom card being the least sensitive.

• So you think this is the most sensitive question?

• What about it makes it sensitive?

• Is the question at the bottom of the pile more sensitive than the questions in the not sensitive pile?

Let’s look at your piles again.

• Do you think other people in your household would sort them the same way? What would they do differently?

• Thinking about answering these questions for the other people in your household instead of answering for yourself, do you want to move any of the cards between the piles? Feel free to do so

• What makes that question/those questions more/less sensitive about someone else instead of yourself?

Question Wording

As I said in the beginning, we’re looking at question wording. For one of the questions on the survey, we have another way we could ask it. Here is a copy of the wording you were asked and a different wording option, it says “Was your sex recorded as male or female on your birth certificate?”

Give them a copy of the original and alternate questions (card F and G)

• What’s the difference between these two versions?

• What is your overall reaction to this version?

• Do you think the second version of the question is easier or harder to understand? How so?

• Which version of the question do you prefer? Can you tell me what about it you like better?

Again, we have another question with another version of the wording. It says “Do you describe yourself as male, female, transgender or something else?”

Give them the alternate gender question (card H and I)

• What’s the difference between these two versions?

• In your opinion, is this question the same as or different than the original?

• What do you think the ‘something else’ is referring to?

• Do you think the second version of the question is easier or harder to understand? How so?

• Which version of the question do you prefer? Can you tell me what about it you like better?

• If reported difficulty with first version: would this version make the question easier to answer than the other?

• If reported discomfort with first version: How would you feel answering this version of the question with other household members present?

Closing Comments

As you might have guessed, we are thinking about asking sexual orientation and gender identity in Current Population Survey, a government survey about employment.

• What do you think about that idea?

• Why do you think the government is interested in sexual orientation and gender identity in a survey about employment?

• If you were asked to provide information about your sexual orientation and gender identity in a government survey about employment, how would you react?

  • How might others in your household react?

• If you were asked to provide information about the sexual orientation and gender identity of everyone else in your household over 15, how would you react?

  • How might others in your household react?

• Is there anything that we could do differently to make it as easy and comfortable as possible for people to provide the information?

• Do you have anything else you would like to tell us about the collection of information about sexual orientation and gender identity?

Closing

That’s all the questions I have for you, thanks for your feedback! Before you go,

• Do you have anything else you would like to tell us that you haven’t had a chance to mention yet?

I want to thank you very much for your participation. I will now give you $40 and I will ask you to sign a receipt form verifying that you received the money.

TURN OFF THE RECORDER. HAND THE CASH INCENTIVE AND VOUCHER TO THE PARTICIPANT.

SECTION IV. flashcards, question cards and, card sorting questions

Answer category cards

Card A

RELATIONSHIP

(42) Opposite-sex Spouse (Husband/Wife)

(43) Opposite-sex Unmarried partner

(44) Same-sex Spouse (Husband/Wife)

(45) Same-sex Unmarried partner

(46) Child

(47) Grandchild

(48) Parent (Mother/Father)

(49) Brother/Sister

(50) Other relative (Aunt, Cousin, Nephew, Mother-in-law, etc.)

(51) Foster child

(52) Housemate/Roommate

(53) Roomer/Boarder

(54) Other nonrelative

Card B

Education

(31) Less than 1st grade

(32) 1st, 2nd, 3rd, or 4th grade

(33) 5th or 6th grade

(34) 7th or 8th grade

(35) 9th grade

(36) 10th grade

(37) 11th grade

(38) 12th grade NO DIPLOMA

(39) HIGH SCHOOL GRADUATE – high school DIPLOMA or the equivalent (For example: GED)

(40) Some college but no degree

(41) Associate degree in college – Occupational/vocational program

(42) Associate degree in college – Academic program

(43) Bachelor’s degree (For example: BA, AB, BS)

(44) Master’s degree (For example: MA, MS, MEng, MEd, MSW, MBA)

(45) Professional School Degree (For example: MD, DDS, DVM, LLB, JD)

(46) Doctorate degree (For example: PhD, EdD)

Card C

RACE

CHOOSE ONE OR MORE

1. White

2. Black or African American

3. American Indian or Alaska Native

4. Asian

5. Native Hawaiian or Other Pacific Islander

Card D

INCOME

Which category on this card represents the total combined income of all members of this FAMILY during the past 12 months? This includes money from jobs, net income from business, farm or rent, pensions, dividends, interest, social security payments and any other money income received by members of this FAMILY who are 15 years of age or older.

(1) Less than $ 5,000 (9) $30,000 to $34,999

(2) $ 5,000 to $ 7,499 (10) $35,000 to $39,999

(3) $ 7,500 to $ 9,999 (11) $40,000 to $49,999

(4) $10,000 to $12,499 (12) $50,000 to $59,999

(5) $12,500 to $14,999 (13) $60,000 to $74,999

(6) $15,000 to $19,999 (14) $75,000 to $99,999

(7) $20,000 to $24,999 (15) $100,000 to $149,999

(8) $25,000 to $29,999 (16) $150,000 or more

Question alternate text

Card E

Which of the following best represents how you think of yourself?

1. Gay or Lesbian

2. Straight, that is not gay, lesbian, or bisexual

3. Bisexual

4. Something else

Card F

Was your sex recorded as male or female at birth?

5. Male

6. Female

Was [NAME]’s sex recorded as male or female at birth?

1. Male

2. Female

Card G

Was your sex recorded as male or female on your birth certificate?

1. Male

2. Female

Was [NAME]’s sex recorded as male or female on their birth certificate?

1. Male

2. Female

Card H

Do you describe yourself as male, female, or transgender?

6. Male

7. Female

8. Transgender

Does [NAME] describe themselves as male, female, or transgender?

1. Male

2. Female

3. Transgender

Card I

Do you describe yourself as male, female, transgender or something else?

1. Male

2. Female

3. Transgender

4. Something else

Does [Name] describe themselves as male, female, transgender or something else?

1. Male

2. Female

3. Transgender

4. Something else

Card Sort Questions (will be shuffled to appear in random order- numbered here to make them easy to read)

1. What is your date of birth?

2. Was your sex recorded as male or female at birth?

3. Do you describe yourself as male, female, or transgender?

4. Are You Currently married, widowed, divorced, separated or never married?

5. Do You have a boyfriend, girlfriend, or partner living in the household?

6. Which of the following best represents how you think of yourself?

Gay or lesbian

Straight, that is not gay, lesbian or bisexual

Bisexual

Something else

7. Did You ever serve on active duty in the U.S. Armed forces?

8. What is the highest level of school you have completed or the highest degree you have received?

9. Are you Spanish, Hispanic , or Latino?

10. I am going to read you a list of five race categories. Please choose one or more races that you consider yourself to be: White; Black or African American; American Indian or Alaska Native; Asian; OR Native Hawaiian or Other Pacific Islander.

11. LAST WEEK, did YOU do ANY work for either pay or profit?

12. Is anyone Deaf or does anyone have serious difficulty hearing?

13. Because of a physical, mental, or emotional condition, does anyone have serious difficulty concentrating, remembering, or making decisions?

14. Because of a physical, mental, or emotional condition, does anyone have difficulty doing errands alone such as visiting a doctor's office or shopping?

15. In what Country were you born?

16. In what Country was your mother born?

17. In what Country was your father born?

18. What is the name of the (company)(non-profit organization) (business) for which you work?

19. Which category represents the total combined income of all members of your FAMILY during the past 12 months? This includes money from jobs, net income from business, farm or rent, pensions, dividends, interest, social security payments and any other money income received (by members of this FAMILY who are 15 years of age or older.)

Appendix B: Paired Cognitive Interviewing Protocol

Cognitive Interview: Individual Interviews Protocol

PARTICIPANT ID #: _________________________ DATE: ____ / ____ / 2016

COGNITIVE INTERVIEWER NAME:

START TIME: ______: ______ AM / PM END TIME: ______: ______ AM / PM

SECTION I. INTERVIEW CONSENT
	(Cognitive Interviewer: Read) PLACE THE CONSENT FORM IN FRONT OF PARTICIPANT

A. Hello, I’m [NAME OF COGNITIVE INTERVIEWER]. I work for the

(Census Bureau/Bureau of Labor Statistics). Thanks for agreeing to help me today.

{(If applicable) Observers:

[Before we get started let me mention that there are some observers watching the interview in another room/ This is XXXX]. [They are/He is/She is] only watching to get an idea of how the questions in the survey we are going to talk about are working.}

Before we start, I would like you to read over the document in front of you. This document explains a little bit about this interview and provides information about your rights as a participant. It also asks for your permission to have this session audio recorded. The main reason we tape these interviews is so that we don't have to rely on notes or our memories later. This allows me to concentrate on what you're saying during the interview. What you say on the tape will be used only for research purposes and cannot be told to anybody else not working on this project as required by law.

PARTICIPANT READS AND SIGNS FORM

Please ask me any questions you have about this document. Once you have finished reading the document, please sign it.

B. IF PARTICIPANT PROVIDES CONSENT TO HAVE THE SESSION AUDIO-TAPED: I will now turn on the audio recorder.

TEST AUDIO RECORDER AND TURN IT ON AGAIN AFTER TEST

A. Introduction

(Cognitive Interviewer: Read/Paraphrase)

“Let me begin by telling you a little more about what we’ll be doing today. With your help, we will be testing some new questions that were developed for a national survey called the Current Population Survey, which is jointly conducted by the Census Bureau and the Bureau of Labor Statistics, and is the primary source of labor force statistics for the United States, like employment and unemployment. I will first ask you some survey questions and you’ll answer the questions just like you would if you were doing it with an interviewer in a regular survey.

Our goal here is to get a better idea how well the new questions work with people such as yourself before we actually use them in the survey. So the purpose of our session today is to collect your thoughts and opinions on these new questions and materials. I just want to remind you that I'm only asking for your thoughts and opinions on these materials. There is no right or wrong answer. We are simply trying to make sure the question wording is clear and easy for most people to understand. Your feedback will be very useful for helping make sure these questions will make sense to other people.

Your participation in this interview and the review of the materials is very important because it will help the Census Bureau and the Bureau of Labor Statistics with improving these questions.

Do you have any questions before we begin?”

SECTION II. COGNITIVE INTERVIEW

As we go through, feel free to share any reactions or comments you have about the questions. Especially let me know if you find anything confusing, surprising, difficult or you have other reactions.

Q1-2. Ask for each member of the household as you build the roster.

1. NAME Now I will ask you some questions about the people who live with you. What are the names of all persons living or staying with you? Start with the name of the person, or one of the persons, who owns or rents your home.

2. RES Is this [your/NAME’s] usual place of residence?

Q3-5. Ask once for household.

3. MCHILD I have listed . . . READ NAMES

Have I missed any babies or small children?

1 Yes (who is that? Add to roster)

2 No

4. MAWAY

Have I missed anyone who usually lives with you but is away now -traveling, at school, or in a hospital?

1 Yes (who is that? Add to roster)

2 No

5. MBOARDER

Have I missed any lodgers, boarders, or persons you employ who live with you?

1 Yes (who is that? Add to roster)

2. No

For Q 6 -7 complete both questions before moving to the next household member. Complete all household members before moving to sex questions.

6. DOB What is [your/NAME’s] date of birth?

7. AGE That means [you/they] are [AGE calculated from DOB] is that correct? (If 15 or over go to SEXB if under 15 go to SEXA)

For household members under 15 complete question 8 for household members 15 and over complete questions 9 and 10. Complete sex questions for all household members before moving to relationship question. Complete relationship question for all household members before moving to next question

8. SEXA What is [your/NAME’s] sex? Skip to 11 REL

9. SEXB [Was your/To the best of your knowledge, was [NAME]’s] sex recorded as male or female at birth?

1. Male

2. Female

3. DK

4. REF

10. Gender [Do you/To the best of your knowledge, does [NAME]] describe [yourself/themselves] as male, female, or transgender?

1. Male

2. Female

3. Transgender

4. DK

5. REF

11. REL IF NOT the person on line 1 of the roster (card A): How [are/is] [you/NAME] related to [fill name of person on line 1 of the Household Roster]?

1. Opposite sex spouse (husband/wife)

2. Opposite sex unmarried partner

3. Same sex spouse (husband/wife)

4.Same sex unmarried partner

5. Child

6. Grandchild

7. Parent

8. Brother/ sister

9. Other relative (aunt, uncle, cousin, in law)

10. Foster child

11. Housemate/roommate

12. Roomer/boarder

13. Other nonrelative

14. DK

15. REF

99. self

For Q12-16 complete questions 12-16 before moving to the next person. Complete the set of questions for everyone in the household before moving to education. Complete these questions only for respondents 15 and over.

12. MARRTL [Are/is] [You/NAME] Currently married, widowed, divorced, separated or never married?

1. Married Proceed to 18. WHOSPOUSE

2. Widowed

3. Divorced

4. Separated Skip to 19. PARTNER

5. Never married

6. DK

7. REF

13. WHOSPOUSE Who is [your/[NAME]’s] spouse? Skip to 21. IDENTITY

14. PARTNER [Do/Does] [You/NAME] have a boyfriend, girlfriend, or partner living in the household? Only for people with no spouse

1. Yes Proceed to 20. WHOPARTNER

2. No

3. DK Skip to 21. IDENTITY

4. REF

15. WHOPARTNER Who is [your/[NAME]’s] partner?

16. IDENTITY [Which/To the best of your knowledge, which] of the following best represents how [you think of yourself/[NAME] thinks of their self]?

1. Gay or Lesbian

2. Straight, that is not gay, lesbian, or bisexual

3. Bisexual

4. Something else

5. DK

6. REF

For Q17complete for everyone in the household before moving to military. Complete these questions only for respondents 15 and over.

17. EDUCATION What is the highest level of school [you/[NAME]] has completed or the highest degree [you/[NAME]] [have/has] received?

(CARD B)

1. 1^st, 2^nd , 3^rd, or 4^th grade

2. 5^th or 6^th grade

3. 7^th or 8^th grade

4. 9^th grade

5. 10^th grade

6. 11^th grade

7. 12^th grade, no diploma

8. High school graduate-high school diploma or equivalent

9. Some college – no diploma

10. Associates degree- occupational / vocational program

11. Associates degree- academic program

12. Bachelors degree (BA, BS, AB)

13. Master’s degree (MS,MA, MBA, MEd, MEng,MSW)

14. Professional degree (MD,DDS,DMV,LLB, JD)

15. Doctorate degree (PhD, EdD)

For Q18-19 complete questions 18-19 before moving to the next person. Complete the set of questions for everyone in the household before moving to HISPA. Complete these questions only for respondents 15 and over.

18. MILITARY Did [You/NAME] ever serve on active duty in the U.S. Armed forces?

1. Yes Proceed to 24. INMILITARY

2. No

3. DK Skip to 26.HISPA

4. REF

19. INMILITARY [Are/is] [You/NAME] still in the Armed forces?

1. Yes

2. No

3. DK

4. REF

For Q20-22 complete questions 20-22 before moving to the next person. Complete the set of questions for everyone in the household before moving to RACEA

20. Hisp A [Are/is] [You/NAME] Spanish, Hispanic , or Latino?

1. Yes

2. No

3. DK

4. REF

21. RACEA I am going to read you a list of five race categories. Please choose one or more races that (you/NAME) (consider yourself/ himself/considers herself) to be: White; Black or African American; American Indian or Alaska Native; Asian; OR Native Hawaiian or Other Pacific Islander. (CARD C)

1. White

2. Black or African American

3. American Indian or Alaska Native Skip to work intro.

4. Asian

5. Native Hawaiian or Other Pacific Islander

6. Other – DO NOT READ > Proceed to 28. RACEB

22. RACEB What is your race?

For Q23-44 complete questions 23-44 before moving to the next person. Complete the set of questions for everyone in the household before moving to disability intro Complete these questions only for respondents 15 and over.

Next, I am going to ask a few questions about work related activities LAST WEEK. By last week I mean the week beginning on Sunday, ____ and ending on Saturday, ___

23. BUSINESS Does anyone in this household have a business or farm?

1.Yes
2. No
3. Don't Know
4. Refused

24. WORK LAST WEEK, did [YOU/NAME] do ANY work for (either) pay (or profit)?

1.Yes skip to 32. LWWORK
2. No
3. Don't Know if 23. BUSINESS=1 proceed to 25.JOB
4. Refused if BUSINESS= 2 skip to 40. WFIND

25. JOB LAST WEEK, (in addition to the business,) did [you/name] have a job either full or part time? Include any job from which [you/name] were temporarily absent.

1.Yes skip to 27. ABSENT
2. No
3. Don't Know proceed to 26. LAYOFF
4. Refused

26. LAYOFF LAST WEEK, were you on layoff from a job?

1.Yes skip to 28. RETURN
2. No go to
3. Don't Know skip to 40. WFIND
4. Refused

27. ABSENT What was the main reason [you/name] were absent from work LAST WEEK?

1. On Layoff proceed to 28. RETURN
2. Slack Work/business conditions
3. Waiting for a new job to begin
4. Own illness/injury/medical problems
5. Does not work in the business skip to 32. LWWORK
6. Other
7. Don't Know
8. Refused

28. RETURN Has your employer given you a date to return to work?

1.Yes
2. No
3. Don't Know
4. Refused

29. R6MONTH Have you been given any indication that you will be recalled to work within the next 6 months?

1.Yes
2. No
3. Don't Know
4. Refused

30. RECALLED Could you have returned to work LAST WEEK if you had been recalled?

1.Yes
2. No
3. Don't Know
4. Refused

31. WHY1 Why is that?

1.Own Temporary Illness
2. Going to School
3. Other (Specify)
4. Don't Know
5. Refused

IF 25. JOB=1 proceed to32. LWWORK IF 25. JOB=2 skip to WFIND

32. LWWORK LAST WEEK, did [you/name] have more than one job, including part-time, evening or weekend work?

1. Yes

2. No

3. Don’t Know

4. Refused

32. HOURSM How many hours per week do [you/name] USUALLY work at [your/his/her] (main) job?
    
    By main job we mean the one at which you USUALLY work the most hours.

32. HOURSO How many hours per week do [you/name] USUALLY work at [your/his/her] other job(s)? If 32. LWWORK = 1

32. EMPTYPE Now I have a few questions about the (main) job -

(a) (at which you worked LAST WEEK. If 24. WORK=1)

(b) (from which you were absent LAST WEEK. IF 24. WORK=2)

Were [you/name] employed by government, by a private company, a non-profit organization, or [were you/was name] self-employed or working in a family business?

1. Government

2. Private company

3. Non-profit organization incl. tax exempt and charitable organizations

4. Self-employed

5. Working in family business

32. COMPANY What is the name of the (company)(non-profit organization) (business) for which [you/name] work?

32. INDUSTRY What kind of business or industry is this? READ IF NECESSARY What do they make or do where you work?

32. OCCUPATION What kind of work (do you/does NAME) do, that is, what is (your/NAME’s) occupation?

32. DUTIES What (are) your usual activities or duties at this job?

(For example: typing, keeping account books, filing, selling cars, operating printing press, laying brick)

Skip to to 44. DIS1

40. WFIND Have you been doing anything to find work during the last 4 weeks?

1.Yes proceed to 41. HOWFIND
2. No
3. Don't Know skip to STARTJOB
4. Refused

41. HOWFIND What are all of the things you have done to find work during the last 4 weeks?

Active: 1. Contacted Employer directly/interview 2. Contacted public Employment agency 3. Contated private employment agency 4. Contacted Friends or relatives 5. Contacted School/university employment center 6. Sent out resumes/filled out applications 7. Check union/professional registers 8. Place or answer ads 9. Other Active

Passive: 10. Looked at Ads 11. Attended job training programs/Courses 12. Other Passive 13. Nothing 14. Don’t Know 15. Refused

42. STARTJOB LAST WEEK, could you have started a job if one had been offered?

1.Yes skip to 44. DIS1
2. No proceed to 43. WHY2
3. Don't Know
4. Refused skip to DIS1

43. WHY2 Why is that?

1. Waiting for a new job to begin
2. Own Temporary Illness
3. Going to School
4. Other
5. Don't Know
6. Refused

For Q44-55 complete before moving to Nativity

We want to learn about people who have physical, mental, or emotional conditions that cause serious difficulty with their daily activities.

44. DIS1 [Are you/ is[NAME]/ is anyone] Deaf or [do you/does [NAME]/does anyone] have serious difficulty hearing?

1. Yes Proceed to 45.DIS1WHO

2. No

3. DK Skip to 46. DIS2

4. REF

45. DIS1WHO Who is that? Anyone else?

46. DIS2 [Are you/ is[NAME]/ is anyone] blind or [do you/does [NAME]/does anyone ]have serious difficulty seeing, even when wearing glasses?

1. Yes Proceed to 47.DIS2WHO

2. No

3. DK Skip to 48. DIS3

4. REF

47. DIS2WHO Who is that? Anyone else?

48. DIS3 Because of a physical, mental, or emotional condition, [do you/does [NAME]/ does anyone] have serious difficulty concentrating, remembering, or making decisions?

1. Yes Proceed to 49.DIS3WHO

2. No

3. DK Skip to 50. DIS4

4. REF

49. DIS3WHO Who is that? Anyone else?

50. DIS4 [Do you/does [NAME]/ does anyone] have serious difficulty walking or climbing stairs?

1. Yes Proceed to 51.DIS4WHO

2. No

3. DK Skip to 52. DIS5

4. REF

51. DIS4WHO Who is that? Anyone else?

52. DIS5 [Do you/does [NAME]/ does anyone] have difficulty dressing or bathing?

1. Yes Proceed to 53.DIS5WHO

2. No

3. DK Skip to 54. DIS6

4. REF

53. DIS5WHO Who is that? Anyone else?

54. DIS6 Because of a physical, mental, or emotional condition, [Do you/does [NAME]/ does anyone] have difficulty doing errands alone such as visiting a doctor's office or shopping?

1. Yes Proceed to 55.DIS6WHO

2. No

3. DK Skip to 56 NATIVITYA

4. REF

55. DIS6WHO Who is that? Anyone else?

For Q56-60 complete questions 56-60 before moving to the next person. Complete the set of questions for everyone in the household before moving to Income

56. NATIVITYA In what Country [were/was] [you/[NAME]] born?

57. NATIVITYB In what Country was [your/[NAME’s]] mother born?

58. NATIVITYC In what Country was [your/[NAME’s]] father born?

If person was born in the U.S., skip to 61; any other answer, proceed to 59.

59. CITIZEN [ARE/Is] [you/[NAME]] a citizen of the United States?

1.Yes
2. No
3. Don't Know
4. Refused

60. YEARUS In what year did [You/[NAME]] come to live in the United States?

61. Income Which category represents [your/name of reference person/the total combined income] [total combined income during the past 12 months/of all members of [your/this] FAMILY during the past 12 months]?

This includes money from jobs, net income from business, farm or rent, pensions, dividends, interest, social security payments and any other money income received (by members of this FAMILY who are 15 years of age or older. CARD D)

1. Less that $5,000

2. 5,000 to 7,499

3. 7,500 to 9,999

4. 10,000 to 12,499

5. 12,500 to 14,999

6. 15,000 to 19,999

7. 20,000 to 24,999

8. 25,000 to 29,999

9. 30,000 to 34,999

10. 35,000 to 39,999

11. 40,000 to 49,999

12. 50,000 to 59,999

13. 60,000 to 74,999

14. 75,000 to 99,999

15. 100,000 to 149,999

16. $150,000 or over

After the interview:

SECTION III. DEBRIEFING QUESTIONS

Now, I am going to ask you some questions about your answers and about the questions themselves. I am really interested in how these questions work for you, so there are no right or wrong answers.

• Overall, what did you think of the interview?

• Was the interview easy or difficult?

• What topics did we cover in this interview? What other topics were covered?

• Did any of the questions stand out for any reason? Which ones? Can you tell me why (question) stood out to you?

• Did any of the questions bother you? Which ones? Can you tell me why (question) bothered you?

In the survey we’re looking at, we ask one person to answer questions for everyone in the household.  Most of the questions are only for household members 15 and older, since it doesn’t make sense to ask about employment for younger children.  For the questions we’ll be focusing on today, we’re only asking about people in your house who are 15 or over.

I’d like to talk about a few particular items from the survey. The first one I’d like to talk about asked “Because of a physical, mental, or emotional condition, does anyone have serious difficulty concentrating, remembering, or making decisions?

• In your own words, what is this question asking?

• What was your overall reaction to this question?

• How did you go about answering this question for yourself?

  • What about for the other people in your household?

• Did you have any difficulty answering this question? Probe if necessary, for yourself or someone else in your household?

• This survey is often conducted in people’s homes. If other members of your household were in the room, would you be comfortable answering this question? For yourself? For others?

• Do you think that there are particular types of government surveys that should be asking this question? What types of surveys?

  • What about employment surveys, what would your reaction be if this question was asked on a government survey about employment?

Moving to another question. This one asked “Was your sex recorded as male or female at birth?”

• In your own words, what is this question asking?

• What was your overall reaction to this question?

• How did you go about answering this question for yourself?

  • What about for the other people in your household?

• Did you have any difficulty answering this question? Probe if necessary, for yourself or someone else in your household?

• If other members of your household were in the room, would you be comfortable answering this question? For yourself? For others?

• Do you think that there are particular types of government surveys that should be asking this question? What types of surveys?

  • What about employment surveys, what would your reaction be if this question was asked on a government survey about employment?

Moving to another question I’d like to get your feedback on. It asked “Do you describe yourself as male, female or transgender?”

• In your own words, what is this question asking?

• What was your overall reaction to this question?

• How did you go about answering this question for yourself?

  • What about for the other people in your household?

• Did you have any difficulty answering this question? Probe if necessary, for yourself or someone else in your household?

• If other members of your household were in the room, would you be comfortable answering this question? For yourself? For others?

• Do you think that there are particular types of government surveys that should be asking this question? What types of surveys?

  • What about employment surveys, what would your reaction be if this question was asked on a government survey about employment?

Moving to another question. This one asked Which category represents the total combined income of all members of your FAMILY during the past 12 months?

• In your own words, what is this question asking?

• What was your overall reaction to this question?

• How did you go about calculating the answer to this question?

• Did you have any difficulty answering this question?

• How certain are you about your inclusion of income information for other members of your family?

• If other members of your household were in the room, would you be comfortable answering this question?

• Do you think that there are particular types of government surveys that should be asking this question? What types of surveys?

  • What about employment surveys, what would your reaction be if this question was asked on a government survey about employment?

We have another question we’d like your feedback on. It asks “which of the following best represents how you think of yourself?”

Give them a copy of the identity question (card H)

• In your own words, what is this question asking?

• What was your overall reaction to this question?

• How did you go about answering this question for yourself?

  • What about for the other people in your household?

• Did you have any difficulty answering this question? Probe if necessary, for yourself or someone else in your household?

• If other members of your household were in the room, would you be comfortable answering this question? For yourself? For others?

• Do you think that there are particular types of government surveys that should be asking this question? What types of surveys?

  • What about employment surveys, what would your reaction be if this question was asked on a government survey about employment?

One last question I want to ask about, which asked LAST WEEK, did [you/name] have more than one job, including part-time, evening or weekend work?

• In your own words, what is this question asking?

• How did you go about answering this question for yourself?

  • What about for the other people in your household?

• Did you have any difficulty answering this question for yourself or anyone else in your household?

• If other members of your household were in the room, would you be comfortable answering this question? For yourself? For others?

• Do you think that there are particular types of government surveys that should be asking this question? What types of surveys? What about employment surveys?

As you know, we are talking to XX, who will be answering the same set of questions about him/herself as well as you and the other members of your household. 

• How do you feel about them answering these questions about you?

• How accurate do you think his/her answers about you will be?

  • How accurate do you think his/her answers about other household members will be?

  • Are there any questions that you think they’re more likely to get wrong than others?

• How accurate do you think your answers about him/her are?

  • How accurate do you think your answers about other household members will be?

  • Are there any questions that you think you’re more likely to get wrong than others?

  • Are there any household members that you’re less certain about than others for any of these topics? Who? Which topic?

• Do you have any concerns about their answering on your behalf?

  • Are there any specific questions that you have concerns about?

  • Are there any specific household members you have concerns about XX answering questions about?

• Do you have any concerns answering on their behalf?

  • How accurate do you think your answers about them are?

  • Are there any specific questions that you have concerns about?

Card Sort

Now we are going to switch to a different task. I am going to give you a stack of cards with survey questions on them. I would like you to sort them into two piles. Place the questions that you found difficult to answer about yourself in one pile and questions you did not find difficult to answer about yourself in another pile.

AFTER TASK IS COMPLETE LAY OUT CARDS SO THEY CAN BE SEEN – ONE LINE FOR “DIFFICULT” QUESTIONS AND ANOTHER FOR “NOT DIFFICULT”

AFTER TASK IS COMPLETE

• How did you decide what to put in the difficult pile?

• Please describe the questions in the ‘difficult’ pile

• What do they have in common?

• Other than difficult, what words would you use to describe this pile?

• Please describe the questions in the ‘not difficult’ pile

• Do you think other people in your household would sort them the same way? What would they do differently?

• Would you have sorted them differently if we had asked about how difficult’ they were to answer about someone else in the household instead of you? How would it have been different?

Take the cards with questions you found difficult and place them in order from top to bottom with the top card being the most difficult and the bottom card being the least difficult.

• So you think this is the most difficult question?

• What did you find most difficult about XXX [the top card]?

  • What about it makes it difficult?

• Is the question at the bottom of the pile more difficult than the questions in the not difficult pile?

Let’s look at your piles again.

• Do you think other people in your household would sort them the same way? What would they do differently?

• Thinking about answering these questions for the other people in your household instead of answering for yourself, do you want to move any of the cards between the piles? Feel free to do so

• What makes that question/those questions more/less difficult to answer about someone else instead of yourself?

TAKE BACK CARDS AND SHUFFLE THEM

I would now like you to sort the cards into two piles again. This time, place the questions that you found sensitive when answering about yourself in one pile and questions you did not find sensitive to answer in another pile.

AFTER TASK IS COMPLETE LAY OUT CARDS SO THEY CAN BE SEEN – ONE LINE FOR “SENSITIVE” QUESTIONS AND ANOTHER FOR “NOT SENSITIVE”

• How did you decide what to put in the sensitive pile?

• Please describe the questions in the ‘sensitive’

  • What do they have in common?

• Other than sensitive, what words would you use to describe this pile?

• Please describe the questions in the ‘not sensitive’ pile

• What do they have in common?

• Do you think other people in your household would sort them the same way? What would they do differently?

• Would you have sorted them differently if we had asked about how sensitive they were to answer about someone else in the household instead of you? How would it have been different?

Take the cards with questions you found difficult and place them in order from top to bottom with the top card being the most difficult and the bottom card being the least sensitive.

• So you think this is the most sensitive question?

• What about it makes it sensitive?

• Is the question at the bottom of the pile more sensitive than the questions in the not sensitive pile?

Let’s look at your piles again.

• Do you think other people in your household would sort them the same way? What would they do differently?

• Thinking about answering these questions for the other people in your household instead of answering for yourself, do you want to move any of the cards between the piles? Feel free to do so

• What makes that question/those questions more/less sensitive about someone else instead of yourself?

Question Wording

As I said in the beginning, we’re looking at question wording. For one of the questions on the survey, we have another way we could ask it. Here is a copy of the wording you were asked and a different wording option, it says “Was your sex recorded as male or female on your birth certificate?”

Give them a copy of the original and alternate questions (card F and G)

• What’s the difference between these two versions?

• What is your overall reaction to this version?

• Do you think the second version of the question is easier or harder to understand? How so?

• Which version of the question do you prefer? Can you tell me what about it you like better?

Again, we have another version of the wording. It says “Do you describe yourself as male, female, transgender or something else?”

Give them the alternate gender question (H an I)

• What’s the difference between these two versions?

• In your opinion, is this question the same as or different than the original?

• What do you think the ‘something else’ is referring to?

• Do you think the second version of the question is easier or harder to understand? How so?

• Which version of the question do you prefer? Can you tell me what about it you like better?

• If reported difficulty with first version: would this version make the question easier to answer than the other?

• If reported discomfort with first version: How would you feel answering this version of the question with other household members present?

Closing Comments

As you might have guessed, we are thinking about asking sexual orientation and gender identity in Current Population Survey, a government survey about employment.

• What do you think about that idea?

• Why do you think the government is interested in sexual orientation and gender identity in a survey about employment?

• If you were asked to provide information about your sexual orientation and gender identity in a government survey about employment, how would you react?

  • How might others in your household react?

• If you were asked to provide information about the sexual orientation and gender identity of everyone else in your household over 15, how would you react?

  • How might others in your household react?

• Is there anything that we could do differently to make it as easy and comfortable as possible for people to provide the information?

• Do you have anything else you would like to tell us about the collection of information about sexual orientation and gender identity?

Closing

That’s all the questions I have for you, thanks for your feedback! Before you go,

• Do you have anything else you would like to tell us that you haven’t had a chance to mention yet?

I want to thank you very much for your participation. I will now give you $40 and I will ask you to sign a receipt form verifying that you received the money.

TURN OFF THE RECORDER. HAND THE CASH INCENTIVE AND VOUCHER TO THE PARTICIPANT.

SECTION IV. flashcards, question cards and, card sorting questions

Answer category cards

Card A

RELATIONSHIP

(42) Opposite-sex Spouse (Husband/Wife)

(43) Opposite-sex Unmarried partner

(44) Same-sex Spouse (Husband/Wife)

(45) Same-sex Unmarried partner

(46) Child

(47) Grandchild

(48) Parent (Mother/Father)

(49) Brother/Sister

(50) Other relative (Aunt, Cousin, Nephew,Mother-in-law, etc.)

(51) Foster child

(52) Housemate/Roommate

(53) Roomer/Boarder

(54) Other nonrelative

Card B

Education

(31) Less than 1st grade

(32) 1st, 2nd, 3rd, or 4th grade

(33) 5th or 6th grade

(34) 7th or 8th grade

(35) 9th grade

(36) 10th grade

(37) 11th grade

(38) 12th grade NO DIPLOMA

(39) HIGH SCHOOL GRADUATE – high school DIPLOMA or the

equivalent (For example: GED)

(40) Some college but no degree

(41) Associate degree in college – Occupational/vocational program

(42) Associate degree in college – Academic program

(43) Bachelor’s degree (For example: BA, AB, BS)

(44) Master’s degree (For example: MA, MS, MEng, Med, MSW, MBA)

(45) Professional School Degree (For example: MD, DDS, DVM,LLB, JD)

(46) Doctorate degree (For example: PhD, EdD)

Card C

RACE

CHOOSE ONE OR MORE

1. White

2. Black or African American

3. American Indian or Alaska Native

4. Asian

5. Native Hawaiian or Other Pacific Islander

Card D

INCOME

Which category on this card represents the total combined income of all members of this FAMILY during the past 12 months? This includes money from jobs, net income from business, farm or rent, pensions, dividends, interest, social security payments and any other money income received by members of this FAMILY who are 15 years of age or older.

(1) Less than $ 5,000 (9) $30,000 to $34,999

(2) $ 5,000 to $ 7,499 (10) $35,000 to $39,999

(3) $ 7,500 to $ 9,999 (11) $40,000 to $49,999

(4) $10,000 to $12,499 (12) $50,000 to $59,999

(5) $12,500 to $14,999 (13) $60,000 to $74,999

(6) $15,000 to $19,999 (14) $75,000 to $99,999

(7) $20,000 to $24,999 (15) $100,000 to $149,999

(8) $25,000 to $29,999 (16) $150,000 or more

Question alternate text

Card E

Which of the following best represents how you think of yourself?

1. Gay or Lesbian

2. Straight, that is not gay, lesbian, or bisexual

3. Bisexual

4. Something else

Card F

Was your sex recorded as male or female at birth?

1. Male

2. Female

Was [NAME]’s sex recorded as male or female at birth?

1. Male

2. Female

Card G

Was your sex recorded as male or female on your birth certificate?

1. Male

2. Female

Was [NAME]’s sex recorded as male or female on their birth certificate?

1. Male

2. Female

Card H

Do you describe yourself as male, female, or transgender?

1. Male

2. Female

3. Transgender

Does [NAME] describe themselves as male, female, or transgender?

1. Male

2. Female

3. Transgender

Card I

Do you describe yourself as male, female, transgender or something else?

1. Male

2. Female

3. Transgender

4. Something else

Does [Name] describe themselves as male, female, transgender or something else?

1. Male

2. Female

3. Transgender

4. Something else

Card Sort Questions (will be shuffled to appear in random order- numbered here to make them easy to read)

1. What is your date of birth?

2. Was your sex recorded as male or female at birth?

3. Do you describe yourself as male, female, or transgender?

4. Are You Currently married, widowed, divorced, separated or never married?

5. Do You have a boyfriend, girlfriend, or partner living in the household?

6. Which of the following best represents how you think of yourself?

Gay or lesbian

Straight, that is not gay, lesbian or bisexual

Bisexual

Something else

7. Did you ever serve on active duty in the U.S. Armed forces?

8. What is the highest level of school you have completed or the highest degree you have received?

9. Are you Spanish, Hispanic, or Latino?

10. I am going to read you a list of five race categories. Please choose one or more races that you consider yourself to be: White; Black or African American; American Indian or Alaska Native; Asian; OR Native Hawaiian or Other Pacific Islander.

11. LAST WEEK, did YOU do ANY work for either pay or profit?

12. Is anyone Deaf or does anyone have serious difficulty hearing?

13. Because of a physical, mental, or emotional condition, does anyone have serious difficulty concentrating, remembering, or making decisions?

14. Because of a physical, mental, or emotional condition, does anyone have difficulty doing errands alone such as visiting a doctor's office or shopping?

15. In what Country were you born?

16. In what Country was your mother born?

17. In what Country was your father born?

18. What is the name of the (company)(non-profit organization) (business) for which you work?

19. Which category represents the total combined income of all members of your FAMILY during the past 12 months? This includes money from jobs, net income from business, farm or rent, pensions, dividends, interest, social security payments and any other money income received (by members of this FAMILY who are 15 years of age or older.)

Appendix C: Focus Group Protocol

CPS Gender Identity Focus group protocol

1. Welcome and topic introduction (5 minutes)

Hello, My name is _________ and I am the moderator for today’s group discussion. I want to thank each of you for coming here today. [DESCRIBE OTHERS SEATED IN THE ROOM OR OBSERVERS IF NEEDED.]

This discussion is one of several focus groups we will be conducting at different sites around the country for the U.S. Census Bureau and the Bureau of Labor Statistics. These focus groups are one of the first steps being taken to help us understand how to ask questions about gender identity on one of our national surveys.

We will spend this time participating in a group discussion. There are no right or wrong answers, because only you know what you are thinking. This focus group will last about 2 hours.

Additionally, - we will not use your names in our reports- and your participation is voluntary.

This session is being recorded so that I have accurate information while writing reports and so I don’t need to take notes while we are talking.

PERMISSIONS

The restroom is located ________. At any time you can excuse yourself to go to the restroom or to get water. However, I ask that only one person be up or out at a time so the conversation can continue.

GUIDELINES: In order for this group to work well, here are some guidelines.

• ^{Please speak one at a time.}

• ^{Speak in a voice as loud as mine so all can hear.}

• ^{Avoid side conversations with your neighbors, but you can piggy back on other people's comments in the group.}

• ^{I want to hear from everyone; however, you do not have to answer every question. Aim for "equal air time" so no one talks too much or too little. Occasionally, I may need to cut someone off to ensure that everyone is heard or to ensure we cover all topics.}

• ^{If you disagree with someone else}’s comment, that’s ok. But please be respectful. Similarly, say what you believe whether or not anyone agrees with you. We’re here to hear your experiences and thoughts, it’s important to know if you’ve had a similar or different experience than others.

• ^{I want to give all of you time to talk so I am not going to answer any questions during the discussion. I will be happy to answer questions at the end.}

Do you have any questions before we begin?

Please take a moment to make sure that your cell phone is on silent or turned off. Let's begin.

2. Warm- up (15 minutes )

Now we're going to practice sharing out thoughts on a topic so we can all get the idea of how this conversation will go. Please introduce yourselves using your first name and tell us where you live and about one activity that you enjoy doing.

**TURN ON RECORDER**

3. Identification (30 minutes)

   1. What do you think I mean when I say gender identity?

   2. How do you identify yourself- that is what term do you use to convey your gender to others?

      1. Is picking a term straightforward for you to do?

      2. Was that always the case?

      3. Would you have used a different term at any other point in your life?

   3. How would you define the term transgender?

4. Gender identity on government surveys (25 minutes)

   1. Do you think it is important or unimportant to include gender identity questions on surveys? Why or why not?

   2. What about government surveys, do you think government surveys should include gender identity questions? Why or why not?

      1. Why do you think the government might be interested in collecting this information?

      2. What do you think they would do with it?

   3. Does the type of survey matter, for example is this information more important on a survey about general population, health, work, crime, etc.? Why or why not?

      1. Which types of surveys would this be important to ask on?

      2. What about a government survey about employment, what do you think about asking a gender identity question there?

   4. Do you have any concerns about including gender identity on government surveys? What are your concerns?

   5. Would you have any concerns or hesitation answering this question on a government telephone survey? What about a face to face survey?

   6. We are doing this research because one of our surveys, one that asks about employment, is considering adding a gender identity question. The goal of adding this question to an employment survey is to allow economists and policy makers to better understand employment amongst specific populations; you’ve probably heard about differences in unemployment rate by state or by race, this would allow for similar estimates to be produced by gender identity. What do you think about that?

      1. Is there anything we can do to make it as easy and as comfortable as possible for people to provide this information?

5. Question wording (25 Minutes)

   1. I have two examples of survey questions, [HAND OUT THE ONE PART AND TWO PART SEX/GENDER IDENTITY QUESTIONS] what do you see as the differences between these questions?

      1. Do you prefer the single question or the two part question? What do you prefer about that one?

      2. Is one easier to answer?

      3. How would you answer the one-part question?

      4. How would you answer the two-part question?

   2. [HAND OUT THE TWO PART QUESTION WITH FOUR DIFFERENT ANSWER OPTIONS] Here is the two-part question again but this time there are different options for the second question. What do you see as the differences between these questions?

      1. Do you have a preference for one set of answer choices over another?

      2. Are there any other answers you would like included?

      3. Would you answer this question differently with this new set of answer choices provided?

   3. In addition to gender identity, we are also considering adding questions about sexual orientation.

      1. Do you think a question about sexual orientation is more or less sensitive than a question about gender identity? Why or why not? [PROVIDE QUESTION TEXT IF ASKED]

      2. Do you think a question about sexual orientation is more or less difficult to answer than a question about gender identity? Why or why not?

6. Gender identity by proxy (15 minutes)

   1. The survey that we are considering adding gender identity questions to is unique in that one person answers the questions for everyone in the household.

   2. Let’s just look at the two part question. [HAND OUT PROXY QUESTION TEXT FOR TWO PART QUESTION] If someone else in your home were answering this question for you, do you think they would have an easy time or hard time answering?

      1. How do you think someone in your household would answer this question for you?

      2. Do you think they’d be comfortable doing so?

      3. Would they have any concerns or hesitations answering it about you on your behalf?

      4. Do you think they would answer it?

      5. Do you think they would find this question sensitive?

      6. Do you think they’d answer the question as you would answer it for yourself?

   3. Is there a way that we should word the question or introduce the topic to address any concerns or hesitation that respondents might have when answering this question, either for themselves or others in their household?

7. Conclusion (5 minutes)

That’s all the questions we have for you today. Does anyone have any additional comments or questions we can answer?

[RESPOND TO COMMENTS AND ANSWER QUESTIONS]

Thank you very much for your participation. You can pick up your incentive from the area where you first signed in.

Focus Group Show Cards

One step question

What is your current gender?

Male

Female

Transgender

Do not identify as male, female or transgender

Two step question

Was your sex recorded as male or female at birth?

3. Male

4. Female

Do you describe yourself as male, female, or transgender?

4. Male

5. Female

6. Transgender

Two step question options

1. Was your sex recorded as male or female at birth?

1. Male

2. Female

2a. Do you describe yourself as male, female, or transgender?

1. Male

2. Female

3. Transgender

2B. Do you describe yourself as male, female, or transgender or something else ?

1. Male

2. Female

3. Transgender

4. Do not identify as male, female, or transgender

2C. How do you describe yourself? (Check one)

1. Male

2. Female

3. Transgender

4. Genderqueer/gender non-conforming

2D. How do you describe yourself? (Check one)

1. Male

2. Female

3. Trans male/ Trans man

4. Trans female/ Trans woman

5. Genderqueer/gender non-conforming

6. Different identity (please state)

Text for sexual orientation questions – optional in case asked for

Which of the following best represents how you think of yourself?

1. Gay or Lesbian

2. Straight, that is not gay, lesbian, or bisexual

3. Bisexual

4. Something else

5. DK

6. REF

To the best of your knowledge, which of the following best represents how NAME] thinks of their self?

1. Gay or Lesbian

2. Straight, that is not gay, lesbian, or bisexual

3. Bisexual

4. Something else

5. DK

6. REF

Two step question for proxy

1. To the best of your knowledge, was [NAME]’s] sex recorded as male or female at birth?

1. Male

2. Female

3. DK

4. REF

2. To the best of your knowledge, does [NAME]] describe themselves as male, female, or transgender?

1. Male

2. Female

3. Transgender

4. DK

REF

18