Note to Reviewer - Cognitive testing of supplemental collection of COVID19-related elements for NLSY97

November 2, 2020

NOTE TO THE REVIEWER OF: OMB CLEARANCE #1220-0141

“Cognitive and Psychological Research”

FROM: Struther Van Horn

Research Statistician

Office of Survey Methods Research

SUBJECT: Submission of Materials for Cognitive Testing of Supplemental Collection of COVID19-related

Elements for NLSY97

Please accept the enclosed materials for approval under the OMB clearance package 1220-0141, “Cognitive and Psychological Research.” In accordance with our agreement with OMB, we are submitting a brief description of the research and the materials to be used in the research.

The maximum number of burden hours is estimated to be 31.

If there are any questions regarding this project, please contact Struther Van Horn at (202-691-7568).

I. Introduction and Purpose

The National Longitudinal Survey of Youth 1997 (NLSY97) is a longitudinal project that follows a sample of American youth born between 1980 and 1984; 8,984 respondents were ages 12-17 when first interviewed in 1997. This ongoing cohort has been surveyed 19 times to date and is now interviewed biennially. The NLSY97 collects extensive information on respondents’ labor market behavior and educational experiences. The survey also includes data on the youths’ family and community backgrounds to help researchers assess the impact of schooling and other environmental factors on these labor market entrants. Data from the NLSY97 also aid in determining how youths’ experiences relate to establishing careers, participating in government programs, and forming families (https://www.bls.gov/nls/nlsy97.htm).

In order to provide a snapshot of how the NLSY97 cohort is/was affected by the COVID-19 pandemic, a supplemental data collection effort is being undertaken between the already scheduled biennial surveys. The general objectives of this supplemental survey include providing summary measures of the effects of the pandemic on hours worked, income, telework, childcare, and mental health; relating these data to NLS’s existing data on demographics, occupation and industry, as well as the respondent’s employment history; and allowing comparisons to similar items in NLSY79.

The objectives of the research described in this clearance are to cognitively test questions designed for a mixed mode approach (self-administered online and CAPI phone). This study will not focus on evaluating the usability of the online instrument, but will incorporate an evaluation of the mode by which the survey is administered (self-administered online, phone) in the design of the study. Cognitive interviews will be used to assess if questions in the supplement work as intended and that respondents can understand and answer them both for themselves and for other members in their household. The methodology for the proposed testing approaches is described next.

II. Methodology

To collect feedback about the questionnaire, this study uses cognitive interviews. Cognitive interviews provide an in-depth understanding of the participant’s thought processes and reactions to the questions. All interviews will be conducted remotely using Microsoft Teams or on the phone.

Participants will complete the survey questions from the NLSY97 COVID Supplement (Appendix 1). In order to evaluate and assess any potential differences of the survey due to mode, participants will either be asked to complete the survey by having an OSMR researcher screen sharing the survey instrument or by having the survey read to the participant over the phone. After the survey questions are answered, participants will be asked debriefing questions to better understand their responses and reactions to the questions, both when reporting for themselves and when reporting information for other household members, if applicable. The survey questions and full semi-scripted cognitive interview testing protocol is included in Appendix 2.

The interviews will be conducted virtually on Microsoft Teams or over the phone, with participants from anywhere in the United States. Observers from BLS may attend interviews.

The findings from all the cognitive interviews will be evaluated qualitatively, and used to arrive at conclusions about the effectiveness of the proposed wording. As this is pretesting, we expect that modifications to the question wordings or response options may be made during the course of the study based on initial results. Although the goals of the testing, and overall design, will remain the same, findings from preliminary results may be used to improve question wording and/or probes. Modifications are likely to range from slight changes to question wording to the order of the questions.

3. Participants

This research design includes up to two rounds of individual cognitive interviews (10 to 15 each round; a total of 25 participants). Individual interviews are planned for mid to late November through early January. However, scheduling will depend on recruitment and participant availability.

Participants will be recruited from the Office of Survey Methods Research participant database consisting of those who have responded to prior recruiting announcements. Recruiting emails are shown in Appendix 3. Participants will also be recruited by posting new recruiting announcements on online bulletin boards and job posting sites (e.g., Craigslist, NextDoor) (Attachment 4). Job posting sites will be targeted in cities with higher populations (e.g., San Francisco, Los Angeles, New York, Chicago, and Washington D.C.)

We will seek to obtain as diverse a sample of participants as possible for the cognitive testing with consideration given to the following characteristics which relate to instrument skip patterns and to various situations the questionnaire is meant to cover:

• Are employed

• Are not employed or consider themselves underemployed

• Living in a household with children

• Living in a household with a partner/spouse who is employed

• Living in a household with a partner/spouse who is unemployed or considers them under employed

• Born from 1970-1990 (to attempt to match the ages of the NLSY97 cohort, born from 1980-1984)

Potential participants will be screened for eligibility using a screening script (Appendix 5)

IV. Burden Hours

Total expected burden for this study is 31.25 hours, as shown in the table below:

	Number Contacted	Time spent Recruiting	Screening Burden	Number of Participants	Session Length	Session Burden	Total Burden
Individual Cognitive Interview	75	5 minutes	6.25 hours	25	60 minutes	25 hours	31.25 hours
Overall Total Burden Hours							31.25 hours

V. Payment to Participants

A $40 incentive will be used to recruit participants ($1000.00 total), to cover the costs associated with participation including internet access and child care. Participants will be informed of the voluntary nature of the study, the maximum length of the study (60 minutes), and that they have the right to stop participating at any time. All interviews will take place over Microsoft Teams or phone.

6. Data Confidentiality

Participants being interviewed via Teams will be shown the consent information and asked for their verbal agreement (Appendix 6). Interview participants interviewed on the phone will be asked for their verbal consent to participate in the study following the script shown in Appendix 6.

6. Attachments

1 – NLSY97 COVID Supplement

2 – Protocol for the cognitive interviews

3 – Email for soliciting volunteers

4 – Online bulletin board /advertisement

5 – Eligibility screening script

6 – Verbal consent outline

Attachment 1: NLSY97 COVID Supplement

NLSY97 COVID Supplement Questionnaire

Intro. The Coronavirus pandemic may have affected your life in many ways. This survey will ask you about some of your recent experiences related to the Coronavirus pandemic. To understand the potential impacts, we’ll ask some background questions about you and your household, and then some specific questions about how the Coronavirus pandemic may have impacted you.

First, we are going to ask about your household.

Q1. Including adults, children, and yourself, how many people in total currently live in your household? Please enter a number. ___________________________________________

If Q1>1 then go to Q1a, if Q1=1 then go to Q3_intro.

Q1a. How many people under 18 years-old currently live in your household?

Please enter a number. ___________________________________________

If Q1a >0 then go to Q1b, if Q1a=0 then go to Q2

Q1b. How many children under 6 years-old currently live in your household?

Please enter a number. ___________________________________________

Q2. Do you have a spouse or partner who currently lives in your household?

1. Yes

2. No

Q3_intro. Throughout most of this survey, we will ask you to think about the last week when responding. By the last week, we mean the last full week before this one.

Q3. Last week, did you do any work for either pay or profit?

	1   Yes (go to Q4)
	0   No (go to Q7)

Q4. Last week, how many hours did you work at all jobs combined?

Please include all hours you worked whether at your normal work site, at home, or in some other location.

____ hours

Q5. How many of those hours did you work at home?

____ hours

If Q4 > Q5 ask Q6Contact_1. Else go to Check12.

Q6Contact_1. Last week, how frequently did your work require you to be in close contact (i.e., within 6 feet) with coworkers, customers, or other people not living in your household?

1. Not at all

2. Rarely

3. Some of the time

4. Most of the time

5. All of the time

Go to Check12.

Q7.Last week, did you have a job (either full time or part time) from which you were temporarily absent?

1 Yes (go to Q8)

0 No (go to Q11)

Q8. What was the main reason you were temporarily absent from your job last week?

	1   I was on layoff or furlough, including because there was not enough work.
	2 My place of employment was closed.
	3   I was waiting for a new job to begin.
	4   I was on vacation or taking leave.
	5   I was sick, injured, or disabled.
	6   I had problems with child care or with my child’s schooling
	7   I had family or personal obligations, including caring for sick family members.
	8   There was a labor dispute or strike.
	9   I was attending school or training.
	10   Other reason

Q8a. Was this due to the Coronavirus pandemic?

1. Yes

2. No

Q9. Last week, were you being paid by your employer for any of the time off?

	1   Yes
	0   No

Q10Contact_2. Last week, how frequently would your work have required you to be in close contact (i.e., within 6 feet) with coworkers, customers, or other people not living in your household?

1. Not at all

2. Rarely

3. Some of the time

4. Most of the time

5. All of the time

6. I don’t know

Go to Check12

Q11. What is your main reason for not working for pay or profit last week?

	1   I was on layoff or furlough, including because there was not enough work.
	2 My place of employment was closed.
	3   I was waiting for a new job to begin.
	4   I was on vacation or taking leave.
	5   I was sick, injured, or disabled.
	6   I had problems with child care or with my child’s schooling
	7  I had family or personal obligations, including caring for sick family members.
	8   There was a labor dispute or strike.
	9    I was attending school or training.
	10  Other reason

Q11a. Was this due to the Coronavirus pandemic?

1. Yes

2. No

Check12: If Q2 = 1 (spouse or partner living in hh), ask Q12; else go to Check15.

Q12.Last week, did your spouse or partner do any work for either pay or profit?

	1   YES (go to Q13)
	0   NO (go to Check15)

Q13. Last week, how many hours did your spouse or partner work at all jobs combined?

Please include all hours worked whether at the normal work site, at home, or in some other location.

____ hours

Q14. How many of those hours did your spouse or partner work at home?

____ hours

Check15. If Q3=1 go to Q15a_intro; else go to Q15

Q15. Please think about the last 12 months. In the last 12 months, have you done any work for either pay or profit?

1 Yes (go to Q15a)

2 No (go to Check16)

Q15a_intro. Please think about the last 12 months.

Q15a. At any point in the last 12 months, did you experience any of the following changes to your work or earnings due to the Coronavirus pandemic?

Changes may have occurred because of government restrictions on people’s activities, because of your or others’ COVID-related illnesses, school or day care closings, or because of the overall changes in the economy because of the Coronavirus pandemic.

1. I stopped working for an employer (yes/no)

2. I started working for a new employer (yes/no)

3. My hours decreased. (yes/no)

4. My hours increased. (yes/no)

5. My earnings, including any overtime pay, tips, and commissions, decreased (yes/no)

6. My earnings, including any overtime pay, tips, and commissions, increased (yes/no)

Check16: If Q1a (number of children under 18) >0 then ask Q16_intro. Else go to Q18_intro.

Q16_intro. Now we are going to ask you some questions about schooling for the children living in your household.

Q16. Last week, were any children living in your household enrolled in a public school, enrolled in a private school, or educated in a homeschool program for Kindergarten through 12th grade or grade equivalent?

1 Yes (go to Q16a)

2 No (go to Q18_intro)

Q16a. In what type(s) of schooling were children living in your household enrolled or educated?

Select all that apply.

▢ 1 Public or private school (taught in person or remotely) (go to Q16b)

▢ 2 A homeschool program (go to Q18_intro)

Q16b. Last week were any classes taught in-person, at school for any children living in your household?

1. Yes

2. No

Q16c. Last week were any classes taught using a remote or distance learning format for any children living in your household?

1. Yes (go to Q17)

2. No (go to Q18_intro)

Q17. Do you agree or disagree with the following statement? Last week remote or distance learning made it difficult for me to work or do other household tasks.

1. Strongly disagree

2. Disagree

3. Agree

4. Strongly agree

Q18_intro. Now we’re going to ask you some questions about your health.

Q18. In general, how is your health?

1. Excellent

2. Very good

3. Good

4. Fair

5. Poor

Q19. Has a doctor or another healthcare professional ever told you that you had the Coronavirus or COVID-19?

1 Yes

0 No

Q20. The following is a list of the ways that you might have felt or behaved recently. After each statement, please indicate how often you felt this way during the past week.

During the past week.....

1. I did not feel like eating; my appetite was poor.

2. I had trouble keeping my mind on what I was doing.

3. I felt depressed.

4. I felt that everything I did was an effort.

5. My sleep was restless.

6. I felt sad.

7. I could not get "going".

0   Rarely/None of the time/1 Day

1   Some/A little of the time/1-2 Days

2   Occasionally/Moderate amount of the time/3-4 Days

3   Most/All of the time/5-7 Days

Thank you

Attachment 2: Cognitive Interview Protocol

Introduction

Hello, my name is ______________ and I work the Bureau of Labor Statistics. Thank you for taking the time to talk with me today.

We are testing new content for the National Longitudinal Surveys (NLS). NLS are a set of surveys designed to gather information on the labor market activities and other significant life events of several groups of men and women over the course of their lives. NLS is looking to add some new questions to learn more about the impact of COVID-19 across the U.S.

Before new survey questions are asked, it’s important to test them out with people like you, to make sure that they make sense and are easy to follow. Your answers will help us improve the questions. There are no right or wrong answers; we’re looking for your reactions and honest feedback. Your name will not be included in the report, and nothing that you say will be attributed directly to you.

Informed Consent:

Upon beginning a remote testing session, participants will read this information or shown the following information and asked for their verbal consent prior to beginning the study:

• We estimate the session will last 30 to 60 minutes.

• Your participation in this research project is voluntary, and you have the right to stop at any time.
  
  

• The Bureau of Labor Statistics is conducting this voluntary study under OMB No. 1220-0141, which expires on March 31, 2021. Without this currently approved number, we could not conduct this research. Your responses are also protected by law:

The Bureau of Labor Statistics, its employees, agents, and partner statistical agencies, will use the information you provide for statistical purposes only and will hold the information in confidence to the full extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act (44 U.S.C. 3572) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent

Do you have any questions before we proceed?

Do you agree to participate?

II. QUESTIONNAIRE COMPLETION

Next, I’m going to ask you to complete the survey. Once you are finished, we will talk about the survey questions and your answers.

If screen sharing the survey:

• I will share my screen, which has the copy of the survey. (Verify screen sharing is working/visible to participant)

• As you complete the questions, please read the questions to yourself but say your answers aloud

• If a question isn’t clear or you don’t understand some wording that is used, please let me know. We want these questions to be clear and easy to understand.

• If there are any questions you would rather not answer, let me know.

If reading the survey/administering over the phone:

• I will read the survey aloud to you, please tell me your answers.

• If a question isn’t clear or you don’t understand some wording that is used, please let me know. We want these questions to be clear and easy to understand.

• If there are any questions you would rather not answer, let me know.

Thanks for your answers so far. I would like to get some general feedback before we talk about some of the questions individually, so I can understand what you were thinking when you answered them.

• What was your reaction to the questions overall?

• Did any of those questions stand out to you for any reason?

• Were there any questions in this survey that you think some people might find sensitive?

• Were there any questions in this survey that you think some people might find difficult?

Intro. The Coronavirus pandemic may have affected your life in many ways. This survey will ask you about some of your recent experiences related to the Coronavirus pandemic. To understand the potential impacts, we’ll ask some background questions about you and your household, and then some specific questions about how the Coronavirus pandemic may have impacted you.

First we are going to ask about your household

Now I would like to discuss some specific questions (to be shown on the screen for video conference interviews).

• Starting with the introduction, now that you’ve finished the rest of the survey, does this introduction make sense?

Q1. Including adults, children and yourself, how many people in total currently live in your household? Please enter a number. ___________________________________________

Q1a. How many people under 18 years-old currently live in your household?

Please enter a number. ___________________________________________

Q1b. How many children under 6 years-old currently live in your household?

Please enter a number. ___________________________________________

Q2. Do you have a spouse or partner who currently lives in your household?

1. Yes

2. No

The next set of questions asked about the people currently in your household.

• Did you have any difficulties in answering these questions?

  • If yes, which questions? Why?

• What does “currently live in your household” mean to you?

Q3_intro. Throughout most of this survey, we will ask you to think about the last week when responding. By the last week, we mean the last full week before this one.

Q3. Last week, did you do any work for either pay or profit?

	1   YES (go to Q4)
	0   NO (go to Q7)

Looking at the next question:

• For this question, you responded ____. Can you tell me more about that?

• Can you tell me what specific days you were thinking of when you answered this question?

  • If necessary, probe on what “last week” means

• In the context of this question, what does “for either pay or profit” mean to you?

Q4. Last week, how many hours did you work at all jobs combined?

Please include all hours you worked whether at your normal work site, at home, or in some other location.

____ hours

Q5. How many of those hours did you work at home?
____ hours

The next two questions ask about the number of hours you worked:

• How did you come up with your answer to the first question?

• If necessary, probe on what types of jobs they are including

• If necessary, probe on how they view the differences between “normal work site, at home or in some other location”

• In your own words, what are the differences between these two questions?

  • If necessary, probe on whether they are counting the hours worked at home as a subset of “all hours” worked.

6. Contact_1. Last week, how frequently did your work require you to be in close contact (i.e., within 6 feet) with coworkers, customers, or other people not living in your household?

1. Not at all

2. Rarely

3. Some of the time

4. Most of the time

5. All of the time

Looking at the next question:

• For this question, you responded ____. Talk me through how you answered this question.

  • If necessary/applicable, probe on answer choice: what do they consider the difference between most of the time and some of the time?

  • If necessary, probe on who they are in close contact with (coworkers, customers, or other people)?

  • If necessary, ask them to describe what “close contact” means for their work.

Q7. Last week, did you have a job (either full time or part time) from which you were temporarily absent?

1 Yes

0 No

Looking at the next question:

• Can you tell me about your answer?

• In your own words, what does “temporarily absent” mean to you?

  • If necessary probe on how many days they were absent

  • If necessary, if they have multiple jobs, probe if they were temporarily absent from one or both/all.

Looking at the next questions:

• 

  Q8. What was the main reason you were temporarily absent from your job last week?

  	1   I was on layoff or furlough, including because there was not enough work.
  	2 My place of employment was closed.
  	3   I was waiting for a new job to begin.
  	4   I was on vacation or taking leave.
  	5   I was sick, injured, or disabled.
  	6   I had problems with child care or with my child’s schooling
  	7   I had family or personal obligations, including caring for sick family members.
  	8   There was a labor dispute or strike.
  	9   I was attending school or training.
  	10   Other reason

  Q8a. Was this due to the Coronavirus pandemic?

  1. Yes

  2. No

  
  

  
  

  Can you tell me more about why you were temporarily absent in the last week?

• I’d like to get your feedback on the provided list reasons why someone could be temporarily absent, did all of these reasons make sense to you?

• Are there any other reasons you think should be added?

• In your own words, what does “layoff” mean to you?

• In your own words, what does “furlough” mean to you?

  • If applicable, probe on “Other reason”

• If yes to 8a: can you tell me more about how the pandemic impacted you being temporarily absent?

• If no to 8a: can you tell me more about this?

Q9. Last week, were you being paid by your employer for any of the time off?

1 Yes

0 No

• Did you have any difficulties in answering this question?

  • If yes, why?

Q10. Contact_2. Last week, how frequently would your work have required you to be in close contact (i.e., within 6 feet) with coworkers, customers, or other people not living in your household?

1. Not at all

2. Rarely

3. Some of the time

4. Most of the time

5. All of the time

6. I don’t know

Looking at the next question:

• Talk me through how you answered this question.

  • If necessary/applicable, probe on answer choice: what do they consider the difference between most of the time and some of the time?

  • If necessary, probe on who they are in close contact with (coworkers, customers, or other people)?

  • If necessary: probe on what their typical work requirements be like, to get at the

Q11. What is your main reason for not working for pay or profit last week?

	1   I was on layoff or furlough, including because there was not enough work.
	2   I was waiting for a new job to begin.
	3   I was on vacation or taking leave.
	4   I was sick, injured, or disabled.
	5   I had problems with child care or with my child’s schooling
	6  I had family or personal obligations, including caring for sick family members.
	7   There was a labor dispute or strike.
	8 Weather prevented me from working.
	9    I was attending school or training.
	10  Other reason

Q11a. Was this due to the Coronavirus pandemic?

1. Yes

2. No

“would your work have required”

Looking at the next questions:

• Can you tell me more about why you were temporarily absent in the week?

• I’d like to get your feedback on the provided list reasons for not working for pay/profit, are there any other reasons you think should be added?

• In your own words, what does “layoff” mean to you?

• In your own words, what does “furlough” mean to you?

  • If applicable, probe on “Other reason”

• If yes to 11a: can you tell me more about how the pandemic impacted you being temporarily absent?

• If no to 11a: can you tell me more about this?

Q12. Last week, did your spouse or partner do any work for either pay or profit?

	1   Yes
	0   No

Q13. Last week, how many hours did your spouse or partner work at all jobs combined?

Please include all hours worked whether at the normal work site, at home, or in some other location.

____ hours

Q14. How many of those hours did your spouse or partner work at home?

____ hours

The next questions ask about the number of hours your spouse/partner worked:

• How well were you able to answer these questions without consulting your spouse/partner?

• When you reported the hours your spouse/partner worked, how exact do you think you are?

• Did you have any difficulties in answering these questions?

  • If yes, which questions? Why?

• 

  Q15. Please think about the past 12 months.

  
  

  In the past 12 months, have you done any work for either pay or profit?

  1 Yes

  2 No

  
  

  
  

  
  

  
  

  If necessary, probe on what types of jobs they are including

• When was the last time you worked for either pay or profit?

• Did you have any difficulties in answering this question?

  • If yes, why?

  • If necessary, probe on if they had any difficulties with the 12 month reference period.

Q15a. At any point in the last 12 months, did you experience any of the following changes to your work or earnings due to the Coronavirus pandemic?

Changes may have occurred because of government restrictions on people’s activities, because of your or others’ COVID-related illnesses, school or day care closings, or because of the overall changes in the economy because of the Coronavirus pandemic.

1. I stopped working for an employer (yes/no)

2. I started working for a new employer (yes/no)

3. My hours decreased. (yes/no)

4. My hours increased. (yes/no)

5. My earnings, including any overtime pay, tips, and commissions, decreased (yes/no)

6. My earnings, including any overtime pay, tips, and commissions, increased (yes/no)

Looking at the next set of question:

• Looking at the first few lines before the questions, did that make sense to you? Did it help you understand the question better?

• (for each ‘yes’ response) You reported ‘yes’ for ____. Can you tell me more about that?

• Did you have any difficulties in answering these questions?

  • If yes, why?

• Can you tell me about how the children living in your household typically attended school before COVID?

• Did you have any difficulties in answering this question?

  • If yes, why?

Q16. Now we are going to ask you some questions about schooling for the children living in your household.

Last week, were any children living in your household enrolled in a public school, enrolled in a private school, or educated in a homeschool program for Kindergarten through 12th grade or grade equivalent?

Select all that apply.

▢ 1 Yes, enrolled in a public or private school (taught in person or remotely)

▢ 2 Yes, educated in a homeschool program

▢ 3 No

• If applicable, probe on how easy or difficult this question is for respondents with multiple children/children with different schooling situations

Q16a In what type(s) of schooling were children living in your household enrolled or educated?

Select all that apply.

▢ 1 Public or private school (taught in person or remotely) (go to Q16b)

▢ 2 A homeschool program (go to Q18_intro)

Q16b. Last week, were any classes taught in-person, at school for any children living in your household?

1. Yes

2. No

Q16c. Last week, were any classes taught using a remote or distance learning format for any children living in your household?

1. Yes

2. No

Looking at these next questions about education:

• Can you tell me more about the schooling situation of your child/children?

• If necessary, can you tell me about how the children living in your household attended school over the last week?

• What does ‘remote or distance learning’ mean to you? Can you provide examples?

Q17. Do you agree or disagree with the following statement?

Last week remote or distance learning made it difficult for me to work or do other household tasks.

1. Strongly disagree

2. Disagree

3. Agree

4. Strongly agree

Looking at the next question:

• In your own words, please tell me what this question is asking

• Can you tell me what you were thinking about when you answered this question?

• If necessary, probe on examples of what about remote/distance learning made it difficult to do work/household tasks

• What does ‘household tasks’ mean to you? Can you provide examples?

Q19. Has a doctor or another healthcare professional ever told you that you had the Coronavirus or COVID-19?

1 Yes

0 No

Looking at the next question:

• Did you have any difficulties in answering this question?

  • If yes, why?

Q20. The following is a list of the ways that you might have felt or behaved recently. After each statement, please indicate how often you felt this way during the past week.

During the past week.....--

1. I did not feel like eating; my appetite was poor.

2. I had trouble keeping my mind on what I was doing.

3. I felt depressed.

4. I felt that everything I did was an effort.

5. My sleep was restless.

6. I felt sad.

7. I could not get "going".

0   Rarely/None of the time/1 Day

1   Some/A little of the time/1-2 Days

2   Occasionally/Moderate amount of the time/3-4 Days

3   Most/All of the time/5-7 Days

Looking at the next question:

• What did you think of this question?

• Were any of the responses more difficult to answer than others? Why?

IV. DEBRIEFING/GENERAL

Thank you very much for that information. I have a few final follow-up questions for you:

• Is there anything you would like to see changed or improved in the survey?

• Is there anything that we have not already discussed that you would like to share?

Thank you very much for your time. We very much appreciate all of your feedback.

Attachment 3: Email to solicit volunteers from OSMR Database

Subject: The U.S. Bureau of Labor Statistics requests your help

Dear [name],

The Bureau of Labor Statistics (BLS) is looking for volunteers between the ages of 30 to 50 to give feedback on a survey about the impact of COVID-19 on employment, childcare, and mental health. We want this survey to be as relevant and useful as possible. We were hoping that you might be available to talk to us about these new questions. We want to be sure that the questions are easy to understand and answer.

Participants will receive $40, paid via electronic Visa card.

How long will this take?

We estimate that this will take between 30 and 60 minutes. Since your participation is completely voluntary, you can end the conversation at any point.

Where will this discussion take place?

We plan to conduct this discussion remotely using video conferencing software called Microsoft Teams. You will not have to download any software to your computer or phone. Telephone interviews are also possible.

I’m interested. What are next steps?

Simply respond to this email indicating your interest, and we will be in touch see if you are eligible to participate and schedule a time.

Sincerely,

Struther Van Horn

Struther Van Horn, Ph.D.

Office of Survey Methods Research

Bureau of Labor Statistics

202-691-7528

Attachment 4- Eligibility screening script

Outgoing Calls

Hello, my name is [FILL recruiter name] calling on behalf of the Bureau of Labor Statistics. May I speak with [FILL name]?

The Bureau of Labor Statistics (BLS) is looking for volunteers between the ages of 30 to 50 to give feedback on a survey about the impact of COVID-19 on employment, childcare, and mental health. This is a new survey and we would like your feedback to be sure the questions are easy to understand and answer.

Sessions will take place over the next few weeks, on video conference or the phone, and take about an hour. Participants will receive $40, paid via electronic Visa card.

By volunteering, you’d be helping us to understand and measure the impact of COVID on the American households.

[Then proceed with screening questions]

Incoming Calls

Thanks for calling about the study. This is a pre-test of new survey about the impact of COVID-19 on employment, childcare, and mental health. We want to test the questions to be sure the questions are easy to understand and answer.

[Then proceed with screening questions]

Screening questions

I have a couple of questions to check if you are eligible to participate.

1. How old are you?

____ Years

2. Including adults, children and yourself, how many people in total currently live in your household?

___________________________________________

If Q2 >2 then ask Q4.

3. How many children currently live in your household?

Please enter a number. ___________________________________________

3a. How many of those are under 6?

Please enter a number. ___________________________________________

4. Do you have a spouse or partner who currently lives in your household?

   • Yes

   • No

If Q5= Yes, ask Q6

5. Does your spouse or partner consider themselves unemployed?

   • Yes

   • No

6. Do you consider yourself:

• Employed

• Unemployed and looking

• Unemployed and not looking

7. Do not ask aloud:

• Male

Female

(If not between the ages of 50 and 30, the participant is NOT eligible – thank them for their interest but do not schedule them at this time, we may follow back up if we need additional participants)

Check their responses against the recruiting list. If they meet the criteria for a group that we still need representation from, schedule them. If not, thank them for the information and let them know that we’ll call them back within a day or two after we confirm availability. Aim to schedule participants representing a mix of these characteristics – for example, some married, and some not married.

Attachment 5: Online Bulletin Board / Advertisement

Volunteers needed to give feedback on a survey [$40 for 1 hour]

The Bureau of Labor Statistics (BLS) is looking for volunteers between the ages of 30 to 50 to give feedback on a survey about the impact of COVID-19 on employment, childcare, and mental health.

Participation is virtual using video conferencing or telephone. Sessions take up 30 to 60 minutes. You’ll answer questions about the effects of the pandemic on employment, telework, childcare, and other aspects of your life.

Participants will receive $40, paid via electronic Visa card. Additional eligibility screening may apply.

Please contact Stella at 202-691-7524 or [email protected].

Attachment 6 - Verbal Consent Outline for Remote Interviews

Upon beginning a remote testing session, participants will read this information or shown the following information through screen sharing on Microsoft Teams and asked for their verbal consent prior to beginning the study. Participants being interviewed over the phone will be read the information.

• We estimate the session will last 30 to 60 minutes.

• Your participation in this research project is voluntary, and you have the right to stop at any time.
  
  

• The Bureau of Labor Statistics is conducting this voluntary study under OMB No. 1220-0141, which expires on March 31, 2021. Without this currently approved number, we could not conduct this research. Your responses are also protected by law:

The Bureau of Labor Statistics, its employees, agents, and partner statistical agencies, will use the information you provide for statistical purposes only and will hold the information in confidence to the full extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act (44 U.S.C. 3572) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent

Do you have any questions before we proceed?

Do you agree to participate?

32