Attachment U: Third Follow-up Survey Question-by-question Justification

ATTACHMENT U

THIRD follow-up SURVEY

Question BY Question JUSTIFICATION

This page intentionally left blank for double sided copying

This document provides the source and justifications for each question on the Third Follow-up Survey (Attachment Q).

Question #	Question text	Included on First & Second Follow-Up Survey?	Source	Justification
i. INTRODUCTION
IN1-IN16	Introduction questions – connecting to the respondent and reminding them about the follow-up survey	Yes	PACT (OMB No. 0970-0403)	These items are used to verify that the interviewer is speaking with the correct respondent.
CV1-CV8	Verifying the respondent is the same from Baseline	Yes	Adapted from PACT (OMB No. 0970-0403)
SECTION A. ECONOMIC STABILITY
A1a-i	In the past month, have you or anyone in your household received any support… From Temporary Assistance for Needy Families, or TANF (this is also known as welfare)? From Supplemental Security Income, or SSI? From Social Security Disability Insurance, or SSDI? From Food stamps or SNAP? From Women, Infants, and Children, or WIC? From Unemployment Insurance? From Housing choice vouchers, Section 8, project-based rental assistance, public housing, housing where an agency helps you pay the rent, or other housing assistance? For heating and cooling your home? For free or reduced price meals?	Yes: A1a-f were included on both the first and second follow-up surveys, and A1h-i were included on the second follow-up survey.	A1a-f ACS (OMB No. 0607-0810) A1h-i FSS (OMB No. 2528-0296)	These items measure follow-up public assistance benefit receipt. We will use them to assess the impact of the program on self-sufficiency.
A1j	How much money have you or anyone in your household received from TANF in the past month? If both you and someone else in your household received money from TANF, please add those amounts together.	Yes	RWtW 30 month (OMB No. 0970-0246)
A2	Are you currently working for pay? Working for pay can include regular paid jobs, odd jobs, temporary jobs, work done in your own business, “under the table” work, or any other types of work you have done for pay.	Yes	CSPED (OMB No. 0970-0439)	These items measure follow-up employment by developing a list of all the jobs the respondent has worked in the past year.
A2a	Have you worked for pay at any time in the past 12 months?	Yes
A3	First I am going to ask about your current job or jobs. Please tell me who you work for. This could be the name of a job, organization, person, or you could work for yourself or have your own business. If you currently work at more than one job, please start with the job where you usually work the most hours.	Yes
A4	Including all types of jobs, do you currently have any other paid jobs?	Yes
A4b	Please tell me who you worked for in the past 12 months. This could be the name of a job, organization, person, or you could work for yourself or have your own business. If you had more than one job, please start with the job where you worked the most hours.	Yes
A4c	Have you had any other paid jobs in the past 12 months that you haven’t told me about?	Yes
A5	Did you receive assistance from an organization or an agency [to help get this job/become self-employed]? Please do not include temp agencies.	Yes	Adapted from RWtW 30 month (OMB No. 0970-0246)	These items measure service receipt.
A6	When did you start working for [JOB NAME/yourself]?	Yes	CSPED (OMB No. 0970-0439)	These items measure follow-up employment by developing a list of all the jobs the respondent has worked in the past year.
A7	Are you still working for [[JOB NAME] /yourself]?	Yes
A7a	When did you stop working at this job?	Yes
A8	Which of the following best describes your employment at that job? [Were/Are] you working… as a regular full-time employee? as a regular part-time employee? for a temporary help agency? for a company that contracts out you or your services? as an independent contractor, independent consultant, or freelance worker? as a day laborer? as an on-call employee? for friends, family, or other people doing part-time tasks or odd jobs? or something else?	Yes	WIA (OMB No. 1205-0504)	These items measure job quality and characteristics. We will use them to: (1) assess the impact of the program on job quality and (2) to inform analysis comparing survey-based and NDNH-based earnings measures.
A8a	[Do/Did] you have taxes taken out of your paycheck for the work you [do/did] at this job?	Yes	CSPED (OMB No. 0970-0439)
A9	How many hours [do/did] you usually work in a week at this job? Your best estimate is fine.	Yes
A10	Now thinking about [being self-employed/your job at [JOB NAME]], how much do/did you get paid, before taxes and deductions, at this job? Please include tips, commissions, and regular overtime. If your pay varies/varied, please provide an average amount. If you are/were paid per job or for completing a particular task, please tell us the total amount you usually make/made per week or per month while doing this type of work.	Yes
A11	Did you always earn [A10 WAGE] per [HOUR/UNIT FROM A02/your current wage] at this job?	Yes
A11a	How much were you paid when you started working at this job before taxes and deductions? If your pay [varies/varied], please provide an average amount.	Yes	RWtW 30 month (OMB No. 0970-0246)	This item measures job progression. We will use it to assess the impact of the program on labor market outcomes.
A12	Which of the following benefits [are/were] available to you at your job… Paid leave for holidays, vacation, or illness? Health insurance or membership in an HMO or PPO plan? Retirement benefits? None of the above	Yes	CSPED (OMB No. 0970-0439)	These items measure job quality and characteristics. We will use them to: (1) assess the impact of the program on job quality and (2) to inform analysis comparing survey-based and NDNH-based earnings measures.
A13	[Have/Had] you been promoted to a higher position with greater responsibility while working at this job?	Yes	Adapted from RWtW 30 month (OMB No. 0970-0246)	These items are measures of career progression used to assess the impact of the program on labor market outcomes.
A14	How likely do you think it is that you will be promoted by [JOB NAME] in the next 12 months?	Yes	RWtW 30 month (OMB No. 0970-0246)
A15	In the past 12 months, was there anything else you did for pay, such as odd jobs, “under the table” jobs, or any other type of work, that we haven’t already talked about?	Yes	PACT (OMB No. 0970-0403)	These items are measures of informal employment used to assess the impact of the program on labor market outcomes.
A16	What is your best guess of how much money you received from these activities in a typical month in the past 12 months? Please do not include money you made from jobs we talked about earlier. We just need your best guess for how much money you’ve received from these activities.	Yes

Question #	Question text	Included on First & Second Follow-Up Survey?	Source	Justification
A17	How satisfied are you with your current [job/jobs]? Would you say very satisfied, somewhat satisfied, or not satisfied?	Yes	PACT (OMB No. 0970-0403)	This item is a measure of job satisfaction.
A18	Are you currently looking for a job?	Yes	Adapted from WIA (OMB No. 1205-0504)	These items measure whether the respondent is in the labor force and actively looking for a job.
A18a	How would you describe your current status? Are you… Temporarily laid off? Retired? In school or training? Unable to work because of caring for another family member? Unable to work because of pregnancy? Sick or disabled? Gave up looking for work? Something else?	Yes
A18b	Since the start of the pandemic in March 2020, have any of the following changes happened to you at work because of COVID-19? Were you temporarily laid off or furloughed? Did you lose your job? Did you quit your job because of the risk of exposure to COVID-19? Did you get a vaccine for COVID-19 because your job required it? Did you quit your job because your job required a vaccine for COVID-19?	No	Adapted from Next Generation of Enhanced Employment Strategies Project First Follow-up Survey (OMB No. 0970-0545)	These items measure whether the respondent has faced changes in employment due to the COVID-19 pandemic.
A19	In the past 12 months, did you do any of the following because there wasn’t enough money? Cut the size of your meals or skip meals because you couldn’t afford enough food? Move in with other people, even for a little while, because of financial problems? Ask to borrow money from friends or family? Go without a phone because you could not afford to pay the bill or buy extra cell phone minutes? Took a payday loan or auto-title loan, or sold or pawned your belongings? Thought about going to the doctor, dentist or hospital, but decided not to because of the cost? Go without prescription medicine? Go without paying utility bills? Did not pay the full amount of the rent or mortgage because you could not afford it? Were evicted from your home or apartment for not paying the rent or mortgage? Filed in court for bankruptcy?	A25a-f were included on both the first and second follow-up surveys. A25g-h were included on the second follow-up survey. A25i-k are newly addeditems.	A25a-f SHARP A25g-h FSS (OMB No. 2528-0296)	These items measure follow-up economic well-being that could be influenced directly or indirectly by coaching programs. We will use them to assess the program’s impact.
A20	Now, I’m going to read you several statements that people have made about their food situation. For these statements, please tell me whether the statement was often true, sometimes true, or never true for you in the last 30 days. I worried whether my food would run out before I got money to buy more The food that I bought just didn’t last, and I didn’t have money to get more I couldn’t afford to eat balanced meals.	No	SNAP ET (OMB No. 0584-0604)
A21	Do you currently own your home or apartment or have a mortgage, rent it, pay some amount toward rent, live rent free with a friend or relative, or do you have some other arrangement?	Yes	WFNJ
A21a	What is your other living arrangement?	Yes
A22	Do you currently use a spending plan or budget to help you meet or keep track of your monthly expenses?	Yes: on the second follow-up survey.	FC (OMB No. 3170-0030)
A23	Do you currently have a checking or savings account of any kind at a bank or a credit union?	Yes: on the second follow-up survey.	FSS (OMB No. 2528-0296)
A24	Suppose [you/your household] had an emergency expense that costs $400. Based on your current situation, how would you pay for this expense? Put it on your credit card Use money in your checking or savings account or use cash on hand Borrow from a friend or family member Use a payday loan, deposit advance, or overdraft Sell something Use a payment plan where you pay “little by little”	Yes: on the second follow-up survey.	FSS (OMB No. 2528-0296)
A25	How much do you have in savings? Savings includes but is not limited to savings accounts, and covers any money you set aside for the future. Your best estimate is fine.	Yes: on the second follow-up survey.	FSS (OMB No. 2528-0296)
A26	How often does your household pay its bills on time?	Yes: on the second follow-up survey.	FC (OMB No. 3170-0030)
A27	In the past 12 months, have you seen your credit report from a credit-reporting agency?	Yes: on the second follow-up survey.	FC (OMB No. 3170-0030)
SECTION B. EMPLOYMENT CHALLENGES AND SKILLS
B1a-h	Now I am going to read you a list of things that some people find challenging in finding and keeping a good job. Please tell me if the following has made it not at all hard, a little hard, somewhat hard, very hard, or extremely hard for you to find or keep a good job in the past 12 months. Not having reliable transportation Not having good enough child care or family help Not having the right clothes or tools for work Not having the right skills or education Having a criminal record A lack of good jobs available in your area Not being able to do certain kinds or amounts of work, training, or school work because of your health The risk that you or someone in your household or family will get sick from COVID-19	B1a-g were included on both the first and second follow-up surveys. B1h is a newly added item.	Adapted from ACS (OMB No. 0607-0810)	These items measure follow-up challenges to employment that are targeted by coaching programs. We will use them to: (1) assess the impact of the program on reducing employment challenges and (2) support the analysis of the mediating factors driving program impacts.
B2a	Do you have a regular child care arrangement you can use so that you can go to a job, a job interview, school, or other appointment?	Yes	Evaluation of Employment Coaching (OMB No. 0970-0506)
B2b	Now I would like to talk about backup child care arrangements. If your child care falls through, would you say that you always, very often, sometimes, rarely, or never have a backup child care arrangement?	Yes
B2c	How many backup child care arrangements do you have?	Yes
B4	Do you currently have a valid driver’s license? By valid we mean that it has not expired, and has not been suspended or taken away.	Yes	Adapted from ACS (OMB No. 0607-0810)
B4a	In the past 12 months, have you ever not been able to apply to a job because you didn’t have a valid driver’s license?	Yes	Evaluation of Employment Coaching (OMB No. 0970-0506)
B5a-i	Please tell me if you strongly disagree, disagree, agree, or strongly agree with the following statements. I know I need to get a job or a better job and really think I should work on finding one. I guess being out of work is not good, but there is nothing I can do about it right now. I set employment goals based on what is important to me or my family. I set long-term employment goals that I hope to achieve (such as finding a job, finding a better job, getting promoted, or enrolling in further education). I set specific short-term goals that will allow me to achieve my long-term employment goals. Based on everything I know about myself, I believe I can achieve my employment goals. When I set employment goals, I think about barriers that might get in my way and make specific plans for overcoming those barriers. Even when I face challenges, I continue to pursue my employment goals. I keep track of my overall progress toward my long-term employment goals and adjust my plans if needed.	Yes	a-b. LASER Questionnaire c-g. Goal Setting Questionnaire h. New Developed by Mathematica i. GSAB	These items measure goal setting and goal pursuit in the context of employment, two key domains that are targeted by coaching programs. We will use them to: (1) assess the impact of the program on goal setting and goal pursuit (2) support the analysis of the mediating factors driving program impacts.
B6.1a-l	During the past month, how often has each of the following behaviors been a problem? Would you say that the following behaviors have been a problem never, sometimes, or often? Item 25 Item 49 Item 52 Item 6 Item 71 Item 75 Item 24 Item 53 Item 9 Item 15 - edited Item 63 - edited Item 2	Yes	BRIEF-A Questionnaire (These Items Are Copyright Protected so they cannot be included in the OMB submission. Further reproduction is prohibited without permission by Psychological Assessment Resources, Inc.)	These items measure five sub-domains of executive functioning that coaching programs could influence: (1) emotional control, (2) initiation, (3) planning and organizing, (4) task monitoring, and (5) self-monitoring. We will use them to: (1) assess the impact of the program on self-regulation skills and (2) support the analysis of the mediating factors driving program impacts.
B6.2a-h	During the past month, how often has each of the following behaviors been a problem? Would you say that the following behaviors have been a problem never, sometimes, or often? a. Item 28 b. Item 69 c. Item 1 d. Item 72 e. Item 23 f. Item 50 g. Item 64 h. Item 70- edited
B7	Since [RA MONTH YEAR], how often have you… Lost your temper with someone other than friends or family? Said something that you later regretted to someone other than friends or family? Decided not to apply for a job because you didn’t think you would get an interview? Overcame a barrier that could have prevented you from finding or keeping a job? Been late for a job, interview, program meeting, class, or training session? Missed an appointment related to work, looking for a job, a program, school, or training, for a reason other than you were sick or ill?	Yes	Evaluation of Employment Coaching (OMB No. 0970-0506)	These items measure manifest behaviors that reflect self-regulation skills and could be influenced by coaching programs. We will use them to: (1) assess the impact of the program on self-regulation skills and (2) support the analysis of the mediating factors driving program impacts.
B8	Next, I’m going to read a list of opinions people have about themselves. After I read each one, I want you to tell me whether you strongly disagree, disagree, agree, or strongly agree. I am able to do things as well as most people. I certainly feel useless at times. All in all, I tend to feel that I am a failure.	Yes	Rosenberg Self-Esteem Scale	These items measure self-esteem. Coaching programs could affect self-esteem, which in turn could affect employment and self-sufficiency outcomes. We will use these items to: (1) assess the impact of the program on self-esteem and (2) support the analysis of the mediating factors driving program impacts.
B9	The next questions are about how you may have felt or behaved in the past 2 weeks. After I read each statement, please tell me how often have you been bothered by any of the following. Would you say not at all, several days, more than half the days, or nearly every day over the last 2 weeks? Little interest or pleasure in doing things? Feeling down, depressed, or hopeless?. Trouble falling or staying asleep, or sleeping too much?. Feeling tired or having little energy? Poor appetite or overeating? Feeling bad about yourself—or that you are a failure or have let yourself or your family down? Trouble concentrating on things, such as reading the newspaper or watching television? Moving or speaking so slowly that other people could have noticed? Or the opposite—being so fidgety or restless that you have been moving around a lot more than usual?	No	Personal Health Questionnaire Depression Scale (PHQ-8)	These items measure depression. Coaching programs could affect depression, which in turn could affect employment and self-sufficiency outcomes. We will use these items to: (1) assess the impact of the program on depression and (2) support the analysis of the mediating factors driving program impacts.
SECTION C. SERVICE RECEIPT
C1a	In the past 12 months, have you attended any workshops or classes that provided instruction on how to find a job? These may have provided help with things such as resume writing, interviewing, and networking.	Yes	Adapted from SNAP ET (OMB No. 0584-0604)	This item measures receipt of employment services. We will use this measure to: (1) describe the employment services that participants received, (2) characterize the employment services that participants would have received in the absence of participation in the program, and (3) support the analysis of the mediating factors driving program impacts.
C2	We are interested in learning if you met one-on-one with someone to receive assistance in finding a job or advancing your career. By one-on-one assistance, we mean meeting with someone whose job it is to help you find a job. This assistance may have been provided by a coach, case manager, counselor, or some other type of worker. It could have been in-person or by phone. In the past 12 months have you met one-on-one with someone to receive job assistance?	Yes	Adapted from SNAP ET (OMB No. 0584-0604)	This item measures receipt of one-on-one services in the vein of coaching. We will use this measure to: (1) describe the one-on-one services that participants received, (2) characterize the one-on-one services that participants would have received in the absence of participation in the program, and (3) support the analysis of the mediating factors driving program impacts.
C5	In the past 12 months, did any coach, case manager, counselor, or other type of worker provide you with names of employers who were interviewing, or set up interviews with employers for you?	Yes	PACT (OMB No. 0970-0403)	These items measure receipt of employment services. We will use these measures to: (1) describe the employment services that participants received, (2) characterize the employment services that participants would have received in the absence of participation in the program, and (3) support the analysis of the mediating factors driving program impacts.
C6	In the past 12 months, did you complete any career assessments to find out what types of jobs would be good for you? A career assessment may be used to help identify a career that is right for you based on your interests and skills.	Yes	Adapted from SNAP ET (OMB No. 0584-0604)
C7	Since [RA MONTH YEAR/FUP COMP MONTH YEAR], did you participate in any education programs? This includes adult basic education or GED courses, English as a Second Language classes, online courses, and college or other types of school. Do not include training programs to develop skills for a particular job or occupation or any other work experience in which all or part of your wages were paid for by a program.	Yes	PACT (OMB No. 0970-0403)	These items measure receipt of employment education and training services. We will use these measures to: (1) describe the employment services that participants received, (2) characterize the employment services that participants would have received in the absence of participation in the program, and (3) support the analysis of the mediating factors driving program impacts.
C7a	Are you participating in the program now?	Yes
C7b	Did you complete the program?	Yes
C7c	Did you receive a diploma or degree from the program?		Evaluation of Employment Coaching (OMB No. 0970-0506)
C8	Since [RA MONTH YEAR/FUP COMP MONTH YEAR], did you participate in any training programs to develop skills for a particular job or occupation? These are sometimes called vocational training programs. Do not include training programs provided by your employer.	Yes	Adapted from PACT (OMB No. 0970-0403)
C8a	Are you participating in the program now?	Yes	PACT (OMB No. 0970-0403)
C8b	Did you complete the program?	Yes
C8c	Did you receive a certificate, license, or diploma from the program?	Yes	Evaluation of Employment Coaching (OMB No. 0970-0506)
SECTION D. DEMOGRAPHICS
D3a	In the past 12 months, were you ever convicted of a crime?	Yes	Evaluation of Employment Coaching (OMB No. 0970-0506)	These items measure conviction history in the past 12 months. We will use follow-up conviction history to: (1) assess the impact of the program on interactions with the criminal justice system and (2) support the analysis of the mediating factors driving program impacts.
D3b	In the past 12 months, were you convicted of a felony?	Yes
D4	What is the highest level of education you have completed?	Yes	Adapted from COBRA (OMB No. 1291-0001)	These items provide demographic characteristics at follow-up. We will use them to (1) describe demographic characteristics at follow-up, (2) assess changes in demographic characteristics from baseline, and (3) support the analysis of the mediating factors driving program impacts.
D5	What is your current marital status – are you now married, separated, divorced, widowed, or have you never been married?	Yes	Adapted from CPS
D6	How many adults age 18 or older currently live in your household at least half the time? Please include yourself.	Yes	JSA (OMB No. 0970-0400)
D7	How many children under age 18 live with you at least half the time? This includes biological, adopted, foster, step, and any other children.	Yes
D8	Were you born in the United States or some other country?	Yes	Evaluation of Employment Coaching (OMB No. 0970-0506)	This item measures a baseline demographic characteristic. We will use it to (1) describe the characteristics of study participants and check that random assignment has created treatment and control groups with similar characteristics, (2) define subgroups, (3) provide control variables for regression models that will increase statistical precision, (4) construct weights to adjust for survey nonresponse, and (5) support analysis of the mediating factors driving program impacts.
D9	Have you received at least one dose of a COVID-19 vaccine?	No	CDC NIS Adult COVID Module (NIS-ACM)	These items measure COVID-19 vaccination status. This information could be useful in interpreting employment patterns given that vaccinated people might be eligible for more jobs than those who are not vaccinated and are less subject to risk of infection or serious disease.
D10	Did you get the full course of doses for the COVID-19 vaccine you received? As a reminder, a full course of the Pfizer and Moderna vaccines required two doses while the Johnson & Johnson vaccine required a single dose.	No	New item developed by Mathematica
SECTION E. CONTACT INFORMATION 2
E1-E7	Verifying respondent’s contact information	Yes	PACT (OMB No. 0970-0403), YouthBuild (OMB No. 1205-0503)	Contact information for the respondent and for additional contacts who might be able to reach the respondent is necessary to locate the respondent for any future follow-up survey.
E8a-E9i	Collecting information for up to two additional contacts	Yes	PACT (OMB No. 0970-0403)
END1-END2	Thanking participant	Yes	PACT (OMB No. 0970-0403)

Sources: acs (american community survey), BRIEF-A Questionnaire (Behavior rating inventory of executive function – adult version), BSF (BUILDING STRONG fAMILIES), FC (evaluation of financial coaching), cps (current population survey), csped (child support noncustodial parent employment demonstration), FSS (Family self-sufficiency program), GSAB (Goal Systems assessment battery), goal setting questionnaire, jsa (job search assistance), laser qUESTIONNAIRE (Lam Assessment on Stages of Employment Readiness), PACT (parents and children together), Rosenberg Self-Esteem Scale, SHARP (student health and risk prevention), snap et (Supplemental nutrition assistance program employment and training), wia (workforce investment act), COBRA (Impact of the ARRa subsidy on cobra take-up) and wfnj (work first new jersey).

B6c-g (Goal Setting Questionnaire) Noonan, P.M., & Gaumer Erickson, A.S. (2017). The skills that matter: Teaching intrapersonal and interpersonal competencies in any classroom. Thousand Oaks, CA: Corwin.

B5 (BRIEF-A) Reproduced by special permission of the Publisher, Psychological Assessment Resources, Inc. (PAR), 16204 North Florida Avenue, Lutz, Florida 33549, from the Behavior Rating Inventory of Executive Function-Adult Version by Robert M. Roth, PhD, Peter K. Isquith, PhD and Gerard A. Gioia, PhD, Copyright 1996, 1998, 2001, 2003, 2004, 2005 by PAR. Further reproduction is prohibited without permission from PAR. For Review Only, Do NOT Copy