Instrument 3: Staff survey

Status: Original and derived artifacts are available for this document.

Download: docx | pdf | html

OMB No.: XXXX-XXXX
Expiration Date: DD/MM/YYYY

TANF Pilot Evaluation 
Staff Survey

PROGRAMMER NOTES: 
UNIVERSAL SOFT CHECK IF MISSING RESPONSE: Please provide a response to this question, or click next to move to the next question. 


COMPUTER ASSISTED WEB INTERVIEW (CAWI) ALL

How to complete the survey
Thank you for taking the time to complete this survey.
    • There are no right or wrong answers.
    • To answer a question, click the box that indicates your response or fill in your response.
    • To continue to the next webpage, press the "Next" button.
    • To go back to the previous webpage, click the "Back" button. Please note that this command is only available in some sections.
    • Do not use the navigation arrows in your browser.
    • If you need to stop before you have finished, you can close the survey and come back to it at any time. The responses you gave before leaving will be securely stored and available when you return to complete the survey. 

    • Select “Next” to begin the survey. 


INTRODUCTION & CONSENT
Your state is among five states that were selected by the U.S. Department of Health and Human Services (HHS) to participate in pilots to try something different in TANF programs, as authorized by the Fiscal Responsibility Act of 2023 (FRA). Mathematica, The Adjacent Possible, and the American Public Human Services Association are conducting the TANF Pilot Evaluation to understand and learn from these pilots under contract with the Administration for Children and Families at HHS.  
What is this survey about?
We want to learn about your experiences with [TANF PROGRAM/PILOT NAME]. Your answers will help us understand how the pilot is working from the people who bring it to life every day. This survey is for staff who work directly with TANF participants, such as case managers, coaches, or employment counselors. You may work for [TANF PROGRAM/PILOT NAME]; another government agency, such as [STATE WIOA PROVIDER OR OTHER AGENCY]; or for a contracted partner organization, such as [PARTNER ORGANIZATION NAMES]. Your insights will help us understand what’s working well and where improvements might be made. Your perspective will provide information to help strengthen TANF programs in your state and across the country.
When should I complete the survey?
Please complete the survey within one week.
How long will the survey take?
This survey takes about 20 minutes, though the time to complete it will vary by person. Your participation in the survey is completely voluntary. You may choose not to respond at all or to skip any questions. Your individual responses will be kept private and will only be used for research purposes and not shared with anyone outside the study team. We have obtained a Certificate of Confidentiality from the National Institutes of Health. This helps us protect participants’ privacy. This means no one can force the study team to give out information that identifies them, even in court. The certificate does not prevent us from making disclosures required by law, such as threats of harm or abuse. We will combine responses for reporting purposes, and we will never report names or identify any responses with a particular person.    
If you agree to complete this survey, you can decline to participate in any potential future activities related to the study by calling Mathematica toll-free at 833-678-3825 and writing a message that you do not wish to complete those activities. You can send this message to Jennifer Herard-Tsiagbey, the study’s Data Collection Director, at [email protected]. Any information we collect about you before you withdraw from the study will be retained as part of the research.  
If you have questions, please contact Jennifer Herard-Tsiagbey, the study’s Survey Director, at [email protected]. Thank you for your time and input!
PAPERWORK REDUCTION ACT OF 1995 (Public Law 104-13) STATEMENT OF PUBLIC BURDEN: Through this information collection, ACF is gathering information to learn more about measuring program performance in TANF programs. Public reporting burden for this collection of information is estimated to average 20 minutes per respondent, including the time for reviewing instructions, gathering and maintaining the data needed, and reviewing the collection of information. This is a voluntary collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information subject to the requirements of the Paperwork Reduction Act of 1995, unless it displays a currently valid OMB control number. The OMB # is 0970-0XXX and the expiration date is XX/XX/XXXX. If you have any comments on this collection of information, please contact Quinn Moore at [email protected].

ALL
Do you agree to participate in this survey? 
m	YES	1 A1
m	NO	0 END1

PROGRAMMER NOTE

WHEN ADMINISTERED WITH TIME USE SURVEY MODULE, SKIP SECTION A AND GO TO SECTION B. 

A. PROFESSIONAL EXPERIENCE 
ALL
A1. 	What is your job title? 
JOB TITLE

(STRING 100)
MISSING	m

ALL
A2. 	Please select the county, jurisdiction, or area of [TANF PROGRAM/PILOT NAME] that you work for from the list below.
	[PREFILLED DROP-DOWN LIST OF JURISDICTIONS SPECIFIC TO EACH STATE]

ALL
A3. 	Which type of organization do you work for?
m	State or local human services agency	1	
m	Other government agency	2	
m	Contracted service provider or partner organization	3
m	Other (specify – string 150)	4	
MISSING	m	

ALL
A4. 	Which of the following best describes your current role? (If you have more than one role, select your primary role or the role you spend most of your time on. If your primary role is eligibility, please select a role from the list below that you do most often, aside from eligibility.)
m	Case manager, coach, or employment counselor. A person who works directly with TANF participants to help them fulfill requirements of the program, set goals, provide supports, and progress to employment.	1	
m	Job developer. A person who identifies employers who have open jobs for TANF participants and might help with matching participants to jobs.	2	
m	Workshop provider. A person who provides instruction to TANF participants in a group setting.	3	
m	Other (specify – string 150) 	4
MISSING	m	
ALL
A5. 	How many years of experience do you have working on [TANF PROGRAM/] in your state?
m	Less than 1 year	1	
m	1 to 2 years	2 	
m	3 to 5 years	3 	
m	6 to 10 years	4 	
m	11 to 15 years	5 	
m	More than 15 years	6 	
MISSING	m

ALL
A6. 	Including your current position, how many years of total experience do you have doing similar work? (Include time at other employers where you did similar work, even if it was in other states.)
m	Less than 1 year	1	
m	1 to 2 years	2 	
m	3 to 5 years	3 	
m	6 to 10 years	4 	
m	11 to 15 years	5 	
m	More than 15 years	6 	
MISSING	m


B. PROGRAM IMPLEMENTATION 
Leadership 
ALL
B1. 	The next questions ask about overall leadership of [TANF PROGRAM/PILOT NAME] . Leadership refers to the range of individuals who oversee and manage the program, set policy, and supervise staff across state, local, and contracted service provider organizations. 
	Please select whether each of the following statements are not at all true, somewhat true, mostly true, or consistently true. 
Select one per row

NOT AT ALL TRUE
SOMEWHAT TRUE
MOSTLY TRUE
CONSISTENTLY TRUE
a.	Leaders communicate a clear vision for what [TANF PROGRAM/PILOT NAME] is supposed to do for participants.
1 m
2 m
3 m
4 m
b.	Leaders focus on the issues that really matter for direct service delivery to [TANF PROGRAM/PILOT NAME] participants.
1 m
2 m
3 m
4 m
c.	Leaders explain the reasons for making changes in [TANF PROGRAM/PILOT NAME] to staff and create supports for those changes to happen.
1 m
2 m
3 m
4 m
d.	Leaders of [TANF PROGRAM/PILOT NAME] encourage staff to share ideas, ask questions, and admit mistakes.
1 m
2 m
3 m
4 m
e.	Leaders clearly explain expectations for [TANF PROGRAM/PILOT NAME] service delivery and hold staff accountable for meeting them.
1 m
2 m
3 m
4 m
f.	Leaders of [TANF PROGRAM/PILOT NAME] actively engage with staff to resolve issues that get in the way of making changes happen.





PROGRAMMER NOTE

IF ANY B1a-f IS LEFT BLANK, CODE AS M.


Work Environment
ALL
B2. 	The next questions are about the work environment for delivering [TANF 	PROGRAM/PILOT NAME]. 
Please select whether you strongly disagree, disagree, agree, or strongly agree with each of the following statements. 
Select one per row

STRONGLY DISAGREE
DISAGREE
AGREE
STRONGLY AGREE
a.	I feel well-informed about decisions and changes in [TANF PROGRAM/PILOT NAME] that affect my work.
1 m
2 m
3 m
4 m
b.	I am encouraged to try new things and learn from mistakes.
1 m
2 m
3 m
4 m
c.	I am encouraged to bring up issues or contribute ideas to help improve [TANF PROGRAM/PILOT NAME].
1 m
2 m
3 m
4 m
d.	I am motivated to apply changes that can improve practice.
1 m
2 m
3 m
4 m
e.	Administrative tasks (e.g., scheduling, standardized processes, paperwork) for [TANF PROGRAM/PILOT NAME] help me do my job.
1 m
2 m
3 m
4 m
f.	The staff I work most closely with on [TANF PROGRAM/PILOT NAME] are open to change and trying new things that could work.
1 m
2 m
3 m
4 m

PROGRAMMER NOTE

IF ANY B2a-f IS LEFT BLANK, CODE AS M.


Staff Contributions and Preparation

ALL
B3. 	These questions ask about the preparation and support you receive to work with [TANF PROGRAM/PILOT NAME] participants and your contributions to [TANF PROGRAM/PILOT NAME] in your agency or organization.
Please select whether you strongly disagree, disagree, agree, or strongly agree with each of the following statements about your work.
Select one per row

STRONGLY DISAGREE
DISAGREE
AGREE
STRONGLY AGREE
a.	I feel confident that I can handle the challenges that come up in working with [TANF PROGRAM/PILOT NAME] participants.
1 m
2 m
3 m
4 m
b.	I receive regular one-on-one supervision to help me support [TANF PROGRAM/PILOT NAME] participants.
1 m
2 m
3 m
4 m
c.	I participate in training to prepare me for new or improved tasks I am expected to do in [TANF PROGRAM/PILOT NAME].
1 m
2 m
3 m
4 m
d.	Ongoing support beyond training is available to help me build my skills for working with [TANF PROGRAM/PILOT NAME] participants.
1 m
2 m
3 m
4 m
e.	Staff who work with [TANF PROGRAM/PILOT NAME] participants have been working in the program for a long time.
1 m
2 m
3 m
4 m
f.	I believe that my work makes an important contribution to the success of [TANF PROGRAM/PILOT NAME].
1 m
2 m
3 m
4 m

PROGRAMMER NOTE

IF ANY B3a-f IS LEFT BLANK, CODE AS M.


Service Delivery
ALL
B4. 	The next questions ask about things that can affect what you do to deliver services to [TANF PROGRAM/PILOT NAME] participants in your agency or organization.
	Please select whether following statements are not at all true, somewhat true, mostly true, or consistently true.
Select one per row

NOT AT ALL TRUE
SOMEWHAT TRUE
MOSTLY TRUE
CONSISTENTLY TRUE
a.	I am limited in what I can do for participants because there are gaps in what [TANF PROGRAM/PILOT NAME] can do or provide.
1 m
2 m
3 m
4 m
b.	I have enough flexibility within the [TANF PROGRAM/PILOT NAME] program requirements to meet participant needs.
1 m
2 m
3 m
4 m
c.	The process is easy for me to refer [TANF PROGRAM/PILOT NAME] participants to services that the program does not provide directly.
1 m
2 m
3 m
4 m
d.	I can manage my tasks and workload to support the [TANF PROGRAM/PILOT NAME] participants I work with.
1 m
2 m
3 m
4 m
e.	Changes we make in [TANF PROGRAM/PILOT NAME] improve my ability to meet participant needs.
1 m
2 m
3 m
4 m
f.	I know when [TANF PROGRAM/PILOT NAME] participants get connected to the services they need from the referrals I make.
1 m
2 m
3 m
4 m

PROGRAMMER NOTE

IF ANY B4a-f IS LEFT BLANK, CODE AS M.


Data and Accountability
ALL
B5. 	The next questions are about how you use information and data and the processes that are in place to support accountability in [TANF PROGRAM/PILOT NAME] in your agency or organization.
	Please select whether you strongly disagree, disagree, agree, or strongly agree with each of the following statements. 
Select one per row

STRONGLY DISAGREE
DISAGREE
AGREE
STRONGLY AGREE
a.	I use information and data to help me improve the quality of my work.
1 m
2 m
3 m
4 m
b.	I understand the information I need to collect and record about [TANF PROGRAM/PILOT NAME] participant engagement and how to do it.
1 m
2 m
3 m
4 m
c.	I regularly try to reengage participants who do not follow-through with agreed on activities.
1 m
2 m
3 m
4 m
d.	I understand when I need to impose sanctions on participants who do not comply with program rules or participation requirements.
1 m
2 m
3 m
4 m
e.	The processes we have in place help me track the progress of [TANF PROGRAM/PILOT NAME] participants who I work with.
1 m
2 m
3 m
4 m

PROGRAMMER NOTE

IF ANY B5a-e IS LEFT BLANK, CODE AS M.


C. Program consistency
ALL
C1. 	The next questions ask about how the tasks you do in [TANF PROGRAM/PILOT NAME]  may or may not have changed recently.
Would you say the way you do each of the following tasks has had no change, a small change, moderate change, or major change in the past 6 months? (If you do not do a task you can select N/A for not applicable.)
Select one per row

NO CHANGE
SMALL CHANGE
MODERATE CHANGE
MAJOR CHANGE
N/A (NOT A TASK I DO)
a.	Doing outreach, orientation, and case management or coaching with [TANF PROGRAM/PILOT NAME] participants
1 m
2 m
3 m
4 m
n m
b.	Providing or connecting [TANF PROGRAM/PILOT NAME] participants with work, education, and training services
1 m
2 m
3 m
4 m
n m
c.	Providing or connecting [TANF PROGRAM/PILOT NAME] participants with work supports or supportive services
1 m
2 m
3 m
4 m
n m
d.	Providing or connecting [TANF PROGRAM/PILOT NAME] participants with personal or family services
1 m
2 m
3 m
4 m
n m
e.	Performing administrative tasks to support case management and service delivery (e.g., case noting and updating participant files)
1 m
2 m
3 m
4 m
n m
f.	Monitoring and verifying participation in activities
1 m
2 m
3 m
4 m
n m
g.	Collecting information on participant outcomes (e.g., employment, earnings, education, and family stability)
1 m
2 m
3 m
4 m
n m
h.	Holding [TANF PROGRAM/PILOT NAME] participants accountable for following through on planned actions or activities
1 m
2 m
3 m
4 m
n m
i.	Applying sanctions to [TANF PROGRAM/PILOT NAME] participants for noncompliance with program requirements
1 m
2 m
3 m
4 m
n m

PROGRAMMER NOTE

IF ANY C1a-j IS LEFT BLANK, CODE AS M.

D. TANF PROGRAM SERVICES
ALL
D1. 	In this section, we want to learn whether [TANF PROGRAM/PILOT NAME] services are able 	to meet the needs of the participants you support.
Thinking about the type and amount of services available, please select whether you strongly disagree, disagree, agree, or strongly agree that [TANF PROGRAM/PILOT NAME] services meet participant needs directly or through referral in each of the following areas. (If your program does not provide or refer to a service you can select N/A for not applicable.) 
Select one per row

STRONGLY DISAGREE
DISAGREE
AGREE
STRONGLY AGREE
N/A (NOT A SERVICE PROGRAM PROVIDES)
a.	knowing how to look and apply for jobs.
1 m
2 m
3 m
4 m
n m
b.	building skills and experience to get and keep a job.
1 m
2 m
3 m
4 m
n m
c.	having a reliable way to travel to work.
1 m
2 m
3 m
4 m
n m
d.	getting basic needs met such as housing and food.
1 m
2 m
3 m
4 m
n m
e.	arranging reliable child care or handling caregiver responsibilities to be able to work.
1 m
2 m
3 m
4 m
n m
f.	addressing issues related to physical health, mental health, or substance use.
1 m
2 m
3 m
4 m
n m
g.	navigating legal issues that interfere with getting or keeping a job.
1 m
2 m
3 m
4 m
n m
h.	understanding family finances, making a budget, or creating a savings plan.
1 m
2 m
3 m
4 m
n m
i.	getting out of an unsafe home or family environment.
1 m
2 m
3 m
4 m
n m

PROGRAMMER NOTE

IF ANY D1a-j IS LEFT BLANK, CODE AS M.

IF ANY D1a-j = 1 or 2, DISPLAY THAT ROW IN D2
SKIP ANY D1a-j IF D1a-k = 3, N, OR M
D2. 	What limits the ability of [TANF PROGRAM/PILOT NAME] services to meet these participant needs? 
	Please select all that apply: Do the type of services offered not meet participant needs, are there not enough services to meet participant needs, and/or are there problems accessing services that limit the ability of TANF services to meet participant needs with . . .
Select all that apply per row

THE TYPE OF SERVICES OFFERED DO NOT MEET NEEDS
NOT ENOUGH SERVICES TO MEET NEEDS
PROBLEMS ACCESSING SERVICES TO MEET NEEDS
a.	knowing how to look and apply for jobs.
1 □
2 □
3 □
b.	building skills and experience to get and keep a job.
1 □
2 □
3 □
c.	having a reliable way to travel to work.
1 □
2 □
3 □
d.	getting basic needs met such as housing or food.
1 □
2 □
3 □
e.	arranging reliable child care or handling caregiver responsibilities to be able to work.
1 □
2 □
3 □
f.	addressing issues related to physical health, mental health, or substance use.
1 □
2 □
3 □
g.	navigating legal issues that interfere with getting or keeping a job.
1 □
2 □
3 □
h.	understanding family finances, making a budget, or creating a savings plan. 
1 □
2 □
3 □
i.	getting out of an unsafe home or family environment.
1 □
2 □
3 □

PROGRAMMER NOTE

IF ANY D2a-j IS LEFT BLANK, CODE AS M.

E. Participant ENGAGEMENT
ALL
E1. 	The next questions ask about how participants engage with you and [TANF PROGRAM/PILOT NAME] in your agency or organization.
Thinking about participants you serve, please select whether each of the following statements is not at all true, somewhat true, mostly true, or consistently true.
Select one per row

NOT AT ALL TRUE
SOMEWHAT TRUE
MOSTLY TRUE
CONSISTENTLY TRUE
a.	Participants set their own goals for what they want to do in [TANF PROGRAM/PILOT NAME].
1 m
2 m
3 m
4 m
b. 	It is easy to reach participants when I need to.
1 m
2 m
3 m
4 m
c. 	Participants regularly come to scheduled meetings with me or other staff.
1 m
2 m
3 m
4 m
d. 	Participants actively communicate with me or other staff.
1 m
2 m
3 m
4 m
e.	Participants follow through with agreed on action steps.
1 m
2 m
3 m
4 m
f.	[TANF PROGRAM/PILOT NAME] is challenging for participants to navigate and identify what they need to do.
1 m
2 m
3 m
4 m
g.	Participants are held accountable for doing certain tasks or activities to stay in [TANF PROGRAM/PILOT NAME].
1 m
2 m
3 m
4 m
h.	Participants say the activities they do and services they get from [TANF PROGRAM/PILOT NAME] help them.
1 m
2 m
3 m
4 m





PROGRAMMER NOTE

IF ANY E1a-h IS LEFT BLANK, CODE AS M.

ALL
E2. 	The next questions are about the fit between the amount of time you spend on different activities in your primary role with [TANF PROGRAM/PILOT NAME] and the time you need to do certain tasks or activities.
Thinking about the amount of time you spend on different activities, do you think you spend barely any time at all, not enough time, the right amount of time, or too much time doing the following:
Select one per row

BARELY ANY TIME AT ALL
NOT ENOUGH TIME
THE RIGHT AMOUNT OF TIME
TOO MUCH TIME
N/A (DOES NOT APPLY TO MY ROLE)
a.	Working directly with participants to set goals, create action plans, or provide or connect them with services?
1 m
2 m
3 m
4 m
n m
b. 	Tracking what participants do and activities they complete in [TANF PROGRAM/PILOT NAME]?
1 m
2 m
3 m
4 m
n m
c.	Making sure participants comply with [TANF PROGRAM/PILOT NAME] rules or requirements?
1 m
2 m
3 m
4 m
n m
d.  Taking action if participants do not comply (initiating sanctions, assessing good cause, or working through appeals)
1 m
2 m
3 m
4 m
n m
e.	Tracking the progress participants make on their own goals?
1 m
2 m
3 m
4 m
n m
f.	Tracking participant outcomes for [TANF PROGRAM/PILOT NAME] success?
1 m
2 m
3 m
4 m
n m






PROGRAMMER NOTE

IF ANY E2a-e IS LEFT BLANK, CODE AS M.


F. PROGRAM PERFORMANCE
ALL
F1. 	The next questions ask about how [TANF PROGRAM/PILOT NAME] measures its success in your agency or organization. 
	Please select whether you strongly disagree, disagree, agree, or strongly agree with each of the following statements. 
Select one per row

STRONGLY DISAGREE
DISAGREE
AGREE
STRONGLY AGREE
a. 	I understand how success for the [TANF PROGRAM/PILOT NAME] is measured.
1 m
2 m
3 m
4 m
b.	I understand my role in measuring the [TANF PROGRAM/PILOT NAME] success.
1 m
2 m
3 m
4 m
c.	The [TANF PROGRAM/PILOT NAME] is tracking the right things to measure overall program success.
1 m
2 m
3 m
4 m
d.	The [TANF PROGRAM/PILOT NAME] is tracking the right things to know if it is helping participants.
1 m
2 m
3 m
4 m
e.	I am held accountable for contributing to [TANF PROGRAM/PILOT NAME] success.
1 m
2 m
3 m
4 m

PROGRAMMER NOTE

IF ANY F1a-e IS LEFT BLANK, CODE AS M.


ALL
F2. 	These final questions ask about how helpful [TANF PROGRAM/PILOT NAME] is in 	supporting the participants you work 	with make progress.	
	Would you say [TANF PROGRAM/PILOT NAME] is not at all helpful, a little helpful, mostly helpful, or very helpful to participants with…  
Select one per row

NOT AT ALL HELPFUL
A LITTLE HELPFUL
MOSTLY HELPFUL
VERY HELPFUL
a.	getting a job?
1 m
2 m
3 m
4 m
b.	keeping a job?
1 m
2 m
3 m
4 m
c.	earning more money?
1 m
2 m
3 m
4 m
d.	not needing public benefits?
1 m
2 m
3 m
4 m
e.	being more financially independent?
1 m
2 m
3 m
4 m
f.	getting more education and building skills?
1 m
2 m
3 m
4 m
g.	improving their physical and mental health?
1 m
2 m
3 m
4 m
h.	improving family stability?
1 m
2 m
3 m
4 m

PROGRAMMER NOTE

IF ANY F2a-i IS LEFT BLANK, CODE AS M.


G. SUBMIT SURVEY
IF CONSENT = 0

END1. 	Thank you for your time. If you clicked “No” by mistake and would like to participate in the survey, please contact [email protected] or 833-678-3825.

IF CONSENT = 1

END2. 	Thank you for completing this survey!
To exit the survey, please close this tab or your internet browser.