Instrument 2: Leadership survey - Supervisors

Status: Original and derived artifacts are available for this document.

Download: docx | pdf | html

OMB No.: XXXX-XXXX
Expiration Date: DD/MM/YYYY

TANF Pilot Evaluation Leadership Survey

PROGRAMMER NOTES: 
UNIVERSAL SOFT CHECK IF MISSING RESPONSE: Please provide a response to this question, or click next to move to the next question. 


COMPUTER ASSISTED WEB INTERVIEW (CAWI) ALL

How to complete the survey
Thank you for taking the time to complete this survey.
    • There are no right or wrong answers.
    • To answer a question, click the box that indicates your response or fill in your response.
    • To continue to the next webpage, press the "Next" button.
    • To go back to the previous webpage, click the "Back" button. Please note that this command is only available in some sections.
    • Do not use the navigation arrows in your browser.
    • If you need to stop before you have finished, you can close the survey and come back to it at any time. The responses you gave before leaving will be securely stored and available when you return to complete the survey. 

    • Select “Next” to begin the survey. 


INTRODUCTION & CONSENT
Your state is among five states that were selected by the U.S. Department of Health and Human Services (HHS) to participate in pilots to try something different in TANF programs, as authorized by the Fiscal Responsibility Act of 2023 (FRA). Mathematica, The Adjacent Possible, and the American Public Human Services Association are conducting the TANF Pilot Evaluation to understand and learn from these pilots under contract with the Administration for Children and Families at HHS. 
What is this survey about?
We want to learn about your experiences with the [TANF PROGRAM/PILOT NAME]. Your answers will help us understand how the pilot is working from the people who bring it to life every day. This survey is for program leadership, such as administrators, program directors or managers, program specialists, or supervisors of staff who work directly with TANF participants. You may work for [TANF PROGRAM/PILOT NAME]; another government agency, such as [STATE WIOA PROVIDER OR OTHER AGENCY]; or for a contracted partner organization, such as [PARTNER ORGANIZATION NAMES]. Your insights will help us understand what’s working well and where improvements might be made. Your perspective will provide information to help strengthen TANF programs in your state and across the country.
When should I complete the survey?
Please complete the survey within one week.
How long will the survey take?
This survey takes about 20 minutes, though the time to complete this survey will vary by person. Your participation in the survey is completely voluntary. You may choose not to respond at all or to skip any questions. Your individual responses will be kept private and will only be used for research purposes and not shared with anyone outside the study team. We have obtained a Certificate of Confidentiality from the National Institutes of Health. This helps us protect participants’ privacy. This means no one can force the study team to give out information that identifies them, even in court. The certificate does not prevent us from making disclosures required by law, such as threats of harm or abuse. We will combine responses for reporting purposes, and we will never report names or identify any responses with a particular person.     
If you agree to complete this survey, you can decline to participate in any potential future activities related to the study by calling Mathematica toll-free at 833-678-3825 and writing a message that you do not wish to complete those activities. You can send this message to Jennifer Herard-Tsiagbey, the study’s Data Collection Director, at [email protected]. Any information we collect about you before you withdraw from the study will be retained as part of the research. 
If you have questions, please contact Jennifer Herard-Tsiagbey, the study’s Survey Director, at [email protected]. Thank you for your time and input!
PAPERWORK REDUCTION ACT OF 1995 (Public Law 104-13) STATEMENT OF PUBLIC BURDEN: Through this information collection, ACF is gathering information to learn more about measuring program performance in TANF programs. Public reporting burden for this collection of information is estimated to average 20 minutes per respondent, including the time for reviewing instructions, gathering and maintaining the data needed, and reviewing the collection of information. This is a voluntary collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information subject to the requirements of the Paperwork Reduction Act of 1995, unless it displays a currently valid OMB control number. The OMB # is 0970-0XXX and the expiration date is XX/XX/XXXX. If you have any comments on this collection of information, please contact Quinn Moore at [email protected].

ALL
Do you agree to participate in this survey? 
m	YES	1 A1
m	NO	0 END1


A. PROFESSIONAL EXPERIENCE
ALL
A1. 	What is your job title? 
JOB TITLE

(STRING 100)
MISSING	m

ALL
A2. 	Please select the county, jurisdiction, or area of [TANF PROGRAM/PILOT NAME] that you work for from the list below.
	[PREFILLED DROP-DOWN LIST OF JURISDICTIONS SPECIFIC TO EACH STATE]

ALL
A3. 	Which type of organization do you work for?
m	State or local human services agency	1	
m	Other government agency	2	
m	Contracted service provider or partner organization	3
m	Other (specify – string 150)	4	
MISSING	m	

ALL
A4. 	Which of the following best describes your current role? (If you have more than one role, select your primary role or the role you spend most of your time on. If your primary role is related to eligibility, please select a role from the list below that you do most often, aside from eligibility.)
m	Administrator. A person who oversees TANF policies, procedures, or operations across an area, region, or district.	1	
m	TANF program director or manager. A person who is responsible for TANF policies, procedures, or operations within a specific location, county, or office and oversees day-to-day operations for case management, coaching, or employment services for TANF participants.	2	
m	TANF program or policy specialist. A person who is responsible for policy, program, data, or special initiatives.	3	
m	Supervisor. A person who provides supervision and support for frontline staff who work directly with TANF participants to provide case management, coaching, or employment services.	4	

m	Other (specify – string 150) 	5	
MISSING	m	
ALL
A5. 	How many years of experience do you have working on the [TANF PROGRAM] in your state?
m	Less than 1 year	1	
m	1 to 2 years	2 	
m	3 to 5 years	3 	
m	6 to 10 years	4 	
m	11 to 15 years	5 	
m	More than 15 years	6 	
MISSING	m

ALL
A6. 	Including your current position, how many years of total experience do you have doing similar work?(Include time at other employers where you did similar work, even if it was in other states.)
m	Less than 1 year	1	
m	1 to 2 years	2 	
m	3 to 5 years	3 	
m	6 to 10 years	4 	
m	11 to 15 years	5 	
m	More than 15 years	6 	
MISSING	m


B. PROGRAM IMPLEMENTATION
ALL
B1. 	The next questions ask about overall leadership of [TANF PROGRAM/PILOT NAME. Leadership refers to the range of individuals who oversee and manage the program, set policy, and supervise staff across state, local, and contracted service provider organizations. 
	Please select whether each of the following statements are not at all true, somewhat true, mostly true, or consistently true. 
Select one per row

NOT AT ALL TRUE
SOMEWHAT TRUE
MOSTLY TRUE
CONSISTENTLY TRUE
a.	Leaders communicate a clear vision to staff about what [TANF PROGRAM/PILOT NAME] is supposed to do for participants.
1 m
2 m
3 m
4 m
b.	Leaders focus on the issues that really matter for direct service delivery to [TANF PROGRAM/PILOT NAME] participants.
1 m
2 m
3 m
4 m
c.	Leaders explain the reasons for making changes in [TANF PROGRAM/PILOT NAME] to staff and create supports for those changes to happen.
1 m
2 m
3 m
4 m
d.	Leaders of [TANF PROGRAM/PILOT NAME] encourage staff to share ideas, ask questions, and admit mistakes.
1 m
2 m
3 m
4 m
e.	Leaders clearly explain expectations for [TANF PROGRAM/PILOT NAME] service delivery and hold staff accountable for meeting them.
1 m
2 m
3 m
4 m
f.	Leaders of [TANF PROGRAM/PILOT NAME] actively engage with staff to resolve issues that get in the way of making changes happen.
1 m
2 m
3 m
4 m

PROGRAMMER NOTE

IF ANY B1a-f IS LEFT BLANK, CODE AS M.




ALL
B2. 	The next questions are about the work environment for delivering [TANF PROGRAM/PILOT NAME]. 
Please select whether you strongly disagree, disagree, agree, or strongly agree with each of the following statements. 
Select one per row

STRONGLY DISAGREE
DISAGREE
AGREE
STRONGLY AGREE
a.	Staff of [TANF PROGRAM/PILOT NAME] are well-informed about decisions and changes that affect their work.
1 m
2 m
3 m
4 m
b.	Staff of [TANF PROGRAM/PILOT NAME] are encouraged to try new things and learn from mistakes.
1 m
2 m
3 m
4 m
c.	Staff of [TANF PROGRAM/PILOT NAME] are encouraged to bring up issues or contribute ideas to help improve the program.
1 m
2 m
3 m
4 m
d.	Staff of [TANF PROGRAM/PILOT NAME] are motivated to apply changes that improve practice.
1 m
2 m
3 m
4 m
e.	Administrative tasks for [TANF PROGRAM/PILOT NAME] help staff do their jobs.
1 m
2 m
3 m
4 m
f.	Staff of [TANF PROGRAM/PILOT NAME] are open to change and trying new things that could work.
1 m
2 m
3 m
4 m






PROGRAMMER NOTE

IF ANY B2a-f IS LEFT BLANK, CODE AS M.


ALL
B3. 	These questions ask about the preparation and support you provide staff to work with [TANF PROGRAM/PILOT NAME] participants and your contributions to [TANF PROGRAM/PILOT NAME] in your agency or organization. 
Please select whether you strongly disagree, disagree, agree, or strongly agree with each of the following statements. 
Select one per row

STRONGLY DISAGREE
DISAGREE
AGREE
STRONGLY AGREE
a.	The [TANF PROGRAM/PILOT NAME] program hires staff with the skills to handle the challenges that come up in working with participants.
1 m
2 m
3 m
4 m
b.	Leaders provide regular one-on-one supervision to help staff as they support [TANF PROGRAM/PILOT NAME] participants.
1 m
2 m
3 m
4 m
c.	The [TANF PROGRAM/PILOT NAME] program provides training that prepares staff for new or improved tasks they are expected to do.
1 m
2 m
3 m
4 m
d.	Ongoing support beyond training is available to help staff build their skills for working with [TANF PROGRAM/PILOT NAME] participants.
1 m
2 m
3 m
4 m
e.	Staff who work with [TANF PROGRAM/PILOT NAME] participants have been working in the program a long time.
1 m
2 m
3 m
4 m
f.	The [TANF PROGRAM/PILOT NAME] program hires staff that are open to change. 
1 m
2 m
3 m
4 m
g.	The [TANF PROGRAM/PILOT NAME] program is able to retain staff who perform well.
1 m
2 m
3 m
4 m






PROGRAMMER NOTE

IF ANY B3a-g IS LEFT BLANK, CODE AS M.


ALL
B4. 	The next questions ask about things that can affect how staff deliver services to [TANF PROGRAM/PILOT NAME] participants in your agency or organization.
Please select whether following statements are not at all true, somewhat true, mostly true, or consistently true. 
Select one per row

NOT AT ALL TRUE
SOMEWHAT TRUE
MOSTLY TRUE
CONSISTENTLY TRUE
a.	Staff are limited in what they can do for participants because there are gaps in what [TANF PROGRAM/PILOT NAME] can do or provide.
1 m
2 m
3 m
4 m
b.	The [TANF PROGRAM/PILOT NAME] program requirements provide flexibility to meet participant needs.
1 m
2 m
3 m
4 m
c.	The process is easy for staff to refer [TANF PROGRAM/PILOT NAME] participants to services that the program does not provide directly.
1 m
2 m
3 m
4 m
d.	Staff can manage their tasks and workload to support [TANF PROGRAM/PILOT NAME] participants.
1 m
2 m
3 m
4 m
e.	Changes leaders make in [TANF PROGRAM/PILOT NAME] improve staff members’ ability to meet participant needs.
1 m
2 m
3 m
4 m

PROGRAMMER NOTE

IF ANY B4a-e IS LEFT BLANK, CODE AS M.


ALL
B5. 	Leaders use data in different ways. The next questions ask about your use of data.  
How often do you use data in the following ways: never, rarely, sometimes, or consistently? (If the data use does not apply to your position select N/A for not applicable.)
Select one per row

NEVER
RARELY
SOMETIMES
CONSISTENTLY
N/A (DOES NOT APPLY TO MY POSITION)
a. 	I use data to make decisions about use of resources for [TANF PROGRAM/PILOT NAME]. 
1 m
2 m
3 m
4 m
n m
b.	I use data to monitor program quality and make improvements for [TANF PROGRAM/PILOT NAME]. 
1 m
2 m
3 m
4 m
n m
c.	I review data to make staffing decisions for [TANF PROGRAM/PILOT NAME]. 
1 m
2 m
3 m
4 m
n m
d.	I use data to make decisions about services available for [TANF PROGRAM/PILOT NAME] participants.
1 m
2 m
3 m
4 m
n m
e.	I am able to use data to examine changes in [TANF PROGRAM/PILOT NAME] participation or progress over time.
1 m
2 m
3 m
4 m
n m
f.	I am able to compare data to targets set by the [TANF PROGRAM/PILOT NAME] to see if program goals are being met.
1 m
2 m
3 m
4 m
n m

PROGRAMMER NOTE

IF ANY B5a-f IS LEFT BLANK, CODE AS M.


ALL
B6. 	The next questions ask about the data that you have access to for the [TANF PROGRAM/PILOT NAME]. 
Please select whether you strongly disagree, disagree, agree, or strongly agree with each of the following statements.
Select one per row

STRONGLY DISAGREE
DISAGREE
AGREE
STRONGLY AGREE
a.	I am confident that data on participant engagement for [TANF PROGRAM/PILOT NAME] is complete and accurate.
1 m
2 m
3 m
4 m
b.	I am confident that the data for [TANF PROGRAM/PILOT NAME] on participant progress is complete and accurate.
1 m
2 m
3 m
4 m
c.	The [TANF PROGRAM/PILOT NAME] data reports are available in a user-friendly way.
1 m
2 m
3 m
4 m
d.	I have access to data that I need in the state or local management information system for [TANF PROGRAM/PILOT NAME].
1 m
2 m
3 m
4 m

PROGRAMMER NOTE

IF ANY B6a-d IS LEFT BLANK, CODE AS M.


C. PROGRAM CONSISTENCY
ALL
C1. 	The next questions ask about how the [TANF PROGRAM/PILOT NAME] program in your agency or organization may or may not have changed recently. 
Would you say each of the following has had no change, a small change, moderate change, or major change in the past 6 months?
Select one per row

 NO CHANGE
SMALL CHANGE
MODERATE CHANGE
MAJOR CHANGE
a.	The activities that participants can engage in or the services they can receive
1 m
2 m
3 m
4 m
b.	The requirements that participants must meet to continue receiving benefits
1 m
2 m
3 m
4 m
c.	The length of time that participants are able to receive benefits
1 m
2 m
3 m
4 m
d.	The extent that the [TANF PROGRAM/PILOT NAME] coordinates with other public benefit programs
1 m
2 m
3 m
4 m
e.	The workload of staff for the [TANF PROGRAM/PILOT NAME] 
1 m
2 m
3 m
4 m

PROGRAMMER NOTE

IF ANY C1a-f IS LEFT BLANK, CODE AS M.


ALL
C2. 	The next questions ask about how tasks are done for [TANF PROGRAM/PILOT NAME] in your agency or organization may or may not have changed recently. 
Would you say these things have had no change, a small change, moderate change, or major change in the past 6 months?
Select one per row

 NO CHANGE
SMALL CHANGE
MODERATE CHANGE
MAJOR CHANGE
a.	Collecting and monitoring metrics on case management and service delivery (e.g., caseloads, work flow, use of services, etc.) for [TANF PROGRAM/PILOT NAME]
1 m
2 m
3 m
4 m
b.	Monitoring and verifying participant participation in activities for [TANF PROGRAM/PILOT NAME]
1 m
2 m
3 m
4 m
c.	Collecting and monitoring information on participant outcomes (e.g., employment, earnings, education, and family stability) for [TANF PROGRAM/PILOT NAME]
1 m
2 m
3 m
4 m
d.	Applying sanctions to [TANF PROGRAM/PILOT NAME] participants for noncompliance with program requirements
1 m
2 m
3 m
4 m

PROGRAMMER NOTE

IF ANY C2a-d IS LEFT BLANK, CODE AS M.


ALL
C3. 	The next questions ask about the amount of time you spend on different activities in your position with the [TANF PROGRAM/PILOT NAME]. 
Do you spend barely any time at all, not enough time, the right amount of time, or too much time doing the following? (If the activity does not apply to your position select N/A for not applicable.)
Select one per row

BARELY ANY TIME AT ALL
NOT ENOUGH TIME
THE RIGHT AMOUNT OF TIME
TOO MUCH TIME
N/A (DOES NOT APPLY TO MY POSITION)
a. 	Supporting direct service delivery
1 m
2 m
3 m
4 m
n m
b.	Monitoring and using information on participant participation or engagement
1 m
2 m
3 m
4 m
n m
c.	Monitoring and using information on participant outcomes (e.g., employment, earnings, education, and family stability)
1 m
2 m
3 m
4 m
n m
d. 	Supporting and assessing staff performance (includes training you provide and receive)
1 m
2 m
3 m
4 m
n m

PROGRAMMER NOTE

IF ANY C3a-d IS LEFT BLANK, CODE AS M.


ALL
C4. 	The next questions ask about how consistent staff are in what they do for participants in the [TANF PROGRAM/PILOT NAME] program. 
	Please select whether you strongly disagree, disagree, agree, or strongly agree with each of the following statements. 
Select one per row

STRONGLY DISAGREE
DISAGREE
AGREE
STRONGLY AGREE
a.	Staff across our program take a consistent approach to tracking participation hours and getting documentation.
1 m
2 m
3 m
4 m
b.	Staff across our program take a consistent approach to helping participants find the services that are right for them.
1 m
2 m
3 m
4 m
c.	Staff consistently follow through on commitments with participants in a timely manner.
1 m
2 m
3 m
4 m
d.	Staff are consistent in when and how they apply sanctions to [TANF PROGRAM/PILOT NAME] participants for noncompliance with program requirements.
1 m
2 m
3 m
4 m

PROGRAMMER NOTE

IF ANY C4a-d IS LEFT BLANK, CODE AS M.


D. TANF PROGRAM SERVICES, PARTICIPANT ENGAGEMENT, AND PERFORMANCE
ALL
D1. 	The next questions ask about how [TANF PROGRAM/PILOT NAME] measures its success in your agency or organization. 
	Please select whether you strongly disagree, disagree, agree, or strongly agree with each of the following statements. 
Select one per row

STRONGLY DISAGREE
DISAGREE
AGREE
STRONGLY AGREE
DON”T KNOW
a. 	Staff understand how success is measured for the [TANF PROGRAM/PILOT NAME].
1 m
2 m
3 m
4 m

b. 	Staff understand their role in measuring [TANF PROGRAM/PILOT NAME] success.
1 m
2 m
3 m
4 m

c. 	The [TANF PROGRAM/PILOT NAME] is tracking the right things to measure overall program success.
1 m
2 m
3 m
4 m

d.	The [TANF PROGRAM/PILOT NAME] is tracking the right things to know if it is helping participants.
1 m
2 m
3 m
4 m

e.	Staff are held accountable for contributing to [TANF PROGRAM/PILOT NAME] success.
1 m
2 m
3 m
4 m

f.	Work requirements are a priority in public benefit programs in our state.
1 m
2 m
3 m
4 m

g.	The [TANF PROGRAM/PILOT NAME] priorities align with other public benefit programs such as SNAP or Medicaid.
1 m
2 m
3 m
4 m
0 m
h.	Pressing priorities in other programs such as SNAP or Medicaid influence how we are able to operate the [TANF PROGRAM/PILOT NAME].
1 m
2 m
3 m
4 m
0 m

PROGRAMMER NOTE

IF ANY D1a-h IS LEFT BLANK, CODE AS M.

ALL
D2. 	These final questions ask about how helpful [TANF PROGRAM/PILOT NAME] is in supporting the participants make progress. 
	Would you say [TANF PROGRAM/PILOT NAME] is not at all helpful, a little helpful, mostly helpful, or very helpful to participants with… 
Select one per row

NOT AT ALL HELPFUL
A LITTLE HELPFUL
MOSTLY HELPFUL
VERY HELPFUL

a.	getting a job?
1 m
2 m
3 m
4 m

b.	keeping a job?
1 m
2 m
3 m
4 m

c.	earning more money?
1 m
2 m
3 m
4 m

d.	not needing public benefits?
1 m
2 m
3 m
4 m

e.	being more financially independent?
1 m
2 m
3 m
4 m

f.	getting more education and building skills?
1 m
2 m
3 m
4 m

g.	improving their physical and mental health?
1 m
2 m
3 m
4 m

h.	improving family stability?
1 m
2 m
3 m
4 m

PROGRAMMER NOTE

IF ANY D2a-i IS LEFT BLANK, CODE AS M.


E. SUBMIT SURVEY
IF CONSENT = 0

END1. 	Thank you for your time. If you clicked “No” by mistake and would like to participate in the survey, please contact [email protected] or 833-678-3825.

IF CONSENT = 1

END2. 	Thank you for completing this survey!
	To exit the survey, please close this tab or your internet browser.